Create a Standardized Nutrient Data Harmonization and Data Retrieval Workflow

[wokenny13]

This PR includes a draft Nutrient Data vignette workflow that different stakeholders can consider with using TADA for harmonizing nutrient data for TN and TP or for general nutrient data discovery. The vignette looks to summarize and capture the different use cases that many folks have expressed interest in with TN and TP harmonization and summation with a use case/case study sections in it.

Builds on the package’s existing data processing methods and focuses on how other stakeholders can use TADA as a standardized method for handling TN and TP summation with WQP data. As part of the review and modification to this vignette, we should determine whether the harmonization template should be modified, and what other items need to be captured for a consistent harmonization workflow. Feedback from stakeholders will help confirm the approach and identify any needed changes.

1. review the harmonization method, any updates to the harmonization table needed?
2. review the TN and TP summation method. Any modification needed?
3. consider how other stakeholders are performing this and compare.

Pull Request Checklist (convert PR to draft if in progress)

Required

• Update your branch from the latest develop and resolve any merge conflicts

• Run devtools::test(), devtools::check(), and devtools::document() locally; ensure tests pass and fix any errors, warnings, or notes. Add new dependencies to DESCRIPTION and document appropriately

• Add/update vignettes for corresponding changes in functionality, list these under articles in _pkgdown.yml, and ensure added/updated vignettes run and build with proper formatting locally

• Request review from at least one developer team member (convert PR to ready for review if it was designated as in progress)

Best practices

• Include a summary of the changes made and relevant context/motivation

• Link issues to auto-close on merge (use Development sidebar or include "Closes #" in the PR)

• Refresh inline/block comments for clarity

• Update roxygen docs and include examples; review help pages

• Add/update tests in tests/testthat; review the bot's coverage report from test-coverage and confirm all changes are covered

Conditional

• If there is a bot spelling comment, run spelling::spell_check_package() locally and fix any misspellings; add approved project terms to WORDLIST with spelling::update_wordlist()

• If tests fail suggesting internal reference files need a refresh, run .TADA_UpdateRefFiles() and .TADA_UpdateExampleData() locally via MaintenanceScheduled.R or trigger the Component File Update GitHub Action

• If new example data files were added, document them in ExampleData.R and include them in MaintenanceScheduled.R for regular refresh

• If columns were added/updated, update RequiredCols.R

• If changes affect other package or the shiny app functions, update those impacted functions accordingly

[github-actions]

Spelling check failed. Details:

Spelling check failed. Found 8 potential misspelling(s). Note: Run spelling::spell_check_package() locally and fix any misspellings; add approved project terms to WORDLIST with spelling::update_wordlist().

  WORD           FOUND IN
dup            NutrientDataWorkflow.Rmd:324
HTF            NutrientDataWorkflow.Rmd:110,256,375,399
Hypoxia        NutrientDataWorkflow.Rmd:110,373
ICIS           NutrientDataWorkflow.Rmd:112
NNI            NutrientDataWorkflow.Rmd:108
reproducibly   NutrientDataWorkflow.Rmd:88
submitters     NutrientDataWorkflow.Rmd:95
WQI            NutrientDataWorkflow.Rmd:95

[github-actions]

coverage-report

File	Coverage		Missing
All files	67%	✅

Minimum allowed coverage is 20%

Generated by 🐒 cobertura-action against ad4d7cb

[hillarymarler]

Are you wanting a review on this now @wokenny13? Or should I wait until you move it out of draft status?

[wokenny13]

Are you wanting a review on this now @wokenny13? Or should I wait until you move it out of draft status?

I opened this PR so that we can keep this as a priority item for us to focus on as a group to review. We will use this PR as a space to start comparing methods with these other projects.

I welcome anyone to make changes to this PR and provide their thoughts now. I left it in the draft status as we are likely going to continue making changes to this vignette.

[hillarymarler]

I am currently reviewing and adding some suggested draft text for the later sections of the workflow.

[github-actions]

Spelling check failed. Details:

Spelling check failed. Found 11 potential misspelling(s). Note: Run spelling::spell_check_package() locally and fix any misspellings; add approved project terms to WORDLIST with spelling::update_wordlist().

  WORD                     FOUND IN
characteristicGroup      NutrientDataWorkflow.Rmd:195
characteristicNamesand   NutrientDataWorkflow.Rmd:195
DataRetrieva             NutrientDataWorkflow.Rmd:182
dup                      NutrientDataWorkflow.Rmd:370
HTF                      NutrientDataWorkflow.Rmd:109,255,421,445
Hypoxia                  NutrientDataWorkflow.Rmd:109,419
ICIS                     NutrientDataWorkflow.Rmd:111
NNI                      NutrientDataWorkflow.Rmd:107
Nowe                     NutrientDataWorkflow.Rmd:312
reproducibly             NutrientDataWorkflow.Rmd:88
submitters               NutrientDataWorkflow.Rmd:95,275

[github-actions]

Spelling check failed. Details:

Spelling check failed. Found 11 potential misspelling(s). Note: Run spelling::spell_check_package() locally and fix any misspellings; add approved project terms to WORDLIST with spelling::update_wordlist().

  WORD                     FOUND IN
characteristicGroup      NutrientDataWorkflow.Rmd:195
characteristicNamesand   NutrientDataWorkflow.Rmd:195
DataRetrieva             NutrientDataWorkflow.Rmd:182
dup                      NutrientDataWorkflow.Rmd:372
HTF                      NutrientDataWorkflow.Rmd:109,255,423,447
Hypoxia                  NutrientDataWorkflow.Rmd:109,421
ICIS                     NutrientDataWorkflow.Rmd:111
NNI                      NutrientDataWorkflow.Rmd:107
Nowe                     NutrientDataWorkflow.Rmd:312
reproducibly             NutrientDataWorkflow.Rmd:88
submitters               NutrientDataWorkflow.Rmd:95,275

[hillarymarler]

I think we should mimic the changes in the order of the TADA mod 1 shiny app workflow and move censored data handling and harmonization (in that order) prior to running key flagging functions. Otherwise, results that would otherwise be he handled by censored data and harmonization functions will be filtered out by the flagging functions before the workflow gets to censored/harmonized steps.

I move those around in the vignette now.

[github-actions]

Spelling check failed. Details:

Spelling check failed. Found 12 potential misspelling(s). Note: Run spelling::spell_check_package() locally and fix any misspellings; add approved project terms to WORDLIST with spelling::update_wordlist().

  WORD                     FOUND IN
characteristicGroup      NutrientDataWorkflow.Rmd:195
characteristicNamesand   NutrientDataWorkflow.Rmd:195
DataRetrieva             NutrientDataWorkflow.Rmd:182
dup                      NutrientDataWorkflow.Rmd:521
HTF                      NutrientDataWorkflow.Rmd:109,255,548,572
Hypoxia                  NutrientDataWorkflow.Rmd:109,546
ICIS                     NutrientDataWorkflow.Rmd:111
IDCensored               NutrientDataWorkflow.Rmd:385
NNI                      NutrientDataWorkflow.Rmd:107
Nowe                     NutrientDataWorkflow.Rmd:312
reproducibly             NutrientDataWorkflow.Rmd:88
submitters               NutrientDataWorkflow.Rmd:95,275

[wokenny13]

I think we should mimic the changes in the order of the TADA mod 1 shiny app workflow and move censored data handling and harmonization (in that order) prior to running key flagging functions. Otherwise, results that would otherwise be he handled by censored data and harmonization functions will be filtered out by the flagging functions before the workflow gets to censored/harmonized steps.

I move those around in the vignette now.

I agree, thank you making those changes!