Skip to content

Add github workflow to validate samples#104

Merged
e-lo merged 14 commits into
TIDES-transit:mainfrom
e-lo:add-validation-workflow
Jul 13, 2023
Merged

Add github workflow to validate samples#104
e-lo merged 14 commits into
TIDES-transit:mainfrom
e-lo:add-validation-workflow

Conversation

@e-lo

@e-lo e-lo commented Feb 1, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

This PR adds a GitHub workflow for sample validation and adds a script to locally validate schemas and samples as well as lint.

Also:

  • linting
  • added requirements for jsonschema
  • updated documentation

@e-lo
Add github workflow to validate samples
ebabddc 
Uses frictionlessdata/repository - which doesn't allow flag `--schema-sync` so will likely need to modify
@e-lo e-lo added 🧹 chore 💻 code Pertains to the infrastructure code labels Feb 1, 2023
@e-lo e-lo requested a review from botanize February 1, 2023 17:40
@e-lo e-lo self-assigned this Feb 1, 2023
@e-lo e-lo mentioned this pull request Feb 1, 2023
Closed
@e-lo

e-lo commented Feb 1, 2023

Copy link
Copy Markdown
Contributor Author

While it doesn't seem like there is the ability to add --schema-sync flag to the frictionless repository action at this time, they have created an issue for it.

You can at this time add an inquiry to the repository action, which in turn can skip various errors such as extra-label. That said, I don't see a way to make sure that optional fields in the schema aren't flagged...

botanize
botanize reviewed Feb 1, 2023
View reviewed changes

@botanize botanize left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, glad they're looking into it. I don't think we should merge this workflow until there's a mechanism to specify --schema-sync, otherwise it will just produce huge reports (1 line per row of the csv!) full of spurious errors.

The alternative is to run frictionless validate --schema-sync directly in the workflow instead of using the frictionless repository workflow.

@e-lo e-lo requested review from botanize and gabriel-korbato July 11, 2023 22:36
botanize
botanize requested changes Jul 12, 2023
View reviewed changes

@botanize botanize left a comment •
edited by e-lo
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi! Overall this is good, but a few specific issues that need to be dealt with.

  • move away from deprecated set-output command
  • deal with dependencies for test script
  • fix jsonschema-cli validation step

See other comments for suggestions.

Joey

Comment thread .github/workflows/validate_samples.yml Outdated
Comment thread .github/workflows/validate_samples.yml Outdated
Comment thread .github/workflows/validate_samples.yml Outdated
Comment thread .github/workflows/validate_samples.yml Outdated
Comment thread .github/workflows/validate_samples.yml
Comment thread test Outdated
Comment thread test Outdated
e-lo added 3 commits July 12, 2023 10:51
@e-lo
eliminate noisy print stmt in main.py
ee00bcc
@e-lo
Move tests to separate scripts in /tests dir
c2cc5a4 
Script will find all executables in /tests and run them.

validate_profile script now downloads the json-schema for profiles from URL

Also:
- update docs
- fix script
@e-lo
add recommendation to use a virtual environment and include links
3321da5
botanize
botanize requested changes Jul 12, 2023
View reviewed changes

@botanize botanize left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some lingering errors in tests

Comment thread tests/validate_samples Outdated
Comment thread tests/test_all Outdated
Comment thread tests/test_all Outdated
e-lo added 3 commits July 12, 2023 11:54
@e-lo
Addressed PR review comments:
77e4291 
set-output --> $GITHUB_ENV
external placeholder.com --> octocons
+ schema-sync
@e-lo
Fix glob search in validate_samples
1fa7ca2 
+ nullglob
** --> *
@e-lo
Fix test progress tracking
778c2aa
@e-lo

e-lo commented Jul 12, 2023

Copy link
Copy Markdown
Contributor Author

@botanize I think i got all your comments addressed in subsequent commits minus the one suggesting that only do validation on the files that actually changed (we can address that when we have files to do).

@e-lo e-lo requested a review from botanize July 12, 2023 19:17
botanize
botanize approved these changes Jul 12, 2023
View reviewed changes

@botanize botanize left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd still like to test this on actual sample data, but everything else seems to work, so let's merge.

@e-lo e-lo merged commit 9624947 into TIDES-transit:main Jul 13, 2023
@e-lo e-lo deleted the add-validation-workflow branch July 13, 2023 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@botanize botanize botanize approved these changes
@gabriel-korbato gabriel-korbato Awaiting requested review from gabriel-korbato

Assignees

@e-lo e-lo

@SorenSpicknall SorenSpicknall

Labels

🧹 chore 💻 code Pertains to the infrastructure code

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@e-lo @botanize @SorenSpicknall