Added workaround for pass-through & small clean up

[RmSchaffert]

• Added workaround for issue where data passed through the DALI pipeline without changes may become corrupted in some cases
  • Implemented: Copy can be added to selected outputs (set up when creating the pipeline)
  • Adjusted documentation & tests accordingly
• Clean-up
  • Added support for assigning uint8 data to SampleDataGroup string data fields inside the pipeline; Note that:
    • This is the type used for strings inside the pipeline
    • Assigning to strings inside the pipeline was previously rejected by type checks
  • Renamed a few identifiers & files to improve clarity & added (internal) documentation to some test helpers

Type of Change

Please select (at least one):

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation / examples / tutorials / demos
• Supporting functionality change (fix or feature in documentation generation, helper scripts, ...)
• Refactoring / internal change
• Other (please describe):

Testing

Checklist for testing:

• Tests added or updated if/as needed
• Repository test runner executed: scripts/run_tests.sh

Optionally, add a brief description.

Documentation, Examples, Tutorials, Demos

Checklist for documentation:

• User-facing documentation updated if/as needed (including API docs)
• Examples / tutorials / demos updated or added (if relevant)
• Limitations and constraints documented (if relevant)
• Performance documented (if relevant)
• Documentation building successful & checks outlined in the Documentation Checks section of the Contribution Guide are performed

Optionally, add a brief description.

Code Quality

Checklist for dependencies:

• Dependencies updated in the relevant pyproject.toml if/as needed
• Code formatted according to the Code Formatting Guide

Optionally, add a brief description.

Related Issues / Context

If applicable, link related issues, discussions etc.

---

DCO / Sign-Off

Please refer to the section on Signing Your Work & Developer Certificate of Origin (DCO)
in the Contribution Guide before submitting your contribution.

References

For additional details, please refer to the Contribution Guide.
The following guides are available (referenced in the Contribution Guide for further details):

• docs/guides/CONTRIBUTION_GUIDE.md
• docs/guides/DEVELOPMENT_GUIDE.md
• docs/guides/DOCUMENTATION_SETUP_GUIDE.md
• docs/guides/FORMATTING_GUIDE.md

Please also refer to the summary checklist in the Contribution Guide,
which is a guideline for what to consider when submitting your contribution and covers the same topics as the checklists above.

[RmSchaffert]

/build

[RmSchaffert]

/build

[RmSchaffert]

/build

[RmSchaffert]

/build

[RmSchaffert]

/build

[xupinjie]

Should we set the default value to True?

It looks like some versions of DALI have a bug that can cause this issue. Would it be possible to automatically adjust this default value based on the DALI version check, instead of simply setting it to false by default? Otherwise, users who are not familiar with the project may encounter abnormal numerical results the first time they use it.

[xupinjie]

For example,

import nvidia.dali as dali
from packaging.version import Version

_DALI_VERSION = Version(dali.__version__)
_DALI_PASSTHROUGH_BUG_FIXED = _DALI_VERSION >= Version("X.Y.Z")

[RmSchaffert]

Good point! Currently, the issue is still present in the newest DALI version. Once it is resolved, we can add the version-based default.

For now, I adjusted the default behavior: If this parameter is not set, all outputs are copied.

However, I would also like to avoid unnecessary overhead. For that, I added a differentiation between setting copy_external_source_passthrough_outputs to True intentionally and leaving it at the default (None). If left at the default, a warning is shown about potential unneeded overhead, and setting any other related parameter (passthrough_copy_field_names etc.) raises an error.

[RmSchaffert]

/build

[RmSchaffert]

/build

[RmSchaffert]

/build

[RmSchaffert]

/build

[RmSchaffert]

/build

[RmSchaffert]

/build