Skip to content

fix(sources): drop empty reader_options before submit (#1648)#1705

Open
SarthakB11 wants to merge 1 commit into
DalgoT4D:mainfrom
SarthakB11:fix/1648-reader-options-empty-string
Open

fix(sources): drop empty reader_options before submit (#1648)#1705
SarthakB11 wants to merge 1 commit into
DalgoT4D:mainfrom
SarthakB11:fix/1648-reader-options-empty-string

Conversation

@SarthakB11

@SarthakB11 SarthakB11 commented Apr 11, 2026

Copy link
Copy Markdown

Fixes #1648.

Symptom

When a user adds an Excel (or other File-type) source in Dalgo without typing anything into the `reader_options` field, form submission fails with:

Error: Failed to check the source connection: Field 'reader_options' is not valid JSON object. https://www.json.org/

The documented workaround on the issue is to manually type `{}` into the field.

Cause

Airbyte's File source declares `reader_options` as a JSON-encoded string field with no default. react-hook-form serializes an untouched input as `''`, which the Dalgo backend then tries to parse as JSON and rejects.

Fix

Extend `cleanConfig()` in `src/components/Sources/SourceForm.tsx` with the same strip-on-empty pattern already used for `start_date`: if `reader_options` comes through as an empty string, delete it from the payload before sending, so the backend falls back to its own default (equivalent to `{}`).

```diff

  • // Remove reader_options if it's an empty string.
  • // In Airbyte's File/Excel source, reader_options is a JSON-encoded string
  • // field; the backend rejects "" with "Field reader_options is not valid
  • // JSON object" (see issue Issue in adding excel sheet as a source in dalgo #1648). Dropping the empty value lets the
  • // backend fall back to its default (equivalent to {}).
  • if (cleaned.reader_options === '') {
  •  delete cleaned.reader_options;
    
  • }
    ```

`cleanConfig` is called on both the check-connection WebSocket payload (`onSubmit`) and the create/update HTTP payload (`handleSaveSource`), so both paths are covered.

Scope

  • Single file, 10 inserted lines.
  • No tests added — the existing `Sources/tests/` tests are form-level integration tests rather than unit tests on `cleanConfig`, and I wanted to keep this PR focused on the minimal fix. Happy to add one if you'd like (e.g., extending `CreateSourceForm.test.tsx` with a case that asserts `reader_options: ''` is absent from the submitted payload).
  • Doesn't address other JSON-as-string fields on other Airbyte connectors; only the one reported in Issue in adding excel sheet as a source in dalgo #1648. Can file a follow-up issue for a more general solution if helpful.

Verification

  • `npx prettier --check src/components/Sources/SourceForm.tsx` — passes.
  • Code change is a literal string comparison plus an object-property delete, matching the existing `start_date` pattern two lines above.

Summary by CodeRabbit

  • Bug Fixes
    • Empty reader options are now excluded from saved and submitted source configurations.
    • Empty, null, or undefined start dates continue to be omitted.
    • Existing source settings remain unchanged when valid values are provided.

@vercel

vercel Bot commented Apr 11, 2026

Copy link
Copy Markdown

@SarthakB11 is attempting to deploy a commit to the support-9965's projects Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai

coderabbitai Bot commented Apr 11, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8c7413b0-ee75-40c6-8ec2-376a1602a8ae

📥 Commits

Reviewing files that changed from the base of the PR and between c60ab5a and b3d0b6f.

📒 Files selected for processing (2)
  • src/components/Sources/SourceForm.tsx
  • src/components/Sources/__tests__/cleanConfig.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/components/Sources/tests/cleanConfig.test.ts
  • src/components/Sources/SourceForm.tsx

Walkthrough

The source form now exports a pure cleanConfig function that removes invalid empty configuration values, including empty reader_options. A Jest suite verifies cleanup, preservation, immutability, and passthrough behavior.

Changes

Source configuration cleanup

Layer / File(s) Summary
Export and apply configuration cleanup
src/components/Sources/SourceForm.tsx
Exports cleanConfig, removes empty or nullish start_date and empty-string reader_options, and removes the former component-local helper.
Validate cleanup behavior
src/components/Sources/__tests__/cleanConfig.test.ts
Tests field removal and preservation, input immutability, and passthrough of unrelated fields.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and clearly describes the main fix: dropping empty reader_options before submission.
Linked Issues check ✅ Passed The change omits empty reader_options, directly addressing the JSON validation error in issue #1648.
Out of Scope Changes check ✅ Passed The refactor and added tests support the fix and do not introduce unrelated functionality.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/components/Sources/SourceForm.tsx (1)

286-309: Add a small regression test for cleanConfig behavior.

Given this was a production bug, please add a focused test that asserts reader_options: '' is removed while valid non-empty values remain unchanged.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/components/Sources/SourceForm.tsx` around lines 286 - 309, Add a focused
unit test for the cleanConfig helper to prevent regressions: import cleanConfig
from SourceForm, then assert that when called with { reader_options: '' } the
returned object does not have reader_options, while { reader_options:
'{"foo":1}' } is preserved unchanged; also include assertions that start_date
values of '', null, and undefined are removed but valid start_date strings
remain. Use the project's Jest/React Testing setup and give the test a clear
name like "cleanConfig removes empty reader_options and empty start_date but
preserves valid values".
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@src/components/Sources/SourceForm.tsx`:
- Around line 286-309: Add a focused unit test for the cleanConfig helper to
prevent regressions: import cleanConfig from SourceForm, then assert that when
called with { reader_options: '' } the returned object does not have
reader_options, while { reader_options: '{"foo":1}' } is preserved unchanged;
also include assertions that start_date values of '', null, and undefined are
removed but valid start_date strings remain. Use the project's Jest/React
Testing setup and give the test a clear name like "cleanConfig removes empty
reader_options and empty start_date but preserves valid values".

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 568fe147-b43b-4b11-bfe9-e689c1b3031c

📥 Commits

Reviewing files that changed from the base of the PR and between cfce3aa and ec33244.

📒 Files selected for processing (1)
  • src/components/Sources/SourceForm.tsx

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
src/components/Sources/SourceForm.tsx (1)

55-56: Harden empty-check for whitespace-only reader_options.

Consider treating trimmed-empty strings as empty too, to avoid accidental " " submissions causing the same backend rejection.

♻️ Suggested patch
-  if (cleaned.reader_options === '') {
+  if (
+    typeof cleaned.reader_options === 'string' &&
+    cleaned.reader_options.trim() === ''
+  ) {
     delete cleaned.reader_options;
   }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/components/Sources/SourceForm.tsx` around lines 55 - 56, The empty-check
for reader_options should treat whitespace-only strings as empty; update the
condition around cleaned.reader_options in SourceForm.tsx (the block that
currently reads "if (cleaned.reader_options === '') { delete
cleaned.reader_options; }") to first confirm it's a string and then check
trimmed value (e.g., use cleaned.reader_options.trim() === '') so
whitespace-only values are deleted as well; also ensure you guard against
null/undefined before calling trim.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/ISSUE_TEMPLATE/bug_report.md:
- Around line 34-37: Normalize example values for the template fields "Device",
"OS", "Browser", and "Version" to use spaced, more readable forms (e.g., change
"iPhone6" to "iPhone 6", "iOS8.1" to "iOS 8.1", and "safari" to "Safari") so the
examples are consistent and clearer; update the example text for the "Device",
"OS", "Browser", and "Version" lines accordingly and keep capitalization
consistent.

In `@README.md`:
- Line 60: Update the README sentence that currently reads "The env varibales
will be picked up from the .env file during the yarn build stage. This will copy
all the env variables starting with NEXT*PUBLIC* (that get embedd into the
javascript code). The env varibales starting without NEXT*PUBLIC* will be picked
up during run time." by correcting the three typos: change both occurrences of
"varibales" to "variables" and change "embedd" to "embedded" so the sentence
uses "variables" and "embedded" consistently.

In `@src/components/Notifications/Notifications.module.css`:
- Around line 11-12: In Notifications.module.css remove the redundant duplicate
declaration of border-bottom (the second "border-bottom: 15px solid `#fff`;") so
only a single border-bottom rule remains; edit the CSS in
Notifications.module.css to delete the duplicate line associated with the
border-bottom property.

---

Nitpick comments:
In `@src/components/Sources/SourceForm.tsx`:
- Around line 55-56: The empty-check for reader_options should treat
whitespace-only strings as empty; update the condition around
cleaned.reader_options in SourceForm.tsx (the block that currently reads "if
(cleaned.reader_options === '') { delete cleaned.reader_options; }") to first
confirm it's a string and then check trimmed value (e.g., use
cleaned.reader_options.trim() === '') so whitespace-only values are deleted as
well; also ensure you guard against null/undefined before calling trim.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 79c06cca-f3d8-48e1-a378-f580435380db

📥 Commits

Reviewing files that changed from the base of the PR and between ec33244 and c60ab5a.

📒 Files selected for processing (16)
  • .github/ISSUE_TEMPLATE/bug_report.md
  • .github/ISSUE_TEMPLATE/c4gt.md
  • .github/ISSUE_TEMPLATE/feature_request.md
  • .github/ISSUE_TEMPLATE/task.md
  • CODE_OF_CONDUCT.md
  • README.md
  • cypress/support/component-index.html
  • jest.config.ts
  • next.config.js
  • sentry.client.config.ts
  • sentry.edge.config.ts
  • sentry.server.config.ts
  • src/components/Notifications/Notifications.module.css
  • src/components/Sources/SourceForm.tsx
  • src/components/Sources/__tests__/cleanConfig.test.ts
  • tsconfig.json
💤 Files with no reviewable changes (2)
  • .github/ISSUE_TEMPLATE/feature_request.md
  • .github/ISSUE_TEMPLATE/task.md
✅ Files skipped from review due to trivial changes (6)
  • tsconfig.json
  • sentry.client.config.ts
  • sentry.edge.config.ts
  • jest.config.ts
  • sentry.server.config.ts
  • cypress/support/component-index.html

Comment thread .github/ISSUE_TEMPLATE/bug_report.md Outdated
Comment on lines +34 to +37
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Normalize example device/OS wording for consistency.

Minor docs quality nit: examples like iPhone6 / iOS8.1 are less readable than spaced forms.

✏️ Suggested wording tweak
-- Device: [e.g. iPhone6]
-- OS: [e.g. iOS8.1]
+- Device: [e.g. iPhone 6]
+- OS: [e.g. iOS 8.1]
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
- Device: [e.g. iPhone 6]
- OS: [e.g. iOS 8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
🧰 Tools
🪛 LanguageTool

[grammar] ~34-~34: Ensure spelling is correct
Context: ...following information):** - Device: [e.g. iPhone6] - OS: [e.g. iOS8.1] - Browser [e.g. st...

(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/ISSUE_TEMPLATE/bug_report.md around lines 34 - 37, Normalize example
values for the template fields "Device", "OS", "Browser", and "Version" to use
spaced, more readable forms (e.g., change "iPhone6" to "iPhone 6", "iOS8.1" to
"iOS 8.1", and "safari" to "Safari") so the examples are consistent and clearer;
update the example text for the "Device", "OS", "Browser", and "Version" lines
accordingly and keep capitalization consistent.

Comment thread README.md Outdated

The env varibales will be picked up from the .env file during the yarn build stage. This will copy all the env variables starting with NEXT_PUBLIC_ (that get embedd into the javascript code). The env varibales starting without NEXT_PUBLIC_ will be picked up during run time.
Run the script:
The env varibales will be picked up from the .env file during the yarn build stage. This will copy all the env variables starting with NEXT*PUBLIC* (that get embedd into the javascript code). The env varibales starting without NEXT*PUBLIC* will be picked up during run time.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fix spelling errors in documentation.

The documentation contains three spelling errors:

  • "varibales" should be "variables" (appears twice)
  • "embedd" should be "embedded"
📝 Proposed fix for spelling errors
-The env varibales will be picked up from the .env file during the yarn build stage. This will copy all the env variables starting with NEXT*PUBLIC* (that get embedd into the javascript code). The env varibales starting without NEXT*PUBLIC* will be picked up during run time.
+The env variables will be picked up from the .env file during the yarn build stage. This will copy all the env variables starting with NEXT*PUBLIC* (that get embedded into the javascript code). The env variables starting without NEXT*PUBLIC* will be picked up during run time.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
The env varibales will be picked up from the .env file during the yarn build stage. This will copy all the env variables starting with NEXT*PUBLIC* (that get embedd into the javascript code). The env varibales starting without NEXT*PUBLIC* will be picked up during run time.
The env variables will be picked up from the .env file during the yarn build stage. This will copy all the env variables starting with NEXT*PUBLIC* (that get embedded into the javascript code). The env variables starting without NEXT*PUBLIC* will be picked up during run time.
🧰 Tools
🪛 LanguageTool

[grammar] ~60-~60: Ensure spelling is correct
Context: ...Step 2: Build the Docker image The env varibales will be picked up from the .env file du...

(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)


[grammar] ~60-~60: Ensure spelling is correct
Context: ...es starting with NEXTPUBLIC (that get embedd into the javascript code). The env vari...

(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)


[grammar] ~60-~60: Ensure spelling is correct
Context: ...bedd into the javascript code). The env varibales starting without NEXTPUBLIC will be p...

(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@README.md` at line 60, Update the README sentence that currently reads "The
env varibales will be picked up from the .env file during the yarn build stage.
This will copy all the env variables starting with NEXT*PUBLIC* (that get embedd
into the javascript code). The env varibales starting without NEXT*PUBLIC* will
be picked up during run time." by correcting the three typos: change both
occurrences of "varibales" to "variables" and change "embedd" to "embedded" so
the sentence uses "variables" and "embedded" consistently.

Comment on lines +11 to +12
border-bottom: 15px solid #fff;
border-bottom: 15px solid #fff;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Remove the duplicate border-bottom declaration.

Line 12 duplicates the border-bottom property from line 11. The second declaration is redundant and should be removed.

🧹 Proposed fix
   border-bottom: 15px solid `#fff`;
-  border-bottom: 15px solid `#fff`;
   border-left: 15px solid transparent;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
border-bottom: 15px solid #fff;
border-bottom: 15px solid #fff;
border-bottom: 15px solid `#fff`;
border-left: 15px solid transparent;
🧰 Tools
🪛 Stylelint (17.6.0)

[error] 11-11: Unexpected duplicate "border-bottom" (declaration-block-no-duplicate-properties)

(declaration-block-no-duplicate-properties)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/components/Notifications/Notifications.module.css` around lines 11 - 12,
In Notifications.module.css remove the redundant duplicate declaration of
border-bottom (the second "border-bottom: 15px solid `#fff`;") so only a single
border-bottom rule remains; edit the CSS in Notifications.module.css to delete
the duplicate line associated with the border-bottom property.

@SarthakB11

Copy link
Copy Markdown
Author

@himanshudube97 @Ishankoradia — friendly bump when one of you has a moment. Small fix for #1648 (DAL-661); CI green and CodeRabbit's nits are on unrelated formatter-touched files. Happy to iterate.

Signed-off-by: Sarthak Bhardwaj <sarthak.bhardwaj21b@iiitg.ac.in>
@SarthakB11
SarthakB11 force-pushed the fix/1648-reader-options-empty-string branch from c60ab5a to b3d0b6f Compare July 11, 2026 23:36
@SarthakB11

Copy link
Copy Markdown
Author

Trimmed the PR to just the fix. Reverted the unrelated Prettier reformats CodeRabbit flagged, so the diff is now only SourceForm.tsx and the new cleanConfig test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Issue in adding excel sheet as a source in dalgo

1 participant