[One Workflow] fix: clarify connector response size limit errors by shahargl · Pull Request #268591 · elastic/kibana

shahargl · 2026-05-10T13:35:55Z

Summary

This PR improves how Workflows reports and handles large responses from spec-generated connectors.

The original issue was discovered while using google_drive.downloadFile in a workflow. A 10-20 MB Google Drive file failed with confusing step-size errors, even when max-step-size looked high enough. The UX made it appear that the workflow memory/output guardrail was the source of the failure, but the actual first limit being hit was often the Actions HTTP client response limit: xpack.actions.maxResponseContentLength.

Before this change, spec connectors could fail inside the Actions Axios client before Workflows had a chance to build or measure the step output. Those failures surfaced as generic connector errors or misleading StepSizeLimitExceeded messages, without making it clear that the response was rejected by Actions before workflow output storage.

Investigation

1. Original Google Drive failure

The first failing case was a workflow that selected a Google Drive file and called:

- name: download_file
  type: google_drive.downloadFile
  connector-id: elastic-drive-shahar
  max-step-size: 1000mb
  with:
    fileId: "{{ steps.get_latest_file.output.files[0].id }}"

The user-facing error implied the workflow step output size limit was the problem. In practice there are two separate limits:

xpack.actions.maxResponseContentLength: enforced by the Actions Axios client before connector output exists.
max-step-size: enforced by Workflows on serialized step output after the connector returns data.

For spec-generated connectors, the Actions Axios limit was being hit first in many cases. Changing only workflow output limits did not always affect the underlying Axios request limit, which made the error misleading.

2. Root cause: Actions transport limit vs workflow output limit

The key distinction:

If Axios rejects with maxContentLength, the connector response never becomes workflow output.
If Axios succeeds but the returned object serializes larger than max-step-size, Workflows rejects it with StepSizeLimitExceeded.

Google Drive made this especially confusing because file downloads are returned as base64. A 17 MB file becomes roughly 22.7 MB of serialized workflow output, so the raw response size and stored workflow output size differ.

This PR makes those cases explicit:

ActionsResponseContentLengthExceeded: the Actions HTTP client rejected the connector response before workflow output was built.
StepSizeLimitExceeded: Workflows built the output, measured it, and rejected the serialized output.

Chosen approach

We discussed a few options and chose the most direct, high-value fix.

Generic header extraction

For normal HTTP responses, the generated spec-connector executor now tries to extract response size from Axios error headers.

Default behavior:

Read content-length from error.response.headers.
If unavailable, read content-length from error.request.res.headers.
Pass the parsed value through errorMeta.contentLengthBytes.

Connector specs can also declare a custom header:

responseSizeHeader?: string;

Example use case: a provider that advertises size with a header like x-resource-size instead of content-length.

Step-level `max-step-size` as the spec-connector runtime override

For spec connectors, explicit step-level max-step-size is now passed down to the generated connector executor as reserved fetcher.max_content_length, which then overrides Axios maxContentLength.

That means this raises both relevant limits for spec connectors:

- name: download_file
  type: google_drive.downloadFile
  connector-id: elastic-drive-shahar
  max-step-size: 30mb
  with:
    fileId: "..."

This is intentionally step-level. We only pass the override when the step explicitly sets max-step-size, so we do not silently change connector transport behavior from global defaults.

Connector-provided metadata for harder cases

Some providers do not expose useful response headers on the failing download response.

Google Drive is the main example. The actual media response was chunked and did not expose content-length, but Drive file metadata already contains the file size.

For those cases, connector code can attach metadata to the thrown error:

setConnectorActionErrorMeta(error, {
  contentLengthBytes: rawFileSizeBytes,
  estimatedOutputBytes: estimatedBase64OutputBytes,
});

The generated executor merges this metadata with any Axios-derived header metadata.

This keeps the framework generic, while still allowing connectors with provider-specific knowledge to provide better hints.

Cases fixed

Google Drive

google_drive.downloadFile now:

Fetches file metadata before download.
Uses Drive size as contentLengthBytes when download fails.
Estimates stored base64 output size.
Attaches both values via setConnectorActionErrorMeta.

Generic spec connectors

The generated executor now:

Accepts reserved fetcher.max_content_length.
Passes it to getAxiosInstanceWithAuth.
Extracts response size from headers on Axios failures.
Merges connector-provided metadata when available.

This covers providers that expose content-length without connector-specific work.

Actions Axios client

getAxiosInstanceWithAuth now accepts an optional maxContentLength override and logs debug metadata for maxContentLength failures.

The debug log includes sanitized response headers and header keys. This helped confirm cases like:

Jina returning transfer-encoding: chunked and no content-length.
S3 returning content-length in request.res.headers.

Sensitive headers are redacted.

Workflow errors

ActionsResponseContentLengthExceeded was added for Actions transport failures.

It distinguishes between:

Cases where max-step-size can help because this is a spec connector override path.
Cases where only xpack.actions.maxResponseContentLength can help.

When exact size is known, the message suggests an exact max-step-size. When exact size is unknown, it avoids pretending that “above 1 MB” is enough and says to set a larger max-step-size.

StepSizeLimitExceeded was also improved to include actual serialized output size when Workflows has already built the output.

Jina

Manual testing with jina.browse found a separate connector bug.

When Axios threw a transport error without response, Jina’s error handling tried to read:

err.response.data

This masked the real maxContentLength error with:

Cannot read properties of undefined (reading 'data')

This PR fixes Jina to only unwrap structured Jina API errors when response.data.code exists, and otherwise rethrow the original transport error.

Amazon S3

Manual testing with amazon_s3.downloadFile found another important case.

S3 has its own connector-level guardrail: if the file exceeds maximumDownloadSizeBytes, it returns a presigned URL instead of content. When that guard is explicitly raised and the download reaches Axios, S3 responses include content-length.

However, the S3 connector wrapped Axios errors in a new AWS S3 error (...), which discarded the original Axios headers before the generic executor could extract them.

This PR preserves size metadata when S3 wraps Axios download errors, so Workflows can still show contentLengthBytes, estimatedOutputBytes, and suggestedLimitBytes.

Testing

Automated tests cover:

Connector action error metadata helpers.
Generated executor fetcher.max_content_length behavior.
Generic response size header extraction.
Connector-provided metadata merge behavior.
Generated params schema allowing reserved fetcher options.
ActionsResponseContentLengthExceeded messaging.
StepSizeLimitExceeded actual/estimated/suggested size messaging.
Google Drive file size hint attachment.
Jina transport error preservation.
S3 metadata preservation when wrapping Axios errors.
Actions Axios maxContentLength override and debug logging.

Commands run:

node scripts/jest src/platform/packages/shared/kbn-connector-specs/src/connector_spec.test.ts src/platform/packages/shared/kbn-connector-specs/src/specs/google_drive/google_drive.test.ts
node scripts/jest src/platform/packages/shared/kbn-connector-specs/src/specs/amazon_s3/amazon_s3_api.test.ts
node scripts/jest src/platform/packages/shared/kbn-connector-specs/src/specs/jina/jina_reader.test.ts
node scripts/jest src/platform/plugins/shared/workflows_execution_engine/server/step/connector_step.test.ts src/platform/plugins/shared/workflows_execution_engine/server/step/errors.test.ts src/platform/plugins/shared/workflows_execution_engine/server/step/node_implementation.test.ts
node scripts/jest x-pack/platform/plugins/shared/actions/server/lib/get_axios_instance.test.ts x-pack/platform/plugins/shared/actions/server/lib/single_file_connectors/generate_executor_function.test.ts x-pack/platform/plugins/shared/actions/server/lib/single_file_connectors/generate_params_schema.test.ts
node scripts/check_changes.ts

Manual validation covered:

Google Drive file download.
Jina browse responses with and without content-length.
Amazon S3 download with connector guardrail and with raised maximumDownloadSizeBytes.
Existing HTTP connector behavior.

References

Closes elastic/security-team#17243

Made with Cursor

nomagick

LGTM on the changes to Jina Reader connector. Thanks for fixing of the error handling bug.

jcger

IIUC we already have an approach to define a specific connector/action error. Please have a look at ConnectorAuthorizationError in #260351. We should try to leverage/follow the same pattern for new errors

jcger · 2026-05-11T09:15:38Z

@@ -18,6 +19,14 @@ const GOOGLE_WORKSPACE_MIME_PREFIX = 'application/vnd.google-apps.';
 const DEFAULT_EXPORT_MIME_TYPE = 'application/pdf';
 // XLSX preserves tabular structure better than PDF for spreadsheets
 const SHEETS_EXPORT_MIME_TYPE = 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet';
+const ESTIMATED_JSON_OUTPUT_OVERHEAD_BYTES = 1024;


where does this number come from exactly? The same value is used in amazon_s3, why 1024 in particular?

re: ConnectorAuthorizationError

Looked at ConnectorAuthorizationError - the key difference is that with auth errors, our code creates and throws the error from scratch. Here the error is created by Axios internally - we just catch it.

We can't subclass AxiosError because we don't have its constructor args, we only have the finished error object. And wrapping it in a new class would mean copying over all Axios properties (response, headers, status, isAxiosError, etc.) manually, which is fragile.

The WeakMap just attaches metadata to the existing Axios error object without touching it, so nothing downstream needs to change. Happy to discuss if you see a cleaner approach though.

re: 1024

It's just a number that's comfortably larger than the JSON envelope size (status, headers, key names, etc.). The actual envelope is a few hundred bytes - 1024 is a safe round upper bound. The S3 128 * 1024 is unrelated (MAX_DOWNLOAD_FILE_SIZE_BYTES). Will add a comment to clarify.

re: ConnectorAuthorizationError

I understand the error is generated by Axios, but we could transform it into a connector framework specific error to keep the same architecture introduced by ConnectorAuthorizationError

What I'd suggest is: add a typed ConnectorResponseSizeLimitError to kbn-connector-specs, catch the Axios size-limit error and throw it as a typed error, have action_executor.ts handle it like ConnectorAuthorizationError (sets errorName), and have connector_step.ts detect errorName === 'ConnectorResponseSizeLimitError' instead of string-matching the message.

This would make the contract between the two layers explicit and typed rather than an untyped bag + string match.

I put together a more complete draft of what I mean here jcger#4

Thanks for the detailed draft! Agree this is the right direction architecturally - a typed error is cleaner than string-matching an שxios internal message.

I'd prefer to land this as a follow-up though, for a few reasons:

It adds changes to action_executor.ts (core Actions code) which widens the review scope beyond this PR's intent

The axios error detection in the interceptor (ERR_BAD_RESPONSE + config comparison) needs careful validation - the current string match, while ugly, is battle-tested

This PR is already at 20+ files; adding the typed error path would nearly double the change set

Happy to either take this as an immediate follow-up PR, or if you'd prefer to own it based on your draft, that works too. The foundation (shared utils, responseSizeHeader, errorMeta) is all in place for the typed error to build on top of.

Although I disagree with some statements, I'm fine moving forward. I'll finish the review of the current approach and you can follow up with the typed error path.

Apmats

Fundamentally looks fine for me and thanks for addressing this.

Apmats · 2026-05-12T11:30:16Z

+
+const connectorActionErrorMeta = new WeakMap<object, ConnectorActionErrorMeta>();
+
+const getFinitePositiveNumber = (value: unknown): number | undefined => {


Maybe export this to reuse across other connectors?

Done - getFinitePositiveNumber is now exported from connector_spec.ts and re-exported from the package index. Both google_drive and amazon_s3 import it from the shared location.

Apmats · 2026-05-12T11:34:06Z

+  return numericValue;
+};
+
+export const setConnectorActionErrorMeta = (


Is this to avoid touching errors themselves? Seems unweildly.

Yes - the error is created by Axios internally, so we can't subclass it or add typed properties. The WeakMap lets us attach metadata without mutating the error object. Open to alternatives if you have a cleaner pattern in mind.

artem-shelkovnikov

I've checked only the connectors part, left comments/questions about mechanical stuff in the PR

artem-shelkovnikov · 2026-05-12T11:06:15Z

+  return numericValue;
+};
+
+const getHeaderValue = ({


Is there a way to de-dupe this and just have one you defined in https://github.com/elastic/kibana/pull/268591/changes#diff-5ef124780665cb9c3f35a4774d968c8584cd255c472466c4f2aea5c1427bb3aaR43?

Done - moved getFinitePositiveNumber, getEstimatedBase64OutputBytes, getHeaderValue, getResponseContentLengthBytes, and ESTIMATED_JSON_OUTPUT_OVERHEAD_BYTES to connector_spec.ts and exported them from the package. Both google_drive and amazon_s3 now import from the shared location.

artem-shelkovnikov · 2026-05-12T11:07:07Z

+const getEstimatedBase64OutputBytes = (rawBytes: number): number =>
+  Math.ceil(rawBytes / 3) * 4 + ESTIMATED_JSON_OUTPUT_OVERHEAD_BYTES;
+
+const getResponseContentLengthBytes = (error: unknown): number | undefined => {


Is there a way to not have it as error: unknown?

Moved to a shared getResponseContentLengthBytes in connector_spec.ts. The error: unknown param type stays since this is called from catch blocks where TS types the caught value as unknown. The as cast is now contained in one place.

artem-shelkovnikov · 2026-05-12T11:54:13Z

+};
+
+const getEstimatedBase64OutputBytes = (rawBytes: number): number =>
+  Math.ceil(rawBytes / 3) * 4 + ESTIMATED_JSON_OUTPUT_OVERHEAD_BYTES;


Where's the math coming from? Also, should it live elsewhere, not in the connector?

Base64 encoding expands 3 raw bytes into 4 characters (Math.ceil(rawBytes / 3) * 4), plus ESTIMATED_JSON_OUTPUT_OVERHEAD_BYTES (1024) as a safe upper bound for the JSON envelope. Added a comment explaining this. Moved to connector_spec.ts so it's not in the connector anymore.

artem-shelkovnikov · 2026-05-12T11:54:39Z

+  const axiosError = error as {
+    response?: { headers?: unknown };
+    request?: { res?: { headers?: unknown } };
+  };


This looks unnecessary

Removed - this local getResponseContentLengthBytes is now imported from the shared connector_spec.ts.

artem-shelkovnikov · 2026-05-12T11:55:52Z

+const getFinitePositiveNumber = (value: unknown): number | undefined => {
+  const numericValue = typeof value === 'string' ? Number(value) : value;
+  if (typeof numericValue !== 'number' || !Number.isFinite(numericValue) || numericValue < 0) {
+    return undefined;
+  }
+
+  return numericValue;
+};


Duplicated function

Fixed - now imported from connector_spec.ts.

artem-shelkovnikov · 2026-05-12T11:56:07Z

+const getEstimatedBase64OutputBytes = (rawBytes: number): number =>
+  Math.ceil(rawBytes / 3) * 4 + ESTIMATED_JSON_OUTPUT_OVERHEAD_BYTES;
+


Fixed - same, imported from shared.

dej611 · 2026-05-12T13:19:49Z

@@ -29,6 +40,18 @@ function escapeQueryValue(value: string): string {
  return value.replace(/\\/g, '\\\\').replace(/'/g, "\\'");
 }

+const getFinitePositiveNumber = (value: unknown): number | undefined => {


can we put this and other reused utilities function into some package/common area and use it instead of redeclaring it every time?

Done - moved getFinitePositiveNumber, getEstimatedBase64OutputBytes, getHeaderValue, getResponseContentLengthBytes, and ESTIMATED_JSON_OUTPUT_OVERHEAD_BYTES to connector_spec.ts and exported from the package. Both connectors now import from the shared location.

dej611 · 2026-05-12T13:34:58Z

+const ACTIONS_MAX_CONTENT_LENGTH_ERROR_PATTERN = /maxContentLength size of (\d+) exceeded/;
+
+const getActionsMaxContentLengthLimit = (errorMessage: string): number | undefined => {
+  const match = errorMessage.match(ACTIONS_MAX_CONTENT_LENGTH_ERROR_PATTERN);


isn't this a bit fragile?

Unfortunately yes — Axios doesn't expose a typed error code for maxContentLength failures, so regex on the internal message is the only detection method. Added a comment explaining this.

dej611 · 2026-05-12T13:36:16Z

+        axiosInstance,
+        secrets
+      );
+      configuredAxiosInstance.interceptors.response.use(undefined, (error: unknown) => {


mind that the configuredAxiosInstance may be a different thing than the one on the input. Perhaps it's worth checking that before attaching the interceptors.

This is a pre-existing pattern - our PR just attaches an interceptor to whatever configure() returns, same as the rest of the function already does. The instance identity question exists regardless of this change.

Surface Actions response-size failures distinctly from workflow output-size failures, and let spec connector steps use max-step-size to raise the generated Axios response limit. Preserve provider size hints for Google Drive and S3 so errors can suggest actionable limits. Closes elastic/security-team#17243 Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

Move getFinitePositiveNumber, getEstimatedBase64OutputBytes, getHeaderValue, getResponseContentLengthBytes, and ESTIMATED_JSON_OUTPUT_OVERHEAD_BYTES to connector_spec.ts. Import from shared location in google_drive and amazon_s3. Apply candidates.find() suggestion in errors.ts. Add comment explaining Axios regex pattern in connector_step.ts. Co-authored-by: Cursor <cursoragent@cursor.com>

- Fix getResponseContentLengthBytes crash on null/primitive input - Preserve error metadata when throwGoogleDriveError creates new error - Fix double getFinitePositiveNumber call in setConnectorActionErrorMeta - Move misplaced size-hint test to downloadFile describe block - Import ESTIMATED_JSON_OUTPUT_OVERHEAD_BYTES in tests instead of magic 1024 - De-duplicate helpers in generate_executor_function.ts (import from @kbn/connector-specs) - Add unit tests for shared helpers (getFinitePositiveNumber, getHeaderValue, getResponseContentLengthBytes, getEstimatedBase64OutputBytes) - Add JSDoc to exported helpers - Add merge-order comment in generate_executor_function.ts Co-authored-by: Cursor <cursoragent@cursor.com>

When contentLengthBytes is smaller than the limit, the header clearly doesn't reflect the actual in-memory size (e.g. compressed responses, chunked transfers). Hide it from the error message to avoid confusion like "limit is 1 MB but content length was 300 KB". Co-authored-by: Cursor <cursoragent@cursor.com>

Jina/Cloudflare decompresses responses before sending them, so content-length reflects the compressed wire size. The actual decompressed size is in x-decompressed-content-length. Use responseSizeHeader to read the correct size for browse and search actions. Co-authored-by: Cursor <cursoragent@cursor.com>

kibanamachine · 2026-07-01T04:49:05Z