Skip to content

Commit initial framework for SDK test harness integration#116

Open
eli-r-ph wants to merge 5 commits into
mainfrom
eli.r/bootstrap-sdk-test-harness
Open

Commit initial framework for SDK test harness integration#116
eli-r-ph wants to merge 5 commits into
mainfrom
eli.r/bootstrap-sdk-test-harness

Conversation

@eli-r-ph
Copy link
Copy Markdown

💡 Motivation and Context

Integrate the SDK test harness library into posthog-rs, including SDK wrapper for capture V0 and V1 test suite, Docker-based test harness, Actions etc.

The "legacy" capture suite should run as expected, with compliant functionality passing. The v1 suite will fail at first: I'm stubbing a 5xx-only endpoint and will implement the new v1 capture submission functionality in a follow on PR.

The new test harness integration will land initially without being required in CI; we can enable that in a follow on once it's smoke-tested as well

💚 How did you test it?

Locally and in CI

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran sampo add to generate a changeset file

@eli-r-ph eli-r-ph requested a review from a team May 28, 2026 02:36
@eli-r-ph eli-r-ph self-assigned this May 28, 2026
@jose-sequeira
Copy link
Copy Markdown

Should use the harness in parallel mode, so that harness tests run faster.

PostHog/posthog-sdk-test-harness#10

Comment thread src/client/async_client.rs
eli-r-ph added 2 commits May 29, 2026 09:12
V1 is always batch internally -- even single capture() calls wrap the
event in a vec. Merge capture_v1 and capture_batch_v1 into one
capture_v1(Vec<Event>) stub in both async and blocking clients.
Refactor adapter to maintain per-test_id isolated state so the harness
can run tests concurrently. Add supports_parallel: true to /health,
pass --concurrency 10 in docker-compose and CI workflows.

~5x speedup (116s -> 22s) for compliance test runs.
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 30, 2026

posthog-rs Compliance Report

Date: 2026-05-30 03:18:26 UTC
Duration: 1359ms

⚠️ Some Tests Failed

2/108 tests passed, 106 failed


Capture_V1 Tests

⚠️ 1/92 tests passed, 91 failed

View Details
Test Status Duration
Endpoint And Method.Targets V1 Endpoint 105ms
Endpoint And Method.Does Not Use Legacy Endpoints 104ms
Required Headers.Has Authorization Bearer Header 104ms
Required Headers.Has Content Type Json 104ms
Required Headers.Has Posthog Sdk Info Format 108ms
Required Headers.Has Posthog Attempt Header 104ms
Required Headers.Has Posthog Request Id 103ms
Required Headers.Has Posthog Request Timestamp 103ms
Required Headers.Has User Agent 103ms
Body Format.Body Has Created At And Batch 101ms
Body Format.No Api Key In Body 99ms
Body Format.No Sent At In Body 99ms
Event Format.Event Has Required Root Fields 100ms
Event Format.Event Uuid Is Valid 99ms
Event Format.Event Timestamp Is Rfc3339 99ms
Event Format.Distinct Id Is String 99ms
Event Format.Distinct Id At Root Not Properties 98ms
Event Format.Custom Properties Preserved 99ms
Event Format.Set Properties Preserved 104ms
Event Format.Set Once Properties Preserved 94ms
Event Format.Groups Properties Preserved 102ms
Event Format.Sdk Generates Uuid If Not Provided 102ms
Event Format.Event Has Required Root Fields Batch 101ms
Event Format.Event Uuid Is Valid Batch 101ms
Event Format.Event Timestamp Is Rfc3339 Batch 106ms
Event Format.Distinct Id Is String Batch 101ms
Event Format.Distinct Id At Root Not Properties Batch 101ms
Event Format.Custom Properties Preserved Batch 101ms
Event Format.Set Properties Preserved Batch 101ms
Event Format.Set Once Properties Preserved Batch 96ms
Event Format.Groups Properties Preserved Batch 101ms
Event Format.Sdk Generates Uuid If Not Provided Batch 99ms
Batch Behavior.Multiple Events In Single Batch 104ms
Batch Behavior.Batch Envelope Smoke 100ms
Batch Behavior.Flush With No Events Sends Nothing 100ms
Batch Behavior.Flush At Triggers Batch 100ms
Batch Behavior.Created At Reflects Batch Creation Time 100ms
Deduplication.Generates Unique Uuids 100ms
Deduplication.Different Events Same Content Different Uuids 99ms
Deduplication.Preserves Uuid On Retry 95ms
Deduplication.Preserves Timestamp On Retry 99ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 98ms
Deduplication.No Duplicate Events In Batch 98ms
Header Behavior On Retry.Attempt Header Starts At One 98ms
Header Behavior On Retry.Attempt Header Increments On Retry 98ms
Header Behavior On Retry.Request Id Preserved On Retry 98ms
Header Behavior On Retry.Different Requests Have Different Request Ids 105ms
Header Behavior On Retry.Request Timestamp Changes On Retry 97ms
Response Format Validation.Success Response Has Uuid Keyed Results 97ms
Response Format Validation.Success Response Has Ok For Each Event 93ms
Response Format Validation.Success No Retry After When All Ok 100ms
Response Format Validation.Success Retry After Present When Retry Events 89ms
Response Format Validation.Success No Retry After When Drop Only 99ms
Response Format Validation.Response Echoes Request Id 99ms
Retry Behavior.Retries On 408 99ms
Retry Behavior.Retries On 500 89ms
Retry Behavior.Retries On 503 99ms
Retry Behavior.Retries On 504 99ms
Retry Behavior.Retryable Errors Have Retry After 99ms
Retry Behavior.Respects Retry After On Retryable Error 97ms
Retry Behavior.Does Not Retry On 400 73ms
Retry Behavior.Does Not Retry On 401 73ms
Retry Behavior.Does Not Retry On 402 103ms
Retry Behavior.Does Not Retry On 413 98ms
Retry Behavior.Does Not Retry On 415 99ms
Retry Behavior.Non Retryable Errors Have No Retry After 98ms
Retry Behavior.Implements Backoff 98ms
Retry Behavior.Max Retries Respected 98ms
Partial Batch Handling.Handles 200 Full Success 98ms
Partial Batch Handling.Handles 200 With All Ok 95ms
Partial Batch Handling.Does Not Retry Dropped Events 60ms
Partial Batch Handling.Does Not Retry Limited Events 121ms
Partial Batch Handling.Prunes Ok Events On Partial Retry 96ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry 95ms
Partial Batch Handling.Retries Only Retry Events From Partial 115ms
Partial Batch Handling.Partial Retry Preserves Uuids 95ms
Partial Batch Handling.Partial Retry Attempt Header Increments 95ms
Partial Batch Handling.Partial Retry Request Id Preserved 94ms
Partial Batch Handling.Respects Retry After On Partial 94ms
Partial Batch Handling.Unknown Result Treated As Terminal 99ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry 104ms
Compression.No Content Encoding When Disabled 70ms
Error Handling.Does Not Retry On Unknown 4Xx 88ms
Event Options.Cookieless Mode Override 78ms
Event Options.Disable Skew Correction Override 96ms
Event Options.Process Person Profile Override 78ms
Event Options.Product Tour Id Override 96ms
Event Options.Unset Options Omitted 78ms
Event Options.Options Override In Batch 97ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties 87ms
Geoip And Historical Migration.Historical Migration Set In Body 120ms
Geoip And Historical Migration.Historical Migration Absent By Default 113ms

Failures

endpoint_and_method.targets_v1_endpoint

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-22c15b03-11fb-4f19-b031-866110b68cf5'

endpoint_and_method.does_not_use_legacy_endpoints

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-8d404c3b-6535-4741-9b50-300597317482'

required_headers.has_authorization_bearer_header

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-198de12c-ea34-4cb9-85c7-003fdf65f29a'

required_headers.has_content_type_json

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-f3727283-e67a-4191-89ea-0b0131dffb31'

required_headers.has_posthog_sdk_info_format

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-a3f4677b-4e7e-4af9-b7de-7e924a67318c'

required_headers.has_posthog_attempt_header

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-3d4d68eb-0098-4632-bad6-7d76def42b54'

required_headers.has_posthog_request_id

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-1c961d21-24d1-4ecc-948f-03396f33eef8'

required_headers.has_posthog_request_timestamp

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-ada99bc6-b836-4ab2-954f-b37ee15a1877'

required_headers.has_user_agent

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-ffcbb334-92f8-4420-bd90-fe226fe10ac8'

body_format.body_has_created_at_and_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-0cde7dde-f97c-41bd-b4df-dbcd4ec44ed1'

body_format.no_api_key_in_body

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-6ca1e061-33f5-4f33-9441-a0bb7477ba55'

body_format.no_sent_at_in_body

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-b0429286-4c41-46c9-9c88-cd2bdcd67c44'

event_format.event_has_required_root_fields

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-7073b5a9-ee56-4172-a870-4765aaa7b454'

event_format.event_uuid_is_valid

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-e59552e6-2b9e-4ffb-9abf-2b6dec2bb9e6'

event_format.event_timestamp_is_rfc3339

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-26dffc3e-c30b-4e3d-bfc8-9ad20a89574b'

event_format.distinct_id_is_string

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-1e671b69-5060-4bd7-899d-7ac426d68ceb'

event_format.distinct_id_at_root_not_properties

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-e6dbf747-d91f-4f4a-b809-6680cde5d79d'

event_format.custom_properties_preserved

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-926c68ec-6c25-4a34-9fbc-aede08cea845'

event_format.set_properties_preserved

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-d1c1f7e2-b176-4704-8ea6-3291d5c748e6'

event_format.set_once_properties_preserved

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-f98b25f6-2915-4774-be1c-6ce074372069'

event_format.groups_properties_preserved

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-37732952-7760-4c88-84c0-c98ec803c3a5'

event_format.sdk_generates_uuid_if_not_provided

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-5ae03994-ce6e-4eb2-82b9-baf4a91becf4'

event_format.event_has_required_root_fields_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-d8c1f00e-d861-4daa-9549-7b97039c12b0'

event_format.event_uuid_is_valid_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-1a9d715a-f0f9-4d8b-8195-729bb16a7b9c'

event_format.event_timestamp_is_rfc3339_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-4929a5ae-a309-42dd-881e-fb5e66c0211a'

event_format.distinct_id_is_string_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-90dedecb-6800-4619-8290-73abc83fbf65'

event_format.distinct_id_at_root_not_properties_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-ba556239-2e0a-42f9-ba69-93e9fb8f1219'

event_format.custom_properties_preserved_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-59ff4cb2-a190-4f6c-afd3-8471c483d6b8'

event_format.set_properties_preserved_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-f90d4d01-480c-49a0-8811-622106ccf102'

event_format.set_once_properties_preserved_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-af5692a0-2ab5-462f-bc55-f49aa05e515b'

event_format.groups_properties_preserved_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-7e95a10c-2508-4ded-b866-e047eb6e6d58'

event_format.sdk_generates_uuid_if_not_provided_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-340ffcfd-0772-4a02-8381-0acf317045e5'

batch_behavior.multiple_events_in_single_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-4ac52446-da34-4355-a101-b7d860a2f7c6'

batch_behavior.batch_envelope_smoke

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-72bc1548-0275-4eab-918e-6d04e0ba3a4f'

batch_behavior.flush_at_triggers_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-2b99fd4e-a014-4ef0-83e7-defbd37ab79c'

batch_behavior.created_at_reflects_batch_creation_time

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-0d76a306-e1e1-45c4-8e77-cb925537a821'

deduplication.generates_unique_uuids

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-acf7f0cb-9322-4d6e-b71f-971e7e2646fc'

deduplication.different_events_same_content_different_uuids

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-85975106-a942-4e0a-9d8e-42a538afa85e'

deduplication.preserves_uuid_on_retry

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-9f9aaf5c-6bcf-47d3-86ad-82761bee7686'

deduplication.preserves_timestamp_on_retry

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-19299f1f-367a-48c2-a309-30dac361bf2d'

deduplication.preserves_uuid_and_timestamp_on_batch_retry

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-de10cd39-0cb7-47ed-a05b-f6b4dce4c839'

deduplication.no_duplicate_events_in_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-551a9aba-f26b-4c3f-8f4b-e8398789b4de'

header_behavior_on_retry.attempt_header_starts_at_one

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-7d4bab5d-a3ea-4531-b690-c1fc58afc082'

header_behavior_on_retry.attempt_header_increments_on_retry

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-1ba711af-1168-4963-b9b4-9919539c7ad1'

header_behavior_on_retry.request_id_preserved_on_retry

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-463f2df2-6b74-4ae4-94b0-f609c2e771be'

header_behavior_on_retry.different_requests_have_different_request_ids

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-9a17d47d-3d09-42ad-bae7-1d482b9d607d'

header_behavior_on_retry.request_timestamp_changes_on_retry

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-4891f0ec-727c-4d1c-a590-9d19cfa80fb7'

response_format_validation.success_response_has_uuid_keyed_results

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-31bd3a43-0491-404a-aa8a-43089da4badf'

response_format_validation.success_response_has_ok_for_each_event

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-5b17389a-de83-4069-8c8b-047ff5b4686d'

response_format_validation.success_no_retry_after_when_all_ok

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-1a501551-f2b1-4e83-8284-88170e298fcc'

response_format_validation.success_retry_after_present_when_retry_events

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-76e0ead1-b931-460d-b12e-109da1e42742'

response_format_validation.success_no_retry_after_when_drop_only

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-bed7570a-6631-4e07-be44-2d19fd3430f7'

response_format_validation.response_echoes_request_id

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-4314de3e-8659-46e7-89a4-383bdf351aad'

retry_behavior.retries_on_408

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-7c13fe2e-421e-4e96-b381-a3b177facd8a'

retry_behavior.retries_on_500

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-25fa7a6d-e7b2-4ccd-913b-efd9af9a6f8a'

retry_behavior.retries_on_503

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-a209404a-3cf6-463e-8d50-c6718e7a1bc9'

retry_behavior.retries_on_504

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-b2706db9-5d22-4e2b-9519-5a8b5dd8d5b0'

retry_behavior.retryable_errors_have_retry_after

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-95eccba9-d2fe-4590-9eb3-b5c368eb6a94'

retry_behavior.respects_retry_after_on_retryable_error

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-f3efc14a-42a1-40be-a2cc-174d1e34eb7a'

retry_behavior.does_not_retry_on_400

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-54ff20b1-e550-4182-bf51-82c64c85f81a'

retry_behavior.does_not_retry_on_401

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-861fdd04-08d4-4f91-aff9-b7fa7f67412b'

retry_behavior.does_not_retry_on_402

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-3b935a79-d9e5-4b02-9cb5-b5e1bd6d513e'

retry_behavior.does_not_retry_on_413

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-a58498e2-7191-4183-bc55-649de5b88413'

retry_behavior.does_not_retry_on_415

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-cb0b0536-92bb-47d2-aa52-99ff1196207f'

retry_behavior.non_retryable_errors_have_no_retry_after

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-92e6d369-36a2-4c52-9947-e16930991b48'

retry_behavior.implements_backoff

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-c6a7b128-e019-4e45-9c44-27c9f402717c'

retry_behavior.max_retries_respected

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-3ab50578-15fd-4f65-92cd-5f001a616ec8'

partial_batch_handling.handles_200_full_success

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-4708802c-c7d5-4634-b165-0f3d7bfbdd27'

partial_batch_handling.handles_200_with_all_ok

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-0d7fbebb-4667-4cc9-8768-da0610343ba5'

partial_batch_handling.does_not_retry_dropped_events

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-fcb5029d-6938-47b9-a954-b5b2577f6bdc'

partial_batch_handling.does_not_retry_limited_events

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-c4a02e53-fdd8-4c03-bd30-1aa642671e73'

partial_batch_handling.prunes_ok_events_on_partial_retry

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-58345ec4-146f-49a7-82b2-d8d6fdd2ad56'

partial_batch_handling.prunes_dropped_events_on_partial_retry

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-96af5530-5879-485c-9492-e81b714de29e'

partial_batch_handling.retries_only_retry_events_from_partial

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-47d96355-ba8f-4988-b348-2d908d01ac31'

partial_batch_handling.partial_retry_preserves_uuids

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-f0b94eda-8d7c-4760-afc1-c106aa6cef1c'

partial_batch_handling.partial_retry_attempt_header_increments

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-1162ea8f-77f5-4e71-aeb8-b8397569d85d'

partial_batch_handling.partial_retry_request_id_preserved

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-7f96773b-40ee-4a53-b284-482fa85632d6'

partial_batch_handling.respects_retry_after_on_partial

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-26a66cd6-3874-4a94-86a0-330917b74dac'

partial_batch_handling.unknown_result_treated_as_terminal

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-f3ac2765-cdf2-4933-9fda-e992d663be0d'

partial_batch_handling.mixed_ok_drop_limited_no_retry

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-ec5ddaaa-dd40-4780-8e67-6a94a623de10'

compression.no_content_encoding_when_disabled

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-c83e764d-7dc2-4ba7-9ab5-181c1a2e36dc'

error_handling.does_not_retry_on_unknown_4xx

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-c81341f9-864b-4395-ba72-0df5ea1f0251'

event_options.cookieless_mode_override

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-49eb3564-1e9b-499d-b0df-61306e8cdef6'

event_options.disable_skew_correction_override

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-54ce3343-9d6e-4ec6-b70f-58cf25d6cc85'

event_options.process_person_profile_override

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-6e375cb5-d79c-40d1-b2cf-6c1f1d4e13f7'

event_options.product_tour_id_override

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-91c7c028-3a4c-4e39-830a-75bdc15f344c'

event_options.unset_options_omitted

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-e42112c4-94c2-4f12-b081-f4f072409d50'

event_options.options_override_in_batch

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-f992b152-4f90-46dc-b86a-c0ef1e638a2f'

geoip_and_historical_migration.geoip_disable_injected_into_properties

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-370ac5a5-b384-4981-a13b-b597882ec5c1'

geoip_and_historical_migration.historical_migration_set_in_body

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-92017a91-2e99-40b7-9565-afc558940ab1'

geoip_and_historical_migration.historical_migration_absent_by_default

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-428f4da4-5c23-4de2-820c-faed5e3f8569'

Feature_Flags Tests

⚠️ 1/16 tests passed, 15 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 86ms
Request Payload.Flags Request Uses V2 Query Param 55ms
Request Payload.Flags Request Hits Flags Path Not Decide 46ms
Request Payload.Flags Request Omits Authorization Header 56ms
Request Payload.Token In Flags Body Matches Init 58ms
Request Payload.Groups Round Trip 80ms
Request Payload.Groups Default To Empty Object 78ms
Request Payload.Person Properties Distinct Id Auto Populated When Caller Omits It 87ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 93ms
Request Payload.Disable Geoip Omitted Defaults To False 76ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 58ms
Request Lifecycle.No Flags Request On Init Alone 73ms
Request Lifecycle.No Flags Request On Normal Capture 106ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 81ms
Request Lifecycle.Mock Response Value Is Returned To Caller 80ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 85ms

Failures

request_payload.request_with_person_properties_device_id

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-5b4a07d8-31a0-4deb-a7e4-dea3e8cf3658'

request_payload.flags_request_uses_v2_query_param

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-f9b0fb1c-fe15-4aa3-9995-116d4d26c2ac'

request_payload.flags_request_hits_flags_path_not_decide

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-26da5a15-c77b-40e7-b514-46bad0200ab9'

request_payload.flags_request_omits_authorization_header

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-66df9fcd-ad31-4b72-9834-a66099c09c97'

request_payload.token_in_flags_body_matches_init

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-d43bf3f9-dc70-4dcd-a675-11ad96126677'

request_payload.groups_round_trip

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-33641d65-f9f0-40f0-bb17-cbb1933418ff'

request_payload.groups_default_to_empty_object

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-fb4dca9d-fc63-4f3d-b842-3ebcbb4d31bc'

request_payload.person_properties_distinct_id_auto_populated_when_caller_omits_it

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-7ee9d019-8039-487a-b071-08813fa5edb4'

request_payload.disable_geoip_false_propagates_as_geoip_disable_false

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-190ecb1f-187c-4cae-9600-82bf42a98307'

request_payload.disable_geoip_omitted_defaults_to_false

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-804d5c3c-dfa8-4a85-872a-28f62294639f'

request_payload.flag_keys_to_evaluate_contains_only_requested_key

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-1670aad4-4f52-48a6-b70a-f3d880f61ac5'

request_lifecycle.no_flags_request_on_normal_capture

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture?test_id=t-d5a7e2a6-0fe1-4209-9970-b9ced9decec9'

request_lifecycle.two_flag_calls_produce_two_remote_requests

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-8d421081-a7ef-4ed8-83cb-55ee9bce1a6e'

request_lifecycle.mock_response_value_is_returned_to_caller

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-ae890413-1984-4206-94e9-87a1c533cfdf'

side_effect_events.get_feature_flag_captures_feature_flag_called_event

404, message='Not Found', url='http://sdk-adapter:8080/get_feature_flag?test_id=t-6d5f6b3b-2f48-4339-926e-09adccb4fafa'

…l_migration, and parallel mode

- Add extra_capture_headers to ClientOptions so the adapter can inject
  X-Test-Id headers on outbound SDK capture requests for parallel test
  isolation.
- Wire disable_geoip and historical_migration from harness InitRequest
  through to SDK ClientOptionsBuilder and AdapterState.
- Add options field to CaptureRequest (wiring deferred to v1-impl branch
  where EventOptions exists on Event).
- Flip SUPPORTS_PARALLEL to true now that adapter injects X-Test-Id.
- Fix Dockerfiles to use rust:1-bookworm instead of rust:latest to
  prevent glibc mismatch with debian:bookworm-slim runtime.
@eli-r-ph eli-r-ph force-pushed the eli.r/bootstrap-sdk-test-harness branch from 2c5e8a6 to 2730399 Compare May 30, 2026 03:15
@eli-r-ph eli-r-ph marked this pull request as ready for review May 30, 2026 03:51
@eli-r-ph eli-r-ph requested a review from a team as a code owner May 30, 2026 03:51
@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented May 30, 2026

Comments Outside Diff (2)

  1. compliance/adapter/src/main.rs, line 498-555 (link)

    P1 Mutex held across async network I/O blocks claimed parallel support

    The instances lock is acquired at the top of capture_event and is never released before client.capture(event).await resolves. Since all test-ids share the same Arc<Mutex<...>>, every concurrent /capture call queues behind this single lock, serializing all traffic regardless of test_id. This directly contradicts SUPPORTS_PARALLEL: bool = true and the --concurrency 10 setting in docker-compose — the test harness will see throughput capped at 1 in-flight capture at a time, which may cause timeout failures in the parallel test suite.

    The fix is to clone (or Arc-wrap) the Client out of the locked section, drop the lock, and then perform the network call and state update in a second short-lived lock acquisition.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: compliance/adapter/src/main.rs
    Line: 498-555
    
    Comment:
    **Mutex held across async network I/O blocks claimed parallel support**
    
    The `instances` lock is acquired at the top of `capture_event` and is never released before `client.capture(event).await` resolves. Since all test-ids share the same `Arc<Mutex<...>>`, every concurrent `/capture` call queues behind this single lock, serializing all traffic regardless of `test_id`. This directly contradicts `SUPPORTS_PARALLEL: bool = true` and the `--concurrency 10` setting in docker-compose — the test harness will see throughput capped at 1 in-flight capture at a time, which may cause timeout failures in the parallel test suite.
    
    The fix is to clone (or `Arc`-wrap) the `Client` out of the locked section, drop the lock, and then perform the network call and state update in a second short-lived lock acquisition.
    
    How can I resolve this? If you propose a fix, please make it concise.
  2. compliance/adapter/src/main.rs, line 535-543 (link)

    P2 pending_events is always 0; increment/decrement happen in the same branch

    In the Ok arm, pending_events is incremented to 1 and then immediately decremented back to 0 in consecutive lines — both execute only after client.capture(event).await has already returned. The field will always read 0 in /state responses. To track genuinely in-flight events, the increment must happen before the lock is released (i.e., before the await), and the decrement after the await in a re-acquired lock — which is the same restructuring needed for the parallelism fix above.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: compliance/adapter/src/main.rs
    Line: 535-543
    
    Comment:
    **`pending_events` is always 0; increment/decrement happen in the same branch**
    
    In the `Ok` arm, `pending_events` is incremented to 1 and then immediately decremented back to 0 in consecutive lines — both execute only after `client.capture(event).await` has already returned. The field will always read 0 in `/state` responses. To track genuinely in-flight events, the increment must happen before the lock is released (i.e., before the await), and the decrement after the await in a re-acquired lock — which is the same restructuring needed for the parallelism fix above.
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
compliance/adapter/src/main.rs:498-555
**Mutex held across async network I/O blocks claimed parallel support**

The `instances` lock is acquired at the top of `capture_event` and is never released before `client.capture(event).await` resolves. Since all test-ids share the same `Arc<Mutex<...>>`, every concurrent `/capture` call queues behind this single lock, serializing all traffic regardless of `test_id`. This directly contradicts `SUPPORTS_PARALLEL: bool = true` and the `--concurrency 10` setting in docker-compose — the test harness will see throughput capped at 1 in-flight capture at a time, which may cause timeout failures in the parallel test suite.

The fix is to clone (or `Arc`-wrap) the `Client` out of the locked section, drop the lock, and then perform the network call and state update in a second short-lived lock acquisition.

### Issue 2 of 2
compliance/adapter/src/main.rs:535-543
**`pending_events` is always 0; increment/decrement happen in the same branch**

In the `Ok` arm, `pending_events` is incremented to 1 and then immediately decremented back to 0 in consecutive lines — both execute only after `client.capture(event).await` has already returned. The field will always read 0 in `/state` responses. To track genuinely in-flight events, the increment must happen before the lock is released (i.e., before the await), and the decrement after the await in a re-acquired lock — which is the same restructuring needed for the parallelism fix above.

Reviews (1): Last reviewed commit: "add extra_capture_headers, adapter wirin..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants