Skip to content

Improve realtime update resilience with request retries and out-of-range data validation#25

Open
FelixINX wants to merge 2 commits into
mainfrom
improve-realtime-resiliency-14159488042253352629
Open

Improve realtime update resilience with request retries and out-of-range data validation#25
FelixINX wants to merge 2 commits into
mainfrom
improve-realtime-resiliency-14159488042253352629

Conversation

@FelixINX

Copy link
Copy Markdown
Member

This PR improves the resiliency of the background processes handling real-time data updates.

  1. Network Resilience (DispatchAgency.php): Added a 10-second timeout and a retry mechanism (3 attempts, 100ms apart) to the Http::get call that fetches transit feed data. A custom closure now ensures it retries on connection exceptions (e.g., cURL error 28 timeouts) and explicitly on 5xx and 400 errors, mitigating transient network failures.
  2. Data Integrity & Bounds Validation (GtfsRtHandler.php, JavascriptGtfsRtHandler.php, NextbusJsonHandler.php): Implemented strict validation checks for real-time data before it is inserted into the database. This prevents database "out of range" errors:
    • Coordinates: Latitude must be between -90 and 90, Longitude must be between -180 and 180.
    • Speed: Must be between 0 and 150 km/h.
    • Odometer: Must be between 0 and 10,000,000.
    • Bearing: Must be between 0 and 360 degrees.
    • Any out-of-bounds value is safely set to null.
    • Removed an agency-specific hardcoded exception for the London Transit Commission (ltc), since the coordinate bounds validation now safely handles invalid positional data globally for all agencies.

PR created automatically by Jules for task 14159488042253352629 started by @FelixINX

- Added automatic retries and timeout for Http requests in DispatchAgency.php.
- Added validation bounds for out-of-range latitude/longitude, speed, odometer, and bearing in GtfsRtHandler.php, JavascriptGtfsRtHandler.php, and NextbusJsonHandler.php.

Co-authored-by: FelixINX <8009156+FelixINX@users.noreply.github.com>
@google-labs-jules

Copy link
Copy Markdown

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

- Added automatic retries and timeout for Http requests in DispatchAgency.php.
- Added validation bounds for out-of-range latitude/longitude, speed, odometer, and bearing in GtfsRtHandler.php, JavascriptGtfsRtHandler.php, and NextbusJsonHandler.php.
- Added composer_install.log to .gitignore

Co-authored-by: FelixINX <8009156+FelixINX@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant