Skip to content

feat: ✨ improve image fetching speeds#171

Merged
highesttt merged 2 commits into
mainfrom
170-feat-improve-image-fetching-speeds
Jun 4, 2026
Merged

feat: ✨ improve image fetching speeds#171
highesttt merged 2 commits into
mainfrom
170-feat-improve-image-fetching-speeds

Conversation

@highesttt

Copy link
Copy Markdown
Collaborator

No description provided.

@highesttt highesttt linked an issue Jun 4, 2026 that may be closed by this pull request
@indent-zero

indent-zero Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor
PR Summary

Speeds up LINE media fetching across image, audio, video, and file handlers by requesting the "original" OBS variant for plain (non-E2EE) media and forwarding the server-provided OBS_POP token as a ?p= query param to hit OBS's cache key directly. Also adds context-cancellable retry waits, a centralized obsHTTPClient() helper, per-stage timing logs for images, and fixes a 202/404 retry-exhaustion error reporting bug.

  • Added OBSDownloadOptions{TID, OBSPop} plus new DownloadOBSWithOptions / DownloadOBSWithSIDOptions entrypoints; legacy DownloadOBS / DownloadOBSWithSID retained as zero-option wrappers
  • New shared lineOBSDownloadOptions(metadata, isPlainMedia) helper now feeds image, audio, video, and file converters so all four benefit from the original-variant + OBS_POP optimization
  • Image success log now reports matrix_media_url (prefers file.URL for encrypted rooms, falls back to mxc) instead of a frequently-empty mxc; redundant full-ContentMetadata dump removed in favor of a single media_category field
  • Routed all OBS upload/download calls through a new obsHTTPClient() helper with OBSClient → HTTPClient → fresh client fallback; OBSClient retains its previous unbounded timeout so large downloads aren't capped
  • Promoted magic numbers to constants (rpcClientTimeout, obsRetryDelay, obsMaxRetries), replaced the 202/404 retry time.Sleep with a select that honors ctx.Done(), and fixed the final-attempt branch so 202/404 exhaustion now returns the dedicated "still processing after N retries" error instead of a generic non-200 message

Issues

All clear! No issues remaining. 🎉

4 issues already resolved
  • The new "Successfully uploaded image" log prints mxc.ParseOrIgnore().String(), but intent.UploadMedia returns an empty mxc for encrypted Matrix rooms (the URL lives in file), so this field is blank for the majority of bridge users and makes the log misleading. (fixed by commit 9ebf55e)
  • Only ConvertImage was wired up to the new OBSDownloadOptions; video/audio/file handlers still call DownloadOBSWithSID and won't forward OBS_POP or request the original TID, so they don't benefit from the same speed-up. (fixed by commit 9ebf55e)
  • OBSClient timeout drops from unbounded to 5 minutes, which can prematurely fail very large video/file downloads on slow links since video/audio/file handlers share this client; images comfortably fit but the broader OBS surface area is affected. (fixed by commit 9ebf55e)
  • The new pre-download debug log uses Interface("content_metadata", data.ContentMetadata), which dumps the entire metadata map (including the server-issued OBS_POP token and other duplicates of the broken-out fields like oid/tid/plain_media). (fixed by commit 9ebf55e)

CI Checks

All CI checks passed on commit 9ebf55e.

@coderabbitai

coderabbitai Bot commented Jun 4, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0e0b691c-3c32-479c-98ba-1a2c8ea57ded

📥 Commits

Reviewing files that changed from the base of the PR and between e97168c and 9ebf55e.

📒 Files selected for processing (5)
  • pkg/connector/handlers/audio.go
  • pkg/connector/handlers/file.go
  • pkg/connector/handlers/image.go
  • pkg/connector/handlers/video.go
  • pkg/line/client.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/connector/handlers/image.go
📜 Recent review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: Lint with 1.25
  • GitHub Check: build-docker
  • GitHub Check: Lint with 1.25
  • GitHub Check: build-docker
🧰 Additional context used
📓 Path-based instructions (2)
**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

**/*.go: Use go fmt for code formatting across all Go files
Use goimports with -local "github.com/highesttt/matrix-line-messenger" flag to group project-local imports correctly
Use zerolog for logging throughout the codebase
Do not use Msgf in logging; use Msg with structured fields instead
Use Stringer interface where applicable in Go code

Files:

  • pkg/connector/handlers/video.go
  • pkg/connector/handlers/file.go
  • pkg/connector/handlers/audio.go
  • pkg/line/client.go
**/!(ltsm)/**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

**/!(ltsm)/**/*.go: Run staticcheck on all Go files excluding pkg/ltsm package (transpiled WASM code)
Run go vet on all Go files excluding pkg/ltsm package (transpiled WASM code)

Files:

  • pkg/connector/handlers/video.go
  • pkg/connector/handlers/file.go
  • pkg/connector/handlers/audio.go
  • pkg/line/client.go
🔇 Additional comments (4)
pkg/line/client.go (1)

47-47: LGTM!

Also applies to: 59-59, 726-730

pkg/connector/handlers/audio.go (1)

47-48: LGTM!

Also applies to: 52-52

pkg/connector/handlers/file.go (1)

38-39: LGTM!

pkg/connector/handlers/video.go (1)

48-48: LGTM!

Also applies to: 50-50, 54-54


📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Downloads now accept extended options for variant selection and server-side parameters.
    • OBS downloads use a retry mechanism to improve success rates.
  • Improvements

    • Added timing metrics and richer logs for image/media processing (download/decrypt/upload).
    • Media handlers (image, audio, video, file) use the new download options.
    • OBS client tuned with improved timeout behavior for more reliable transfers.

Walkthrough

This PR extends the LINE OBS client with configurable download options and dedicated HTTP client routing, then instruments image handling to measure operation timings and log structured metrics. New OBS download methods accept TID and OBSPop parameters with improved context-aware retry logic; upload operations are routed through a dedicated OBS HTTP client; and image conversion now tracks and reports download, decryption, and upload durations.

Changes

OBS Download Options and Image Instrumentation

Layer / File(s) Summary
OBS client imports, constants, and HTTP routing
pkg/line/client.go
Adds net/url import, rpcClientTimeout, obsRetryDelay, obsMaxRetries; NewClient creates RPC and OBS HTTP clients and obsHTTPClient() selects the OBS client.
OBS upload routing to obsHTTPClient
pkg/line/client.go
Routes UploadOBSPlain, UploadOBSWithSID, and UploadOBSWithOIDAndSID through obsHTTPClient() instead of HTTPClient.
OBS download API with options and retry
pkg/line/client.go
Adds OBSDownloadOptions {TID, OBSPop}, DownloadOBSWithOptions, and DownloadOBSWithSIDOptions; builds OBS URL with optional path-escaped TID and p=<OBSPop> query, acquires encrypted token, and implements context-aware retry for 202/404 via obsHTTPClient().
Image handler: download, timing, and helpers
pkg/connector/handlers/image.go
ConvertImage derives OBSDownloadOptions (including OBSPop and TID="original" for plain original media), measures download_duration, decrypt_duration, and upload_duration, includes durations in error and success logs, and adds lineMediaCategory and lineOBSDownloadOptions helpers.
Audio/File/Video handlers: use OBSDownloadOptions
pkg/connector/handlers/{audio,file,video}.go
ConvertAudio, ConvertFile, and ConvertVideo compute lineOBSDownloadOptions(metadata,isPlain) and call the new DownloadOBSWithSIDOptions, reusing the same options in recovery retries.

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 3

❌ Failed checks (1 warning, 2 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title claims to improve image fetching speeds, but the changes actually implement OBS download options and timing metrics across multiple media handlers (image, audio, video, file). The title is vague and doesn't accurately reflect the scope of modifications. Consider a more descriptive title like 'feat: add OBS download options and timing metrics' to better reflect the actual changes across multiple media types.
Description check ❓ Inconclusive No pull request description was provided by the author, making it impossible to assess whether the intended changes are properly documented. Add a meaningful pull request description explaining the purpose of OBS download options, timing metrics implementation, and how these changes improve the media handling pipeline.
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 170-feat-improve-image-fetching-speeds

Comment @coderabbitai help to get the list of available commands and usage tips.

Comment thread pkg/line/client.go Outdated
Comment thread pkg/connector/handlers/image.go Outdated
Comment thread pkg/connector/handlers/image.go Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
pkg/line/client.go (1)

727-752: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Handle exhausted 202/404 retries before the generic non-200 branch.

Line 727 only retries while attempt < obsMaxRetries. On the last 202/404, execution falls through to Line 737 and returns the generic HTTP error instead, so Line 752 is unreachable and callers never get the intended “still processing after N retries” failure.

Suggested fix
-		if (resp.StatusCode == 202 || resp.StatusCode == 404) && attempt < obsMaxRetries {
-			resp.Body.Close()
-			select {
-			case <-time.After(obsRetryDelay):
-			case <-ctx.Done():
-				return nil, ctx.Err()
-			}
-			continue
-		}
+		if resp.StatusCode == 202 || resp.StatusCode == 404 {
+			resp.Body.Close()
+			if attempt == obsMaxRetries {
+				return nil, fmt.Errorf("OBS download failed: media still processing after %d retries", obsMaxRetries)
+			}
+			select {
+			case <-time.After(obsRetryDelay):
+			case <-ctx.Done():
+				return nil, ctx.Err()
+			}
+			continue
+		}
@@
-	return nil, fmt.Errorf("OBS download failed: media still processing after %d retries", obsMaxRetries)
+	return nil, fmt.Errorf("OBS download failed: media still processing after %d retries", obsMaxRetries)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/line/client.go` around lines 727 - 752, The 202/404 handling currently
only retries when attempt < obsMaxRetries, but on the final 202/404 it falls
through into the generic non-200 branch so the "still processing after N
retries" error is never returned; update the response-status handling block (the
branch that checks resp.StatusCode == 202 || resp.StatusCode == 404) to
explicitly handle the exhausted-retries case: close resp.Body and return an
error like "OBS download failed: media still processing after %d retries" when
attempt >= obsMaxRetries, otherwise sleep on obsRetryDelay and continue
retrying; keep use of obsMaxRetries, obsRetryDelay, ctx, attempt and the
surrounding OBS download logic intact.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/connector/handlers/image.go`:
- Around line 41-48: The current log call dumps the entire data.ContentMetadata
map (Interface("content_metadata", data.ContentMetadata)) which exposes vendor
metadata; remove that Interface field and instead extract and log only the
specific metadata keys you need (e.g., media type, filename, size or other
derived fields) by safely reading from data.ContentMetadata (check types and
existence) and add them as individual Str/Int/Bool fields in the same
h.Log.Debug() chain; update the logging sequence in the handler where the
Debug() is built (the chain that includes oid, data.ID, downloadOptions.TID,
etc.) to include only those explicit metadata fields or omit metadata entirely.

---

Outside diff comments:
In `@pkg/line/client.go`:
- Around line 727-752: The 202/404 handling currently only retries when attempt
< obsMaxRetries, but on the final 202/404 it falls through into the generic
non-200 branch so the "still processing after N retries" error is never
returned; update the response-status handling block (the branch that checks
resp.StatusCode == 202 || resp.StatusCode == 404) to explicitly handle the
exhausted-retries case: close resp.Body and return an error like "OBS download
failed: media still processing after %d retries" when attempt >= obsMaxRetries,
otherwise sleep on obsRetryDelay and continue retrying; keep use of
obsMaxRetries, obsRetryDelay, ctx, attempt and the surrounding OBS download
logic intact.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d75fbacd-b9d0-419a-a3ab-872f13de69ef

📥 Commits

Reviewing files that changed from the base of the PR and between 663cd34 and e97168c.

📒 Files selected for processing (2)
  • pkg/connector/handlers/image.go
  • pkg/line/client.go
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: build-docker
  • GitHub Check: Lint with 1.25
  • GitHub Check: build-docker
  • GitHub Check: Lint with 1.25
🧰 Additional context used
📓 Path-based instructions (2)
**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

**/*.go: Use go fmt for code formatting across all Go files
Use goimports with -local "github.com/highesttt/matrix-line-messenger" flag to group project-local imports correctly
Use zerolog for logging throughout the codebase
Do not use Msgf in logging; use Msg with structured fields instead
Use Stringer interface where applicable in Go code

Files:

  • pkg/connector/handlers/image.go
  • pkg/line/client.go
**/!(ltsm)/**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

**/!(ltsm)/**/*.go: Run staticcheck on all Go files excluding pkg/ltsm package (transpiled WASM code)
Run go vet on all Go files excluding pkg/ltsm package (transpiled WASM code)

Files:

  • pkg/connector/handlers/image.go
  • pkg/line/client.go

Comment thread pkg/connector/handlers/image.go
@highesttt highesttt merged commit 7175614 into main Jun 4, 2026
10 checks passed
@highesttt highesttt deleted the 170-feat-improve-image-fetching-speeds branch June 4, 2026 06:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

feat: ✨ improve image fetching speeds

1 participant