Skip to content

Latency trackers and latency reduction optimizations#12

Open
AuroraH456 wants to merge 4 commits into
mainfrom
aurora/latency-reduction-real
Open

Latency trackers and latency reduction optimizations#12
AuroraH456 wants to merge 4 commits into
mainfrom
aurora/latency-reduction-real

Conversation

@AuroraH456

Copy link
Copy Markdown
Collaborator
  • Adds time duration printout for each stage of processing
  • Before sending the screenshot to OpenAI for screen analysis, this will convert the screenshot from PNG to JPEG and decrease the quality. This decreases latency by 2-4 seconds (out of the original 10-ish) as most of the latency associated with using OpenAI API to analyze the screenshot comes from sending an image over to the OpenAI servers (this fact was found out experimentally through using different models of GPT).
  • Uses a smaller model of GPT (gpt-4.1-nano instead of gpt-4o) for faster speed (screen analysis is fairly simple and shouldn't require a big model). However, this doesn't reduce latency by much as most of the processing time is be spent on sending the screenshot to OpenAI.

@ecoArcGaming

Copy link
Copy Markdown
Collaborator

lgtm 🔥

@AuroraH456

Copy link
Copy Markdown
Collaborator Author

Changed OpenAI TTS to Piper TTS, which decreases latency by 1-2 seconds. Piper TTS needs to be downloaded and run locally on the server; the download command is in the README file. The TTS models should be downloaded in the voice_response folder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants