Possibility to increase default timeout on http-operations

[Almewty]

I've been using this with a local LLM model on my machine for a bit now and it works well for me.
But my local LLM is slow, so slow that the call times out sometimes, the LLM did not stop responding or crash, it simply didn't finish yet.

Is it possible to increase the timeout? I did not find a way to do this elegantly.

This is the error in the log, I am using 1.14.0

I/O error on POST request for "http://xxxx:8080/v1/chat/completions": Read timed outorg.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://xxxx:8080/v1/chat/completions": Read timed out at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.createResourceAccessException(DefaultRestClient.java:776) at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.exchangeInternal(DefaultRestClient.java:624) at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.exchange(DefaultRestClient.java:572) at org.springframework.web.client.RestClient$RequestHeadersSpec.exchange(RestClient.java:747) at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.executeAndExtract(DefaultRestClient.java:921) at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.body(DefaultRestClient.java:830) at org.remus.giteabot.ai.openai.OpenAiClient.executeRequest(OpenAiClient.java:247) at org.remus.giteabot.ai.openai.OpenAiClient.chatWithTools(OpenAiClient.java:101) at org.remus.giteabot.ai.AuditingAiClient.lambda$chatWithTools$3(AuditingAiClient.java:75) at org.remus.giteabot.ai.AuditingAiClient.audited(AuditingAiClient.java:81) at org.remus.giteabot.ai.AuditingAiClient.chatWithTools(AuditingAiClient.java:75) at org.remus.giteabot.agent.loop.AgentLoop.callAiWithRetry(AgentLoop.java:324) at org.remus.giteabot.agent.loop.AgentLoop.run(AgentLoop.java:91) at org.remus.giteabot.prworkflow.agentreview.AgentReviewService.runReviewLoop(AgentReviewService.java:427) at org.remus.giteabot.prworkflow.agentreview.AgentReviewService.reviewPullRequest(AgentReviewService.java:166) at org.remus.giteabot.prworkflow.agentreview.AgentReviewWorkflow.run(AgentReviewWorkflow.java:144) at org.remus.giteabot.prworkflow.PrWorkflowOrchestrator.run(PrWorkflowOrchestrator.java:146) at org.remus.giteabot.prworkflow.PrWorkflowOrchestrator.run(PrWorkflowOrchestrator.java:104) at org.remus.giteabot.prworkflow.PrWorkflowOrchestrator.runAll(PrWorkflowOrchestrator.java:82) at org.remus.giteabot.admin.BotWebhookService.reviewPullRequest(BotWebhookService.java:112) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:359) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:158) at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:112) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at org.remus.giteabot.ai.AiAuditContextClearingTaskDecorator.lambda$decorate$0(AiAuditContextClearingTaskDecorator.java:26) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source)Caused by: java.net.SocketTimeoutException: Read timed out at java.base/sun.nio.ch.NioSocketImpl.timedRead(Unknown Source) at java.base/sun.nio.ch.NioSocketImpl.implRead(Unknown Source) at java.base/sun.nio.ch.NioSocketImpl.read(Unknown Source) at java.base/sun.nio.ch.NioSocketImpl$1.read(Unknown Source) at java.base/java.net.Socket$SocketInputStream.read(Unknown Source) at org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149) at org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280) at org.apache.hc.core5.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:250) at org.apache.hc.core5.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:56) at org.apache.hc.core5.http.impl.io.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:331) at org.apache.hc.core5.http.impl.io.HttpRequestExecutor.execute(HttpRequestExecutor.java:196) at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.lambda$execute$0(InternalExecRuntime.java:236) at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$InternalConnectionEndpoint.execute(PoolingHttpClientConnectionManager.java:798) at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.execute(InternalExecRuntime.java:233) at org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:120) at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:200) at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:195) at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:168) at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:112) at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:110) at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87) at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:55) at org.apache.hc.client5.http.classic.HttpClient.executeOpen(HttpClient.java:183) at org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:99) at org.springframework.http.client.AbstractStreamingClientHttpRequest.executeInternal(AbstractStreamingClientHttpRequest.java:87) at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:80) at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.exchangeInternal(DefaultRestClient.java:614) ... 29 more

[tmseidel]

@Almewty thanks for the issue. Indeed, currently there is no possibility to set a timeout, we have to introduce a global-setting to overwrite the default timeout value. Scope is globally (easiest), which impacts all timeouts for all AI-Providers and GIT-Providers.

[tmseidel]

Implemenation nodes
Currently we instantiate the RestClients not via Spring, so the Spring's autoconfiguration won't be applied. The change would be to let Spring instantiate our requried RestClient-Instannces (via Spring's ObjectProvider), so that we can use the autoconfiguration with spring and their possibility to overwrite its defaults, e.g. in the docker-compose.yml.
The new docker-env-variables are:
HTTP_CLIENT_CONNECT_TIMEOUT (default 10s) and
HTTP_CLIENT_READ_TIMEOUT (default 120s)

[tmseidel]

Verified in 1.15.0