feat(instances): cache server lookups to reduce API calls

[lukasmetzner]

Feature is marked as experimental.

[codecov]

Codecov Report

❌ Patch coverage is 80.21978% with 36 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.76%. Comparing base (547798d) to head (5245ef7).

Files with missing lines	Patch %	Lines
internal/servercache/evalservercache.go	29.16%	17 Missing ⚠️
internal/config/config.go	27.27%	6 Missing and 2 partials ⚠️
internal/servercache/perservercache.go	86.79%	6 Missing and 1 partial ⚠️
hcloud/cloud.go	81.81%	1 Missing and 1 partial ⚠️
internal/servercache/allservercache.go	96.29%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1252      +/-   ##
==========================================
- Coverage   71.01%   67.76%   -3.26%     
==========================================
  Files          24       28       +4     
  Lines        2677     2835     +158     
==========================================
+ Hits         1901     1921      +20     
- Misses        604      740     +136     
- Partials      172      174       +2     

Flag	Coverage Δ
e2e	?
unit	67.76% <80.21%> (+0.70%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jooola]

I was hoping to have a single cache implementation, where the mode could be picked on the fly (or at least it would be easy to do so).

My idea was that both cache "modes" use a cache entry struct with an expiration date (as it is implement in the per server cache), and decide on the refresh actions to perform based on the mode: if mode is per server, only refresh that server, if the mode is all, refresh all servers.

This would mean with the mode "all", during a refresh iterating over all cache entries to update the expiredAt property.

With this in mind, we could use the same cache for both modes, and even add options in the ByName or ByID methods, to pass e.g. cache.ByName(ctx, "my-server", WithMode("all"), WithExpiration(5 *time.Second))

[lukasmetzner]

I like the idea, I will take a look and explore the implementation detail of it.