Add support for memcached extstore

[seanankenbruck]

Are there plans to add support for memcached extstore to the Mimir caching components? This feature has been enabled for Loki and is briefly referenced in PR #9149 , but it doesn't look like it has been fully integrated into the helm chart as of yet. In Loki, it is as simple as enabling memcachedExporter and specifying persistence on the relevant cache components. This then adds two additional arguments to the memcached container in the cache statefulset.

ext_path=/data/file:<size>G
ext_wbuf_size=<buffer size>

I've attempted to replicate this configuration by setting memcachedExporter to enabled and using the available volumeClaimTemplates and extraVolumeMounts parameters but the container arguments are not added. I guess these could be added manually to the rendered helm chart, but it would be ideal if they were automatically updated.

[seanankenbruck]

If the above is feasible, I'd be happy to work on a PR to add support for it.

[martensson]

This would be a very nice addition. We are currently using ext_path in Grafana Loki with large disks and it increased the query performance by a lot. Would be very nice to see if this could work for Mimir as well

[56quarters]

We have an internal issue to look at extstore but it hasn't been a priority for a few reasons:

• Our cache usage is different than Loki, we tend to cache far less data
• There are a lot of operational concerns to think about (how to provision the SSDs across all CSPs, exclusive node pools, bandwidth requirements, etc.)
• As it is now, we (Grafana Cloud) have a pretty good cache hit rate with our existing cache setup without it costing too much

If you'd like to work on a PR to add support for extended Memcached options in the Helm chart so that you can experiment with it yourself, that'd be great. However, we don't want to add an "enable_extstore = true" style option yet since we're not prepared to support it.