Trickster returns empty response at instant query with ALB Prometheus

[keithshum]

Hello team,

User reports when running the following "instant" query returns no response, but "range" works.

Possibly related to #936.

Thanks!

[crandles]

#857 includes a potential fix, recommend testing against it

[keithshum]

Hi @crandles

Unlike issue #936, commit 4fe9e60 does not make a different, still seeing empty response.

[keithshum]

Just tested with commit fa8d85e from 970 and still getting empty response.

{
    "results": {
        "A": {
            "status": 200,
            "frames": [
                {
                    "schema": {
                        "refId": "A",
                        "meta": {
                            "typeVersion": [
                                0,
                                0
                            ],
                            "custom": {
                                "calculatedMinStep": 15000
                            },
                            "executedQueryString": "Expr: max_over_time(spectra_service_usage{dom_env=\"prod\", oci_region=\"us-ashburn-1\"}[1h])\n\nStep: 15s"
                        },
                        "fields": []
                    },
                    "data": {
                        "values": []
                    }
                }
            ]
        }
    }
}

[keithshum]

At first I thought max_over_time makes a different, then metric only also returns no response anyway.

[keithshum]

@crandles @jranson

I think it's related to these errors:

time=2026-04-07T22:51:39.597921822Z app=trickster level=error event="vector unmarshaling error" caller=pkg/backends/prometheus/transformations.go:71 detail="unexpected EOF" provider=prometheus
time=2026-04-07T22:51:40.358070159Z app=trickster level=error event="vector unmarshaling error" caller=pkg/backends/prometheus/transformations.go:71 detail="invalid character 'o' in literal null (expecting 'u')" provider=prometheus

These errors show up pretty often at instant query type.

As a side note, I see these errors when testing count_values and some backend members missing from output.

[jranson]

I'm looking at fixing these in #938