Hi all,
I'm preparing a practitioner case study applying SCI v1.1.0 to an LLM-tooling-layer intervention (an MCP server that compresses the context delivered to LLM coding assistants, reducing per-task input tokens — and therefore per-task inference energy — by a measured 15–25% on end-to-end agentic workflows and 36–74% on retrieval-only benchmarks against an already-optimised retrieval baseline).
I have read SPEC.md and the open/closed issue history. The spec's methodology for I and M is fully defined and unambiguous on principle. What I'd like working-group input on is a recurring practical obstacle that arises when a meaningful portion of E and M lives at a third-party SaaS API provider (frontier-model providers being the immediate case, but the same pattern applies to any externally-consumed inference, search, or compute API).
1. Disclosure gap on the inference-side grid (I)
The spec is unambiguous: "If the electricity consumption is connected to a grid, the short run marginal, long run marginal, or average emissions grid intensity of that grid shall be used." For an LLM coding assistant, the dominant E term sits at the provider's inference data centre, so I should reflect that grid — not the buyer's grid.
In practice, frontier-model API providers do not expose, on a per-request basis, which data-centre region served the request. The spec's rule is correct; applying it requires data the provider does not currently publish.
Question: is there a working-group preference between (a) using a published average across the provider's known regions, (b) using the provider's stated headline residency (which may not match the actual serving region), or (c) using the buyer's grid as a transparently flagged approximation? Or does the working group consider this a disclosure issue best escalated upstream to the providers rather than papered over in the SCI calculation?
2. Disclosure gap on M for multi-tenant accelerator hardware
The spec fully defines M = TE × (TiR/EL) × (RR/ToR). The methodology is not in question.
The buyer-side practical problem is that, for a single inference call against a frontier-model provider, none of TE (embodied emissions of the accelerator), EL (expected lifespan of the accelerator), or ToR (total resources on the device — relevant when many tenants share a single accelerator) are published. RR and TiR are knowable in principle, but the other three terms collapse the calculation without provider disclosure.
The eShoppen case study under case-studies/ sets a useful precedent: it documents excluded components for which the multiplication factors weren't available and explicitly notes the data was requested from the SCI Open Data project. I'd like to know whether the working group endorses that approach as a general convention for SaaS-consumed components — i.e., omit M with a documented justification ("upstream M not provider-disclosed; reported O-only") — or whether there's a recommended industry-estimate fallback for accelerator-class hardware that the group has converged on.
Thanks for any guidance the group can offer. Happy to share the draft case study privately if useful for grounding the discussion. The intellectual hook is that the per-token energy controversy (estimates spanning roughly an order of magnitude in published sources) cancels out: the per-task SCI ratio is invariant under the choice of per-token energy e, so a measured token reduction translates directly into an SCI reduction without needing to pick a side on the per-token-energy fight. That property makes the LLM-tooling layer an unusually clean fit for the spec, modulo the disclosure gaps above.
Hi all,
I'm preparing a practitioner case study applying SCI v1.1.0 to an LLM-tooling-layer intervention (an MCP server that compresses the context delivered to LLM coding assistants, reducing per-task input tokens — and therefore per-task inference energy — by a measured 15–25% on end-to-end agentic workflows and 36–74% on retrieval-only benchmarks against an already-optimised retrieval baseline).
I have read SPEC.md and the open/closed issue history. The spec's methodology for
IandMis fully defined and unambiguous on principle. What I'd like working-group input on is a recurring practical obstacle that arises when a meaningful portion ofEandMlives at a third-party SaaS API provider (frontier-model providers being the immediate case, but the same pattern applies to any externally-consumed inference, search, or compute API).1. Disclosure gap on the inference-side grid (
I)The spec is unambiguous: "If the electricity consumption is connected to a grid, the short run marginal, long run marginal, or average emissions grid intensity of that grid shall be used." For an LLM coding assistant, the dominant
Eterm sits at the provider's inference data centre, soIshould reflect that grid — not the buyer's grid.In practice, frontier-model API providers do not expose, on a per-request basis, which data-centre region served the request. The spec's rule is correct; applying it requires data the provider does not currently publish.
Question: is there a working-group preference between (a) using a published average across the provider's known regions, (b) using the provider's stated headline residency (which may not match the actual serving region), or (c) using the buyer's grid as a transparently flagged approximation? Or does the working group consider this a disclosure issue best escalated upstream to the providers rather than papered over in the SCI calculation?
2. Disclosure gap on
Mfor multi-tenant accelerator hardwareThe spec fully defines
M = TE × (TiR/EL) × (RR/ToR). The methodology is not in question.The buyer-side practical problem is that, for a single inference call against a frontier-model provider, none of
TE(embodied emissions of the accelerator),EL(expected lifespan of the accelerator), orToR(total resources on the device — relevant when many tenants share a single accelerator) are published.RRandTiRare knowable in principle, but the other three terms collapse the calculation without provider disclosure.The eShoppen case study under
case-studies/sets a useful precedent: it documents excluded components for which the multiplication factors weren't available and explicitly notes the data was requested from the SCI Open Data project. I'd like to know whether the working group endorses that approach as a general convention for SaaS-consumed components — i.e., omitMwith a documented justification ("upstreamMnot provider-disclosed; reportedO-only") — or whether there's a recommended industry-estimate fallback for accelerator-class hardware that the group has converged on.Thanks for any guidance the group can offer. Happy to share the draft case study privately if useful for grounding the discussion. The intellectual hook is that the per-token energy controversy (estimates spanning roughly an order of magnitude in published sources) cancels out: the per-task SCI ratio is invariant under the choice of per-token energy
e, so a measured token reduction translates directly into an SCI reduction without needing to pick a side on the per-token-energy fight. That property makes the LLM-tooling layer an unusually clean fit for the spec, modulo the disclosure gaps above.