Clarify Telemetry Policy update behaviour for sets of policies

[jackshirazi]

What are you trying to achieve?

This relates to Telemetry Policy.

I have some types of updates I need to clarify. I'll state the updates and follow with examples to explain the update conflicts and how I see the spec applies. Relevant extracts from the spec are given in the Additional context section. In the examples, X, Y, Z are policy IDs.

1. Absence of a previously applied policy means reset that policy to default (or unset it)
   • Example: Provider A sends policies X, Y and Z as the update of the first set of policies, then a subsequent update also from provider A only has Y and Z.
   • Example summary: update 1: providerA(X, Y, Z); later update 2: providerA(Y,Z)
   • Outcome: my interpretation is that on update 2, policy X is now unset, ie X should be reset to its default
   • Explanation: the "full set replacement" semantics implies any policy missing from the set is equivalent to un-setting the policy
2. Can a set of policies from one source override a set of policies from a higher priority source, ie is the prioritization only among policies in the same received update or for all updates all together
   • Example: the Opamp provider sends policies X, Y and Z, then the Http provider subsequently sends policies Y and Z
   • Example summary: update 1: opamp(X, Y, Z); later update 2: http(Y,Z)
   • Outcome: my interpretation is that on update 2, policy Y and Z from the http source are dropped, so policy X, Y and Z from the update 1 opamp source remain
   • Alternative outcome: If prioritization applies separately to each update (eg both opamp and http providers could send updates at the same time and that would be one update) my interpretation would be that on update 2, policy Y and Z from the http source are applied, while policy X from the update 1 opamp source remains. This seems to be a possible interpretation, but I think not the intended one?
   • Explanation: the prioritisation applies to all policies across updates, so the http update being a lower priority policy update for policy IDs that are already applied, get dropped.
3. Assuming policy prioritization is global across the whole lifetime (ie the "Outcome" from point 2 above is correct, and the "Alternative outcome" is not valid), then are previously dropped policies applicable if a policy is unset
   • Example: the Opamp provider sends policies X, Y and Z, then the Http provider subsequently sends policies Y and Z, then the Opamp provider sends policies X and Y (so unsetting Z)
   • Example summary: update 1: opamp(X, Y, Z); later update 2: http(Y,Z); even later update 3: opamp(X, Y);
   • Outcome: my interpretation is that on update 3, opamp policy Z is reset to default
   • Alternative outcome: the http(Y,Z) from update 2 although not applied when update 2 is received, are retained as potential applicable state, which means that when opamp policy Z is unset, http policy Z is now applied
   • Explanation: "dropped" suggests policies are not expected to be retained for later application
4. A set of policies from a same priority but different source that excludes a previously applied policy does NOT reset that policy
   • Example: the Http(url1) provider sends policies X, Y and Z, then the Http(url2) provider subsequently sends policies Y and Z
   • Example summary: update 1: http1(X, Y, Z); later update 2: http2(Y,Z)
   • Outcome: my interpretation is that on update 2, policy Y and Z from Http(url2) are now applied and considered to be from the Http(url2) source, while policy X remains applied and remains from Http(url1) source
   • Explanation: This last interpretation assumes that different providers are not bucketed into one provider category, ie Http(url1) and Http(url2) are considered different provider categories. This needs clarifying, is “provider” a provider type/category, or each configured provider instance/source. Are Http(url1) and Http(url2) separate provider instances/scopes. If both currently supply Y, what is the tie-breaker (assuming no merge is possible for example sampling rate is set to 50% in one and 10% in the other). Does http2(Y,Z) remove only http2’s prior snapshot, or all HTTP policies?

I need clarification on whether my interpretations are correct, and probably we should update the spec to include the clarifications

Additional context.

The Telemetry Policy recommends full set replacement:

Prefer full-set replacement over patching. ... A provider
SHOULD send the full list of active policies and the client SHOULD atomically
replace its local set.

Policies are scoped to the provider:

Report policy status back to the provider. Transport protocols SHOULD
provide a mechanism for clients to report per-policy status (match counts,
errors) back to the provider. This feedback loop enables providers to detect
misconfigured or ineffective policies. Status SHOULD be scoped to each provider
— a provider only receives status for the policies it supplies.

And prioritisation applies depending on the provider source

Resolve duplicate policy IDs by provider priority. When multiple providers
supply a policy with the same ID, the client must decide which one to keep.
Implementations SHOULD assign each provider a priority — for example, OpAMP (1),
HTTP (2), FILE (3), CUSTOM (user-defined) — where a lower number is higher
priority.

[jackshirazi]

@jaronoff97 @jsuereth @dashpole @menderico