[telemetry_chargeback] FVT - Implement data validation capabilities for loki rate#399
[telemetry_chargeback] FVT - Implement data validation capabilities for loki rate#399ayefimov-1 wants to merge 5 commits into
Conversation
|
Build failed (check pipeline). Post ❌ telemetry-openstack-meta-content-provider-master FAILURE in 27m 39s |
|
recheck |
|
Build failed (check pipeline). Post ❌ telemetry-openstack-meta-content-provider-master FAILURE in 35m 57s |
|
recheck |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 50m 10s |
ayefimov-1
force-pushed
the
alexy_ck_job_8
branch
from
924c026 to
1f4c3aa
Compare
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 3h 06m 04s |
ayefimov-1
force-pushed
the
alexy_ck_job_8
branch
from
1f4c3aa to
b5c3e26
Compare
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 50m 24s |
ayefimov-1
force-pushed
the
alexy_ck_job_8
branch
from
b5c3e26 to
0609336
Compare
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 59m 15s |
|
This change depends on a change that failed to merge. Change #388 is needed. |
|
recheck |
|
Merge Failed. This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset. |
|
recheck |
|
Merge Failed. This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset. |
|
recheck |
|
Merge Failed. This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset. |
ayefimov-1
force-pushed
the
alexy_ck_job_8
branch
from
8befed4 to
fe24ad5
Compare
|
Merge Failed. This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset. |
|
recheck |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 37m 44s |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 33m 36s |
|
recheck |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 49m 02s |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 46m 06s |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 52m 09s |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 47m 26s |
|
recheck |
|
recheck |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 57m 12s |
|
Added depends-on Recheck |
|
recheck |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 49m 39s |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 33m 43s |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 38m 54s |
|
Build failed (check pipeline). Post ❌ telemetry-openstack-meta-content-provider-master FAILURE in 20m 02s |
|
recheck |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 46m 16s |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 3h 44m 25s |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 3h 45m 24s |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 3h 06m 18s |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 57m 35s |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 47m 14s |
|
Zuul encountered a syntax error while parsing its Job telemetry-container-image-content-provider not defined The problem appears in the "openstack-k8s-operators/mysqld_exporter" project stanza: project: in "openstack-k8s-operators/mysqld_exporter/.zuul.yaml@main", line 2 |
2 similar comments
|
Zuul encountered a syntax error while parsing its Job telemetry-container-image-content-provider not defined The problem appears in the "openstack-k8s-operators/mysqld_exporter" project stanza: project: in "openstack-k8s-operators/mysqld_exporter/.zuul.yaml@main", line 2 |
|
Zuul encountered a syntax error while parsing its Job telemetry-container-image-content-provider not defined The problem appears in the "openstack-k8s-operators/mysqld_exporter" project stanza: project: in "openstack-k8s-operators/mysqld_exporter/.zuul.yaml@main", line 2 |
|
recheck |
|
Zuul encountered a syntax error while parsing its Job telemetry-container-image-content-provider not defined The problem appears in the "openstack-k8s-operators/mysqld_exporter" project stanza: project: in "openstack-k8s-operators/mysqld_exporter/.zuul.yaml@main", line 2 |
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 53m 51s |
This commit consolidates all enhancements to the telemetry_chargeback role: Major Features: - Implement job-level total rate accumulation across scenarios - Add total_rate_job variable to track cumulative rates - Display accumulated total at job completion - Rename total_rating → total_rate_scenario for clarity CloudKitty Rate Calculation Fixes: - Fix transformation order: conversion BEFORE mutation (per CloudKitty docs) - Add factor/offset unit conversion support (e.g., bytes to GiB) - Implement mutate transformations (CEIL, FLOOR, NUMBOOL, NOTNUMBOOL) - Add fraction parser for factor values (e.g., '1/1048576') Data Generation Improvements: - Remove time_step_gap for exact timestep division - Generate timesteps that exactly divide the time range - Set end_time to 2 hours before current time (not current time) - Fix sequential scenario time ranges (no overlap) Test Scenario Updates: - Update test_dyn_basic.yml: step_seconds 14400→600, add factor conversions - Update test_static.yml: step_seconds 7200→600, add factor conversions - Use realistic CloudKitty unit conversion patterns - Increase limit: 50→1500 to handle more data entries Lint and Validation: - Fix yamllint line length warnings (80 char limit) - Use multi-line YAML formatting (>-) for long lines - Pass ansible-lint production profile - Python PEP8 compliant Files Changed: - defaults/main.yml: Increase limit, add output file paths - vars/main.yml: Add total_rate_job initialization - tasks/main.yml, run_test_scenarios.yml: Add accumulation logic - tasks/loki_total_rate.yml: Add job total display - files/gen_db_summary.py: Fix rate calculation, rename field - files/gen_synth_loki_data.py: Fix timestep generation, add end_time offset - files/test_*.yml: Update step_seconds, add factor fields Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
ayefimov-1
force-pushed
the
alexy_ck_job_8
branch
from
68e97f0 to
f48fd72
Compare
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 40m 24s |
The scenario names in cloudkitty_test_scenarios were defined with .yml extension, but tasks/main.yml strips the extension and vars/main.yml adds it back, resulting in paths like 'test_static.yml.yml'. Fixed by removing .yml extension from scenario names in defaults/main.yml. The regex_replace in tasks/main.yml will now operate on names without extensions, and vars/main.yml will correctly add .yml once. Before: test_static.yml → (strip .yml) → test_static → (add .yml) → test_static.yml ✓ After: test_static → (no match) → test_static → (add .yml) → test_static.yml ✓
This reverts commit 77c41ea.
During the squash, the task that sets cloudkitty_test_file dynamically
was lost. This task overrides the vars/main.yml defaults with
scenario-specific values at runtime.
Without this task, cloudkitty_test_file uses the vars/main.yml default
which includes {{ scenario_name }}.yml, but scenario_name already has
.yml extension from cloudkitty_test_scenarios list, resulting in
double extensions like 'test_static.yml.yml'.
This restores the dynamic variable setting task that was in the
working version, allowing scenario_name to include .yml extension
in the list while the task constructs the correct file path.
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 40m 24s |
The dynamic variable setting task was incorrectly added in commit d3eccc9 but is NOT present in master and is NOT needed. The variables cloudkitty_data_file, cloudkitty_synth_totals_file, and cloudkitty_test_file are already defined in defaults/main.yml and vars/main.yml with {{ scenario_name }} placeholders. Ansible automatically re-evaluates these variables in each loop iteration with the current scenario_name value - no explicit override is needed. This restores gen_synth_loki_data.yml to match master's working implementation and should fix the double .yml extension issue (test_dyn_basic.yml.yml) seen in CI. Master's approach: 1. defaults/vars define variables with {{ scenario_name }} 2. tasks/main.yml regex_replace strips .yml from scenario names 3. Loop sets scenario_name (without .yml) 4. Ansible re-evaluates variables automatically 5. Result: correct paths with single .yml extension
|
Build failed (check pipeline). Post ✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 33m 35s |
1 participant
Data validation capabilities for loki rate
Authored-by: @ayefimov-1
AI Assisted by: Claude