Skip to content

Implement total rate accumulation and enhance telemetry_chargeback role#412

Open
ayefimov-1 wants to merge 2 commits into
masterfrom
alexy_ck_job_8a
Open

Implement total rate accumulation and enhance telemetry_chargeback role#412
ayefimov-1 wants to merge 2 commits into
masterfrom
alexy_ck_job_8a

Conversation

@ayefimov-1

Copy link
Copy Markdown
Contributor

This commit consolidates all enhancements to the telemetry_chargeback role:

Major Features:

  • Implement job-level total rate accumulation across scenarios
  • Add total_rate_job variable to track cumulative rates
  • Display accumulated total at job completion
  • Rename total_rating → total_rate_scenario for clarity

CloudKitty Rate Calculation Fixes:

  • Fix transformation order: conversion BEFORE mutation (per CloudKitty docs)
  • Add factor/offset unit conversion support (e.g., bytes to GiB)
  • Implement mutate transformations (CEIL, FLOOR, NUMBOOL, NOTNUMBOOL)
  • Add fraction parser for factor values (e.g., '1/1048576')

Data Generation Improvements:

  • Remove time_step_gap for exact timestep division
  • Generate timesteps that exactly divide the time range
  • Set end_time to 2 hours before current time (not current time)
  • Fix sequential scenario time ranges (no overlap)

Test Scenario Updates:

  • Update test_dyn_basic.yml: step_seconds 14400→600, add factor conversions
  • Update test_static.yml: step_seconds 7200→600, add factor conversions
  • Use realistic CloudKitty unit conversion patterns
  • Increase limit: 50→1500 to handle more data entries

Lint and Validation:

  • Fix yamllint line length warnings (80 char limit)
  • Use multi-line YAML formatting (>-) for long lines
  • Pass ansible-lint production profile
  • Python PEP8 compliant

Files Changed:

  • defaults/main.yml: Increase limit, add output file paths
  • vars/main.yml: Add total_rate_job initialization
  • tasks/main.yml, run_test_scenarios.yml: Add accumulation logic
  • tasks/loki_total_rate.yml: Add job total display (new file)
  • files/gen_db_summary.py: Fix rate calculation, rename field
  • files/gen_synth_loki_data.py: Fix timestep generation, add end_time offset
  • files/test_*.yml: Update step_seconds, add factor fields

ayefimov-1 and others added 2 commits July 11, 2026 13:35
This commit consolidates all enhancements to the telemetry_chargeback role:

Major Features:
- Implement job-level total rate accumulation across scenarios
- Add total_rate_job variable to track cumulative rates
- Display accumulated total at job completion
- Rename total_rating → total_rate_scenario for clarity

CloudKitty Rate Calculation Fixes:
- Fix transformation order: conversion BEFORE mutation (per CloudKitty docs)
- Add factor/offset unit conversion support (e.g., bytes to GiB)
- Implement mutate transformations (CEIL, FLOOR, NUMBOOL, NOTNUMBOOL)
- Add fraction parser for factor values (e.g., '1/1048576')

Data Generation Improvements:
- Remove time_step_gap for exact timestep division
- Generate timesteps that exactly divide the time range
- Set end_time to 2 hours before current time (not current time)
- Fix sequential scenario time ranges (no overlap)

Test Scenario Updates:
- Update test_dyn_basic.yml: step_seconds 14400→600, add factor conversions
- Update test_static.yml: step_seconds 7200→600, add factor conversions
- Use realistic CloudKitty unit conversion patterns
- Increase limit: 50→1500 to handle more data entries

Lint and Validation:
- Fix yamllint line length warnings (80 char limit)
- Use multi-line YAML formatting (>-) for long lines
- Pass ansible-lint production profile
- Python PEP8 compliant

Files Changed:
- defaults/main.yml: Increase limit, add output file paths
- vars/main.yml: Add total_rate_job initialization
- tasks/main.yml, run_test_scenarios.yml: Add accumulation logic
- tasks/loki_total_rate.yml: Add job total display (new file)
- files/gen_db_summary.py: Fix rate calculation, rename field
- files/gen_synth_loki_data.py: Fix timestep generation, add end_time offset
- files/test_*.yml: Update step_seconds, add factor fields

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The regex_replace in tasks/main.yml was not working reliably in the CI
environment, causing scenario_name to retain the .yml extension.

When vars/main.yml adds .yml to construct cloudkitty_test_file, this
resulted in paths like 'test_dyn_basic.yml.yml'.

Solution:
- Store scenario names WITHOUT .yml extension in defaults/main.yml
- Remove regex_replace from tasks/main.yml (no longer needed)
- vars/main.yml adds .yml when constructing file paths

This is simpler and more reliable than depending on regex_replace.
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/603cb1778cd64792b1c76182fe1dfeda

✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 2h 37m 08s
functional-chargeback-tests-osp18 FAILURE in 1h 50m 28s
✔️ functional-tests-osp18 SUCCESS in 2h 14m 23s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant