[integration] Tracetest

[agardnerIT]

Tool / Product Name

tracetest

Goal

I want to use tracetest within a Keptn sequence to validate my open telemetry traces.

#help-integrations on Slack

• Yes

No Existing Integrations

• Yes

Assisted

No

Any other info?

No response

If you need this integration, click the thumbs up emoji 👍 .
If you are willing to "sponsor" (ie. technically support or co-develop) this integration, click the heart emoji ❤️.

[agardnerIT]

Work in progress....

config.yml

scheme: http
endpoint: tracetest.tracetest.svc.cluster.local:8080

---

testdef.yaml

id: b5706f39-d2f6-4a12-ba9e-fe6b3f4732c5
name: POST import pokemon
description: ""
trigger:
  type: http
  httpRequest:
    url: http://demo-pokemon-api.demo/pokemon/import
    method: POST
    headers:
    - key: Content-Type
      value: application/json
    body: '{ "id": 52 }'
  grpc:
    protobufFile: ""
    address: ""
    method: ""
testDefinition:
- selector: span[name = "POST /pokemon/import"]
  assertions:
  - tracetest.span.duration <= 50ms
  - http.status_code = 200
  - tracetest.response.body contains 52
- selector: span[name = "queue.synchronizePokemon send"]
  assertions:
  - messaging.payload contains 52
- selector: span[name = "create pokeshop.pokemon"]
  assertions:
  - messaging.payload contains 52

---

app.py

import json
import subprocess

test_result = subprocess.run([
    'tracetest',
    '--config',
    '/keptn/files/config.yml',
    'test',
    'run',
    '--definition',
    '/keptn/files/testdef.yaml',
    '--wait-for-result'
    ], capture_output=True)

test_result_json = json.loads(test_result.stdout)

check_results = []

for selector_result in test_result_json['testRun']['result']['results']:

    selector_query = selector_result['selector']['query']
    #print(f"Got selector_query: {selector_query}")

    for assertion_result in selector_result['results']:

        assertion_name = assertion_result['assertion']['attribute']
        if "allPassed" in assertion_result:
            #print(f"Assertion: {assertion_name} passed for selector: {selector_query}")
            check_results.append({
                "name": assertion_name,
                "selector": selector_query,
                "status": "Pass"
            })
        else:
            #print(f"Assertion: {assertion_name} failed for selector: {selector_query}")
            check_results.append({
                "name": assertion_name,
                "selector": selector_query,
                "status": "Fail"
            })

-----------------

# Output Results for Checks.
passed_checks = sum(1 for check in check_results if check['status'] == "Pass")
failed_checks = sum(1 for check in check_results if check['status'] == "Fail")
pass_percentage = round(passed_checks / len(check_results) * 100)

print(f"{passed_checks} checks passed")
print(f"{failed_checks} checks failed")
print(f"Pass percentage: {pass_percentage}%")

for check in check_results:
    print(f"Assertion: {check['name']} for selector: {check['selector']} status: {check['status']}")

if pass_percentage == 0:
    print("-------------")
    print("No tests passed, failing the task...")
    # If this script exits with a non-zero exit code, the task will fail as desired
    exit(1)

---

apiVersion: v2
actions:
  - name: "Run tracetest"
    events:
      - name: "sh.keptn.event.test.triggered"
    tasks:
      - name: "Run tracetest"
        #imagePullPolicy: "Always"
        files:
          - '/files'
        image: "gardnera/python:tracetest"
        cmd:
          - "python3"
        args:
          - '/keptn/files/app.py'

[agardnerIT]

Keptn syncs its projects to an upstream Git repo. Here's mine: https://github.com/agardnerIT/keptn-tracetest

Things start with the shipyard. I have decided that tracetest will respond to the test task.

Flip to the dev branch and look at service-1/job/config.yaml this denotes what happens (where service-1 is my microservice). I am running a custom container which is just Python with tracetest inside. I copy the files from /files into the container. The files are available under the /keptn directory inside the container. So files/app.py on Git becomes /keptn/files/app.py inside the container.

The container python /keptn/files/app.py.

The messages and logic above fundamentally just runs tracetest as a Python subprocess. The rest of the script is just fancy parsing so I can fail the Keptn test task only if all my checks fail.

---

What's next?

I'd like a way to push these results to Prometheus (or another backend) like as described here. Then we can use tracetest results in the Keptn quality gate.

@yllek is it possible for tracetest to push metrics out itself without me writing custom Python logic? Something like k6s exporter concept?

[agardnerIT]

Started on v2 logic now: pushing metrics to Prom.

[danielbdias]

hi @agardnerIT ! I'm Daniel from the Tracetest team and we're working on integrating Tracetest into Keptn, we've opened an issue that I believe it could be merged with this issue.

We built documentation on how we can use Tracetest with Keptn as a job, does it make sense to you? Is there any missing feature that we can add to it?

Thanks!