-
Task
-
Resolution: Unresolved
-
Normal
-
None
-
None
-
None
At the moment, we're using Kafka Streams when aggregating billable usages.
The workflow is:
1.- The tally service produces messages containing a number of tally snapshots into the topic "platform.rhsm-subscriptions.billable-usage":
{"severity":"INFO","logger":"org.candlepin.subscriptions.tally.SnapshotSummaryProducer","thread":"rhsm-subscriptions-task-processor-0-C-1","message":"2024-09-09T23:10:24.794+0000 [thread=rhsm-subscriptions-task-processor-0-C-1] [level=INFO] [category=org.candlepin.subscriptions.tally.SnapshotSummaryProducer] - Produced 1 TallySummary messages\n","properties":{"trace_id":"4beae9a48365393b8ec4decc4c2a934f","trace_flags":"01","span_id":"50f747eb8e947021"}}
2.- Then, these messages are consumed by Kafka Streams in the billable usage service:
{"severity":"INFO","message":"2024-09-09 23:00:21,202 INFO [org.can.sub.bil.usa.BillableUsageAggregate] (platform.rhsm-subscriptions.swatch-billable-usage-aggregator-2122ca6f-eca9-4a24-97a5-7604e463c429-StreamThread-1) Adding billableUsage: org.candlepin.subscriptions.billable.usage.BillableUsage@2e99447f[uuid=378c75c2-4ffb-474c-beda-52020322b77d,orgId=11618340,tallyId=839000f8-83bf-4cfe-a5f8-2b2e83c95a92,billingProvider=aws,billingAccountId=204833053756,snapshotDate=2024-09-06T12:00:55.554174Z,productId=rhel-for-x86-els-payg-addon,sla=Premium,usage=Production,status=failed,errorCode=<null>,billedOn=<null>,metricId=vCPUs,value=2822.0,billingFactor=<null>,vendorProductCode=<null>,hardwareMeasurementType=PHYSICAL,currentTotal=<null>] to aggregate with aggregateId: eada50b6-b4da-45c3-b298-40b42458679c, totalValue:164506.0, remittanceUuids:... and windowTimestamp: 2024-09-09T23:00Z\n"}
Note that the "trace_id" header from the first message is missed / not propagrated.
When using Quarkus Smallrye Messaging Kafka consumers, this works out of the box, but using Kafka Streams, this seems not to be supported.
Why is this very important? Propagate the "trace_id" header is a must to see how the workflow went from end to end (from the tally service to the final swatch producer aws/azure service).
Done
- We would like to have a unit test that validates that the headers are being set appropriately