As a swatch developer, I'd like swatch to be resilient to latency differences from different data sources. We are currently dependent on assumptions about timing of event records, and using the "timestamp", which is provided by the data source (e.g. prometheus) in our queries. By introducing "created" and "updated" columns, and refactoring our queries to use those, we can make event processing resilient to data latency.

Reference: (see record time at https://martinfowler.com/eaaDev/timeNarrative.html#dimensions )

There are two use cases:

• Normal operation: only new events exist in the last hour, handle with existing event processing logic.
• Update happened (e.g. retally): one or more events updated in the last hour is was created in the previous hour. Because we do not store multiple versions of an event, we must recalculate the affected data (hourly record, daily record, and the monthly totals impacted). This should simply trigger our existing retally logic.

Done:

• Created and updated timestamp columns are maintained for records.
• Existing event records use the timestamp column to populated "created" and "updated" columns.
• Hourly tally queries for events created or updated in the last hour rather than events about the last hour.
• Hourly tally triggers retally logic iff there were updated records created outside the hour's time range.

QE:

Test scenario:

1. Add prometheus data
2. Perform metering
3. This will have updated timestamp as current time
4. Wait for few seconds
5. Perform metering again
6. Updated timestamp changed to latest timestamp and greater than timestamp in step 3.

Add updated timestamp in https://gitlab.cee.redhat.com/smqe/smqe-tools/-/blob/master/smqe_tools/__init__.py#L1605