Loading...

XML

Word

Printable

Type: Bug
Resolution: Unresolved
Priority: Minor
Fix Version/s: None
Affects Version/s: 4.18.z
Component/s: Networking / ptp
Labels:
- Dual
- T-GM
- Telco5G

Activity Type:
Quality / Stability / Reliability
Blocked:
False
Blocked Reason:

Hide

None

Show
None
Story Points:
None
Severity:
Important
Regression:
None
Latest Status Summary:

Hide
8/27: lowered priority to Minor
8/25: QE need to test with just T-GM profile
8/14: reassigned to Jack. analyzing logs
7/3: QE to verify with more backing data

Show
8/27: lowered priority to Minor 8/25: QE need to test with just T-GM profile 8/14: reassigned to Jack. analyzing logs 7/3: QE to verify with more backing data

Target Backport Versions:

4.18.z, 4.19.z, 4.20.z
Target Version:

4.21
Release Blocker:
None
Sprint:
CNF RAN Sprint 277, CNF RAN Sprint 278
sprint_count:
2

RH Private Keywords:

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

PX Impact Score:

Release Note Status:
None
Release Note Type:
None
Release Note Text:
None

Escape Reason:
None
Escape Impact:
None
Corrective Measures:
None
SDLC stage when should've been found:
None

Description of problem:

    Event noise upon T-GM clock locking from freerun

Version-Release number of selected component (if applicable):

    ptp-operator.v4.18.0-202506301635

How reproducible:

    100%

Steps to Reproduce:

    1. Listen to sync status state change events; oc -n openshift-ptp logs ds/linuxptp-daemon cloud-event-proxy --tail 0 -f | grep "event.sync.sync-status.synchronization-state-change"
    2. Cause GNSS loss for T-GM; `oc -n openshift-ptp exec ds/linuxptp-daemon -c linuxptp-daemon-container -- ubxtool -P 29.20 -p COLDBOOT`     
    3. Let the clock enter FREERUN
    4. Allow the GNSS To recover; kill the loop
    5. Upon locking the event noise occurs.

Actual results:
A large array of events, some of which are conflicting

    "event.sync.sync-status.synchronization-state-change"
time="2025-07-02T19:24:38Z" level=debug msg="event sent {\n    \"id\": \"3de41abb-e495-42df-9aa2-b8eebe98370c\",\n    \"type\": \"event.sync.sync-status.synchronization-state-change\",\n    \"source\": \"/sync/sync-status/sync-state\",\
n    \"dataContentType\": \"application/json\",\n    \"time\": \"2025-07-02T19:24:38.000730333Z\",\n    \"data\": {\n      \"version\": \"1.0\",\n      \"values\": [\n        {\n          \"ResourceAddress\": \"/cluster/node/helix65.lab
.eng.rdu2.redhat.com/sync/sync-status/sync-state\",\n          \"data_type\": \"notification\",\n          \"value_type\": \"enumeration\",\n          \"value\": \"LOCKED\"\n        }\n      ]\n    }\n  }"
time="2025-07-02T19:24:38Z" level=debug msg="event sent {\n    \"id\": \"3de41abb-e495-42df-9aa2-b8eebe98370c\",\n    \"type\": \"event.sync.sync-status.synchronization-state-change\",\n    \"source\": \"/sync/sync-status/sync-state\",\
n    \"dataContentType\": \"application/json\",\n    \"time\": \"2025-07-02T19:24:38.904459744Z\",\n    \"data\": {\n      \"version\": \"1.0\",\n      \"values\": [\n        {\n          \"ResourceAddress\": \"/cluster/node/helix65.lab
.eng.rdu2.redhat.com/sync/sync-status/sync-state\",\n          \"data_type\": \"notification\",\n          \"value_type\": \"enumeration\",\n          \"value\": \"LOCKED\"\n        }\n      ]\n    }\n  }"
time="2025-07-02T19:24:39Z" level=debug msg="event sent {\n    \"id\": \"3de41abb-e495-42df-9aa2-b8eebe98370c\",\n    \"type\": \"event.sync.sync-status.synchronization-state-change\",\n    \"source\": \"/sync/sync-status/sync-state\",\
n    \"dataContentType\": \"application/json\",\n    \"time\": \"2025-07-02T19:24:39.030617093Z\",\n    \"data\": {\n      \"version\": \"1.0\",\n      \"values\": [\n        {\n          \"ResourceAddress\": \"/cluster/node/helix65.lab
.eng.rdu2.redhat.com/sync/sync-status/sync-state\",\n          \"data_type\": \"notification\",\n          \"value_type\": \"enumeration\",\n          \"value\": \"LOCKED\"\n        }\n      ]\n    }\n  }"
time="2025-07-02T19:24:41Z" level=debug msg="event sent {\n    \"id\": \"3de41abb-e495-42df-9aa2-b8eebe98370c\",\n    \"type\": \"event.sync.sync-status.synchronization-state-change\",\n    \"source\": \"/sync/sync-status/sync-state\",\
n    \"dataContentType\": \"application/json\",\n    \"time\": \"2025-07-02T19:24:41.655907634Z\",\n    \"data\": {\n      \"version\": \"1.0\",\n      \"values\": [\n        {\n          \"ResourceAddress\": \"/cluster/node/helix65.lab
.eng.rdu2.redhat.com/sync/sync-status/sync-state\",\n          \"data_type\": \"notification\",\n          \"value_type\": \"enumeration\",\n          \"value\": \"LOCKED\"\n        }\n      ]\n    }\n  }"
time="2025-07-02T19:24:41Z" level=debug msg="event sent {\n    \"id\": \"3de41abb-e495-42df-9aa2-b8eebe98370c\",\n    \"type\": \"event.sync.sync-status.synchronization-state-change\",\n    \"source\": \"/sync/sync-status/sync-state\",\
n    \"dataContentType\": \"application/json\",\n    \"time\": \"2025-07-02T19:24:41.969500813Z\",\n    \"data\": {\n      \"version\": \"1.0\",\n      \"values\": [\n        {\n          \"ResourceAddress\": \"/cluster/node/helix65.lab
.eng.rdu2.redhat.com/sync/sync-status/sync-state\",\n          \"data_type\": \"notification\",\n          \"value_type\": \"enumeration\",\n          \"value\": \"LOCKED\"\n        }\n      ]\n    }\n  }"
time="2025-07-02T19:25:29Z" level=debug msg="event sent {\n    \"id\": \"3de41abb-e495-42df-9aa2-b8eebe98370c\",\n    \"type\": \"event.sync.sync-status.synchronization-state-change\",\n    \"source\": \"/sync/sync-status/sync-state\",\
n    \"dataContentType\": \"application/json\",\n    \"time\": \"2025-07-02T19:25:29.95690114Z\",\n    \"data\": {\n      \"version\": \"1.0\",\n      \"values\": [\n        {\n          \"ResourceAddress\": \"/cluster/node/helix65.lab.
eng.rdu2.redhat.com/sync/sync-status/sync-state\",\n          \"data_type\": \"notification\",\n          \"value_type\": \"enumeration\",\n          \"value\": \"HOLDOVER\"\n        }\n      ]\n    }\n  }"
time="2025-07-02T19:25:31Z" level=debug msg="event sent {\n    \"id\": \"3de41abb-e495-42df-9aa2-b8eebe98370c\",\n    \"type\": \"event.sync.sync-status.synchronization-state-change\",\n    \"source\": \"/sync/sync-status/sync-state\",\
n    \"dataContentType\": \"application/json\",\n    \"time\": \"2025-07-02T19:25:31.266169415Z\",\n    \"data\": {\n      \"version\": \"1.0\",\n      \"values\": [\n        {\n          \"ResourceAddress\": \"/cluster/node/helix65.lab
.eng.rdu2.redhat.com/sync/sync-status/sync-state\",\n          \"data_type\": \"notification\",\n          \"value_type\": \"enumeration\",\n          \"value\": \"LOCKED\"\n        }\n      ]\n    }\n  }"
time="2025-07-02T19:25:31Z" level=debug msg="event sent {\n    \"id\": \"3de41abb-e495-42df-9aa2-b8eebe98370c\",\n    \"type\": \"event.sync.sync-status.synchronization-state-change\",\n    \"source\": \"/sync/sync-status/sync-state\",\
n    \"dataContentType\": \"application/json\",\n    \"time\": \"2025-07-02T19:25:31.834544956Z\",\n    \"data\": {\n      \"version\": \"1.0\",\n      \"values\": [\n        {\n          \"ResourceAddress\": \"/cluster/node/helix65.lab
.eng.rdu2.redhat.com/sync/sync-status/sync-state\",\n          \"data_type\": \"notification\",\n          \"value_type\": \"enumeration\",\n          \"value\": \"LOCKED\"\n        }\n      ]\n    }\n  }"
time="2025-07-02T19:25:59Z" level=debug msg="event sent {\n    \"id\": \"3de41abb-e495-42df-9aa2-b8eebe98370c\",\n    \"type\": \"event.sync.sync-status.synchronization-state-change\",\n    \"source\": \"/sync/sync-status/sync-state\",\
n    \"dataContentType\": \"application/json\",\n    \"time\": \"2025-07-02T19:25:59.964667942Z\",\n    \"data\": {\n      \"version\": \"1.0\",\n      \"values\": [\n        {\n          \"ResourceAddress\": \"/cluster/node/helix65.lab
.eng.rdu2.redhat.com/sync/sync-status/sync-state\",\n          \"data_type\": \"notification\",\n          \"value_type\": \"enumeration\",\n          \"value\": \"HOLDOVER\"\n        }\n      ]\n    }\n  }"
time="2025-07-02T19:26:01Z" level=debug msg="event sent {\n    \"id\": \"3de41abb-e495-42df-9aa2-b8eebe98370c\",\n    \"type\": \"event.sync.sync-status.synchronization-state-change\",\n    \"source\": \"/sync/sync-status/sync-state\",\
n    \"dataContentType\": \"application/json\",\n    \"time\": \"2025-07-02T19:26:01.185116778Z\",\n    \"data\": {\n      \"version\": \"1.0\",\n      \"values\": [\n        {\n          \"ResourceAddress\": \"/cluster/node/helix65.lab
.eng.rdu2.redhat.com/sync/sync-status/sync-state\",\n          \"data_type\": \"notification\",\n          \"value_type\": \"enumeration\",\n          \"value\": \"LOCKED\"\n        }\n      ]\n    }\n  }"
time="2025-07-02T19:26:01Z" level=debug msg="event sent {\n    \"id\": \"3de41abb-e495-42df-9aa2-b8eebe98370c\",\n    \"type\": \"event.sync.sync-status.synchronization-state-change\",\n    \"source\": \"/sync/sync-status/sync-state\",\
n    \"dataContentType\": \"application/json\",\n    \"time\": \"2025-07-02T19:26:01.772831817Z\",\n    \"data\": {\n      \"version\": \"1.0\",\n      \"values\": [\n        {\n          \"ResourceAddress\": \"/cluster/node/helix65.lab
.eng.rdu2.redhat.com/sync/sync-status/sync-state\",\n          \"data_type\": \"notification\",\n          \"value_type\": \"enumeration\",\n          \"value\": \"LOCKED\"\n        }\n      ]\n    }\n  }"

Expected results:
A single event

        "event.sync.sync-status.synchronization-state-change" time="2025-07-02T19:24:38Z" level=debug msg="event sent {\n \"id\": \"3de41abb-e495-42df-9aa2-b8eebe98370c\",\n \"type\": \"event.sync.sync-status.synchronization-state-change\",\n \"source\": \"/sync/sync-status/sync-state\",\ n \"dataContentType\": \"application/json\",\n \"time\": \"2025-07-02T19:24:38.000730333Z\",\n \"data\": {\n \"version\": \"1.0\",\n \"values\": [\n {\n \"ResourceAddress\": \"/cluster/node/helix65.lab .eng.rdu2.redhat.com/sync/sync-status/sync-state\",\n \"data_type\": \"notification\",\n \"value_type\": \"enumeration\",\n \"value\": \"LOCKED\"\n }\n ]\n }\n }"

Additional info:

    This noise may increase flakiness of QE tests which listen to this event.

links to

redhat-cne/cloud-event-proxy#598: OCPBUGS-58326: emit overall SyncStateChange using aggregated node state; dedupe dup…

Assignee:: Aneesh Puttur

Reporter:: Daniel Popsuevich

Need Info From:: None

Contributors:: None

QA Contact:: Bonnie Block

Doc Contact:: None

Votes:: 0 Vote for this issue

Watchers:: 6 Start watching this issue

Created:: 2025/07/02 7:53 PM

Updated:: 2025/10/09 2:49 PM

Details

Description

Attachments

Issue Links

Easy Agile Planning Poker

Activity

People

Dates

Hide