-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
4.16
Description of problem:
Version-Release number of selected component (if applicable):
How reproducible:
Steps to Reproduce:
1. Deploy sno with ptp gm config
2. Modify the ptp profile ptpClockThresholds values to trigger FREERUN events
3. Monitor cloud-consumer-deployment pod
Failure occurred after changing ptpClockThresholds to trigger FREERUN:
STEP: Validate no [FREERUN] event received via pod: cloud-consumer-deployment-676b95f989-lmgb5 - /var/lib/jenkins/workspace/ocp-far-edge-vran-tests/cnf-gotests/test/ran/ptp/tests/ptp_events_and_metrics.go:303 @ 06/18/24 15:32:48.278 STEP: Modify the ptp profile ptpClockThresholds values to trigger FREERUN events - /var/lib/jenkins/workspace/ocp-far-edge-vran-tests/cnf-gotests/test/ran/ptp/tests/ptp_events_and_metrics.go:318 @ 06/18/24 15:33:09.306 STEP: Validate clock state changed to [FREERUN] in ptp events via pod: cloud-consumer-deployment-676b95f989-lmgb5 - /var/lib/jenkins/workspace/ocp-far-edge-vran-tests/cnf-gotests/test/ran/ptp/tests/ptp_events_and_metrics.go:323 @ 06/18/24 15:33:09.335
Checked the logs at this step. At 19:30 consumer is showing events:
oc logs -n cloud-events cloud-consumer-deployment-676b95f989-lmgb5 [...] time="2024-06-18T19:30:08Z" level=debug msg="Got CurrentState: {\"specversion\":\"0.3\",\"id\":\"6f0942c6-5cc7-435b-aa7e-1510eed6e0ad\",\"source\":\"/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/gnss-status/gnss-sync-status\",\"type\":\"event.sync.gnss-status.gnss-state-change\",\"subject\":\"/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/gnss-status/gnss-sync-status\",\"datacontenttype\":\"application/json\",\"time\":\"2024-06-18T19:30:08.904030828Z\",\"data\":{\"version\":\"v1\",\"values\":[{\"resource\":\"/cluster/node/helix60.lab.eng.rdu2.redhat.com/ens7fx/master\",\"dataType\":\"notification\",\"valueType\":\"enumeration\",\"value\":\"SYNCHRONIZED\"},{\"resource\":\"/cluster/node/helix60.lab.eng.rdu2.redhat.com/ens7fx/master\",\"dataType\":\"metric\",\"valueType\":\"decimal64.3\",\"value\":\"14\"},{\"resource\":\"/cluster/node/helix60.lab.eng.rdu2.redhat.com/ens7fx/master/gpsFix\",\"d ataType\":\"metric\",\"valueType\":\"decimal64.3\",\"value\":\"5\"}]}} "
At 19:31 the consumer reports error 400 from cloudNotifications:
time="2024-06-18T19:31:03Z" level=info msg="checking for rest service health\n" time="2024-06-18T19:31:03Z" level=info msg="health check http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043/health " time="2024-06-18T19:31:03Z" level=info msg="rest service returned healthy status" time="2024-06-18T19:31:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/sync-status/os-clock-sync-state/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}" time="2024-06-18T19:31:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/ptp-status/ptp-clock-class-change/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}" time="2024-06-18T19:31:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/ptp-status/lock-state/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}" time="2024-06-18T19:31:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/gnss-status/gnss-sync-status/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}" time="2024-06-18T19:32:03Z" level=info msg="checking for rest service health\n" time="2024-06-18T19:32:03Z" level=info msg="health check http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043/health " time="2024-06-18T19:32:03Z" level=info msg="rest service returned healthy status" time="2024-06-18T19:32:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/sync-status/os-clock-sync-state/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}" time="2024-06-18T19:32:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/ptp-status/ptp-clock-class-change/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}" time="2024-06-18T19:32:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/ptp-status/lock-state/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}" time="2024-06-18T19:32:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/gnss-status/gnss-sync-status/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}"
Actual results:
Consumer stops receiving events
Expected results:
Consumer continues to receive events
Additional info:
ptp-operator.v4.16.0-202406131906 Cluster version is 4.16.0-rc.5 kernel 5.14.0-427.22.1.el9_4.x86_64+rt
- blocks
-
OCPBUGS-38950 cloud events consumer reports error 400 and stops receiving events after modifying ptpconfig
-
- Verified
-
- is duplicated by
-
OCPBUGS-38284 V2 subscriptions not cleaned up when consumer is lost
-
- Closed
-
- links to