Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-38950

cloud events consumer reports error 400 and stops receiving events after modifying ptpconfig

XMLWordPrintable

    • Moderate
    • Yes
    • CNF RAN Sprint 258
    • 1
    • False
    • Hide

      None

      Show
      None
    • N/A
    • Release Note Not Required
    • Done
    • 8/22: Affecting QE using Consumer APP.Asked more data from automation team

      This is a clone of issue OCPBUGS-35799. The following is the description of the original issue:
      —
      Description of problem:

       

      Version-Release number of selected component (if applicable):

       

      How reproducible:

       

      Steps to Reproduce:

      1. Deploy sno with ptp gm config
      2.  Modify the ptp profile ptpClockThresholds values to trigger FREERUN events
      3. Monitor cloud-consumer-deployment pod

      Failure occurred after changing ptpClockThresholds to trigger FREERUN:

      STEP: Validate no [FREERUN] event received via pod: cloud-consumer-deployment-676b95f989-lmgb5 - /var/lib/jenkins/workspace/ocp-far-edge-vran-tests/cnf-gotests/test/ran/ptp/tests/ptp_events_and_metrics.go:303 @ 06/18/24 15:32:48.278
  STEP: Modify the ptp profile ptpClockThresholds values to trigger FREERUN events - /var/lib/jenkins/workspace/ocp-far-edge-vran-tests/cnf-gotests/test/ran/ptp/tests/ptp_events_and_metrics.go:318 @ 06/18/24 15:33:09.306
  STEP: Validate clock state changed to [FREERUN] in ptp events via pod: cloud-consumer-deployment-676b95f989-lmgb5 - /var/lib/jenkins/workspace/ocp-far-edge-vran-tests/cnf-gotests/test/ran/ptp/tests/ptp_events_and_metrics.go:323 @ 06/18/24 15:33:09.335

      Checked the logs at this step.  At 19:30 consumer is showing events:

      oc logs -n cloud-events cloud-consumer-deployment-676b95f989-lmgb5
[...]
time="2024-06-18T19:30:08Z" level=debug msg="Got CurrentState: {\"specversion\":\"0.3\",\"id\":\"6f0942c6-5cc7-435b-aa7e-1510eed6e0ad\",\"source\":\"/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/gnss-status/gnss-sync-status\",\"type\":\"event.sync.gnss-status.gnss-state-change\",\"subject\":\"/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/gnss-status/gnss-sync-status\",\"datacontenttype\":\"application/json\",\"time\":\"2024-06-18T19:30:08.904030828Z\",\"data\":{\"version\":\"v1\",\"values\":[{\"resource\":\"/cluster/node/helix60.lab.eng.rdu2.redhat.com/ens7fx/master\",\"dataType\":\"notification\",\"valueType\":\"enumeration\",\"value\":\"SYNCHRONIZED\"},{\"resource\":\"/cluster/node/helix60.lab.eng.rdu2.redhat.com/ens7fx/master\",\"dataType\":\"metric\",\"valueType\":\"decimal64.3\",\"value\":\"14\"},{\"resource\":\"/cluster/node/helix60.lab.eng.rdu2.redhat.com/ens7fx/master/gpsFix\",\"d
ataType\":\"metric\",\"valueType\":\"decimal64.3\",\"value\":\"5\"}]}} "

      At 19:31 the consumer reports error 400 from cloudNotifications:

      time="2024-06-18T19:31:03Z" level=info msg="checking for rest service health\n"
time="2024-06-18T19:31:03Z" level=info msg="health check http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043/health "
time="2024-06-18T19:31:03Z" level=info msg="rest service returned healthy status"
time="2024-06-18T19:31:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/sync-status/os-clock-sync-state/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}"
time="2024-06-18T19:31:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/ptp-status/ptp-clock-class-change/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}"
time="2024-06-18T19:31:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/ptp-status/lock-state/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}"
time="2024-06-18T19:31:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/gnss-status/gnss-sync-status/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}"
time="2024-06-18T19:32:03Z" level=info msg="checking for rest service health\n"
time="2024-06-18T19:32:03Z" level=info msg="health check http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043/health "
time="2024-06-18T19:32:03Z" level=info msg="rest service returned healthy status"
time="2024-06-18T19:32:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/sync-status/os-clock-sync-state/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}"
time="2024-06-18T19:32:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/ptp-status/ptp-clock-class-change/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}"
time="2024-06-18T19:32:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/ptp-status/lock-state/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}"
time="2024-06-18T19:32:08Z" level=error msg="CurrentState:error 400 from url http://127.0.0.1:8089/api/ocloudNotifications/v1/cluster/node/helix60.lab.eng.rdu2.redhat.com/sync/gnss-status/gnss-sync-status/CurrentState, {\"error\":\"{\\\"message\\\":\\\"client is not registered with the event publisher http://ptp-event-publisher-service-helix60.openshift-ptp.svc.cluster.local:9043 \\\"}\\n\"}"

       

      Actual results:

      Consumer stops receiving events

      Expected results:

      Consumer continues to receive events

      Additional info:

      ptp-operator.v4.16.0-202406131906
Cluster version is 4.16.0-rc.5
kernel 5.14.0-427.22.1.el9_4.x86_64+rt

              jacding@redhat.com Jack Ding
              openshift-crt-jira-prow OpenShift Prow Bot
              Bonnie Block Bonnie Block
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: