Uploaded image for project: 'CentOS Stream Pipeline'
  1. CentOS Stream Pipeline
  2. CS-2963

Monitoring of fedmsg-hub on mbs-backend.stream.rdu2.redhat.com

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Done
    • Icon: Major Major
    • None
    • None
    • Infrastructure
    • None
    • 5
    • False
    • Hide

      None

      Show
      None
    • False
    • None
    • Testable
    • 2025 Sprint 7

      fedmsg-hub wasn't running again on the MBS backend, so I restarted it. Can we set up monitoring, and/or automatic restart?

      (Note it's called fedmsg-hub, but it's actually the internal UMB)

      Find out as CS-2960 was reported. I restarted it by hand for now. 

      Here's the log:

      [root@mbs-backend asamalik]# journalctl -fu fedmsg-hub
      
      ...
      
      Jul 22 13:19:26 mbs-backend.stream.rdu2.redhat.com fedmsg-hub[2132544]: [2025-07-22 13:19:26][moksha.hub    INFO] Lost connection.  Reason: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion: Connection lost.

      Last time this happened was January 2025 (at least that's what my notes say). So it's not very frequent, but annoying nevertheless as builds are stuck in the build state and it's not obvious what's going on to the maintainer.

              farrotin@redhat.com Fabian Arrotin
              asamalik@redhat.com Adam Samalik
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: