Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-19560

[rhosp17.1] swift swift_container_replicator container in unhealthy state due to slow replication.

XMLWordPrintable

    • 5
    • False
    • Hide

      None

      Show
      None
    • False
    • ?
    • openstack-tripleo-common-15.4.1-17.1.20250908140822.e5b18f2.el9ost
    • None
    • Proposed
    • Swift Sprint 3, Swift Sprint 4
    • 2
    • Moderate
    • Customer Case Attached

      swift swift_container_replicator container in unhealthy state due to slow replication.

      29 hours ago  Up 2 minutes (unhealthy)              swift_container_replicator
      
      > >cat /var/cache/swift/container.recon
      > >{"container_auditor_pass_completed": 1220.1614661216736, "container_audits_failed": 0, "container_audits_passed": 488066, "container_audits_since": 1756560192.6049159, "container_updater_sweep": 4923.433475255966, "replication_last": 1756564267.295261, "replication_stats": {"attempted": 244033, "deferred": 0, "diff": 4118, "diff_capped": 0, "empty": 0, "failure": 0, "hashmatch": 0, "no_change": 239915, "remote_merge": 0, "remove": 0, "rsync": 0, "start": 1756559386.646013, "success": 244033, "ts_repl": 0}, "replication_time": 4880.64924788475}
      

      This setup is 3 object storage nodes with one dedicated disk each.

      Replication seems to work but it is significantly slower than the healthcheck. Its likely the disks is slow.

      Tried to modify the default config but still having issues with unhealthy container:

      [container-replicator]
      concurrency = 8
      interval = 240
      

      Looking for review and assistance with the following:

      1. Config that will allow swift_container_replicator container to stay healthy.

      2. How to modify container-replicator config in the OOO templates. It looks like the following should work but does not:

      
        ObjectStorageExtraConfig:
          swift::config::swift_container_config:
            container-replicator/concurrency:
              value: 2
            container-replicator/interval:
              value: 240
      
      From /etc/puppet/modules/swift/README.md
      
      #### swift_container_config
      

      Same as `swift_config`, but path is `/etc/swift/container-server.conf`

              cschwede@redhat.com Christian Schwede
              mflusche@redhat.com Mathew Flusche
              rhos-storage-swift
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: