Uploaded image for project: 'Satellite'
  1. Satellite
  2. SAT-40153

Satellite upgrade from 6.17.5 to 6.18 failed during Pulpcore database migration due to incompatible versions detected for some of the pulpcore plugins

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • 6.18.3
    • 6.17.5, 6.16.5.5
    • Upgrades
    • False
    • Satellite Rocket Refinement
    • sat-rocket
    • None
    • None
    • None
    • None

      During the upgrade of Red Hat Satellite from version 6.17.5 → 6.18, the process failed while running the pulpcore-manager migrate --noinput step. The migration aborted due to incompatible Ansible plugin versions detected across pulp workers.

      Some services were still running during the upgrade, which may have contributed to the failure.

      2025-11-06 11:42:46 [ERROR ] [configure] 'pulpcore-manager migrate --noinput' returned 1 instead of one of [0]
      2025-11-06 11:42:46 [ERROR ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: change from 'notrun' to ['0'] failed: 'pulpcore-manager migrate --noinput' returned 1 instead of one of [0]
      2025-11-06 11:47:00 [NOTICE] [configure] System configuration has finished.Error 1: Puppet Exec resource 'pulpcore-manager migrate --noinput' failed. Logs:
        /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]
          Adding autorequire relationship with File[/var/lib/pulp]
          Adding autorequire relationship with User[pulp]
          Starting to evaluate the resource (2759 of 2957)
          Evaluated in 7.20 seconds
        Exec[pulpcore-manager migrate --noinput](provider=posix)
          Executing 'pulpcore-manager migrate --noinput'
        /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns
          Operations to perform:
            Apply all migrations: ansible, auth, certguard, container, contenttypes, core, file, rpm, sessions
          Running migrations:
            Applying ansible.0056_collectionversion_sha256... OK
            Applying ansible.0057_collectionversion_sha256_migrate... OK
            Applying ansible.0058_fix_0056_regression... OK
            Applying ansible.0059_collectionversion_unique_sha256...Traceback (most recent call last):
            File "/usr/bin/pulpcore-manager", line 8, in <module>
              sys.exit(manage())
                       ^^^^^^^^
            File "/usr/lib/python3.12/site-packages/pulpcore/app/manage.py", line 11, in manage
              execute_from_command_line(sys.argv)
            File "/usr/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
              utility.execute()
            File "/usr/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
              self.fetch_command(subcommand).run_from_argv(self.argv)
            File "/usr/lib/python3.12/site-packages/django/core/management/base.py", line 412, in run_from_argv
              self.execute(*args, **cmd_options)
            File "/usr/lib/python3.12/site-packages/django/core/management/base.py", line 458, in execute
              output = self.handle(*args, **options)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            File "/usr/lib/python3.12/site-packages/django/core/management/base.py", line 106, in wrapper
              res = handle_func(*args, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            File "/usr/lib/python3.12/site-packages/django/core/management/commands/migrate.py", line 356, in handle
              post_migrate_state = executor.migrate(
                                   ^^^^^^^^^^^^^^^^^
            File "/usr/lib/python3.12/site-packages/django/db/migrations/executor.py", line 135, in migrate
              state = self._migrate_all_forwards(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^
            File "/usr/lib/python3.12/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
              state = self.apply_migration(
                      ^^^^^^^^^^^^^^^^^^^^^
            File "/usr/lib/python3.12/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
              state = migration.apply(state, schema_editor)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            File "/usr/lib/python3.12/site-packages/django/db/migrations/migration.py", line 132, in apply
              operation.database_forwards(
            File "/usr/lib/python3.12/site-packages/pulpcore/migrations.py", line 48, in database_forwards
              raise RuntimeError(
          RuntimeError: Incompatible versions detected (ansible >= 0.23.0 needed):
            - 'ansible'='0.22.4' with pulp worker '1615@satellite-fqdn'
            - 'ansible'='0.22.4' with pulp worker '1616@satellite-fqdn'
            - 'ansible'='0.22.4' with pulp worker '1612@satellite-fqdn'
            - 'ansible'='0.22.4' with pulp worker '1605@satellite-fqdn'
            - 'ansible'='0.22.4' with pulp worker '1608@satellite-fqdn'
            - 'ansible'='0.22.4' with pulp worker '1607@satellite-fqdn'
            - 'ansible'='0.22.4' with pulp worker '1614@satellite-fqdn'
            - 'ansible'='0.22.4' with pulp worker '1609@satellite-fqdn'
          Please shutdown or upgrade the outdated components before you continue the migration.
          change from 'notrun' to ['0'] failed: 'pulpcore-manager migrate --noinput' returned 1 instead of one of [0]

      In another case it was :

              state = self.apply_migration(
                      ^^^^^^^^^^^^^^^^^^^^^
            File "/usr/lib/python3.11/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
              state = migration.apply(state, schema_editor)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            File "/usr/lib/python3.11/site-packages/django/db/migrations/migration.py", line 132, in apply
              operation.database_forwards(
            File "/usr/lib/python3.11/site-packages/pulpcore/migrations.py", line 48, in database_forwards
              raise RuntimeError(
          RuntimeError: Incompatible versions detected (core >= 3.50.0 needed):
            - 'core'='3.49.39' with pulp worker '1080@satellite-fqdn'
            - 'core'='3.49.39' with pulp worker '1085@satellite-fqdn'
            - 'core'='3.49.39' with pulp worker '1086@satellite-fqdn'
            - 'core'='3.49.39' with pulp worker '1081@satellite-fqdn'
          Please shutdown or upgrade the outdated components before you continue the migration.

      Steps to Reproduce:

      1. Attempt upgrade from Satellite 6.17.5 → 6.18.
      1. During the upgrade, ensure some pulp/ansible-related services remain running.
      1. Observe the upgrade failure during pulpcore-manager migrate --noinput.

      Expected Result:
      Upgrade should complete successfully with all pulpcore migrations applied.

              Unassigned Unassigned
              rhn-support-sabhasin Sahil Bhasin
              Sayan Das
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated: