Uploaded image for project: 'Serverless logic'
  1. Serverless logic
  2. SRVLOGIC-865

DB Mibrator Job is not created with the correct image tag

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Blocker Blocker
    • 1.38.0
    • 1.37.1
    • Cloud:Operator
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • Critical

      When a user configures the SonataFlowPlatform to use the "job" based database migration strategy, when the OSL Operator is being updated from  version 1.37.1 to 1.38.0 the following case was detected. (might not always happen depending on the cluster status)

      While we are in in 1.37.1:

      1. A SonataFlowPlatform with the "job" based database migration strategy is deployed.
      2. The corresponding db-migrator  Job is created

      During the migration to 1.38.0:

      1. we delete the db-migrator Job (otherwise, with current implementation it'll never be created again)
      2. The Operator upgrade from 1.37.1 to 1.38.0 is executed
      3. as part of the migration, it's expected that the SonataFlowPlatform is reconcyled again, and the db-migrator Job is created again. With the db-migrator job image correspnding to version 1.38.0, and thus the DB updates executed.

      However, it was detected that there are runs where a "recon request" is picked up by the old instance (while the new operator instance is not yet the leader), and in that particular case, the old instance re-creates the migration Job, but with the old image. And thus, no DB updates.

      Then, the new Operator version, will still recevie the "recon request", but, since the db-migrator job was already created (with the old version) , it's not created again.

      An thus, the required DB changes for 1.38.0 are no applied.

      We must provide a more robust mechanism.
       
      Workaround: Systems that fall in the scenario above, can recover from this state, by doing something like this:
      1) delete the DB migrator job
      2) updated the SonfataFlowPlaform with a silly change, and apply it again.

      However this is far from ideal.

              wmedvede@redhat.com Walter Medvedeo
              wmedvede@redhat.com Walter Medvedeo
              Jakub Schwan Jakub Schwan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:

                  Estimated:
                  Original Estimate - 3 days
                  3d
                  Remaining:
                  Remaining Estimate - 3 days
                  3d
                  Logged:
                  Time Spent - Not Specified
                  Not Specified