Currently the connect plugins being tested are created on the fly in tests by Strimzi by downloading all required bits from the artifacts server.
Each bit (being it jar or archive) is placed into a separate folder which names somehow prefabricated by Strimzi (e.g. Archive: /opt/kafka/plugins/debezium-connector-jdbc/ec5f0d19. According to Stimzi developers these folder are not random, but are hashes of either maven coordinates or URL of the bits. However, in our tests the folder is always different, e.g. Archive: /opt/kafka/plugins/debezium-connector-jdbc/9d5e09c7.zip in one test run and Archive: /opt/kafka/plugins/debezium-connector-jdbc/ec5f0d19.zip in the other.
Same applies for other bits, like Apicurio or our additiona jackson libs which should override old jakson libs used in Apicurio:

STEP 94/119: RUN 'mkdir' '-p' '/opt/kafka/plugins/debezium-connector-jdbc/8a124eb3'       && 'curl' '-f' '-L' '--output' '/opt/kafka/plugins/debezium-connector-jdbc/8a124eb3/8a124eb3.jar' 'http://debezium-artifact-server.debezium-ocp-1142.svc.cluster.local:8080/jackson/jackson-databind-2.16.2.redhat-00001.jar'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1600k  100 1600k    0     0  78.1M      0 --:--:-- --:--:-- --:--:-- 82.2M
--> 8e57a2d10b15
STEP 95/119: RUN 'mkdir' '-p' '/opt/kafka/plugins/debezium-connector-jdbc/bcbd63a8'       && 'curl' '-f' '-L' '--output' '/opt/kafka/plugins/debezium-connector-jdbc/bcbd63a8/bcbd63a8.jar' 'http://debezium-artifact-server.debezium-ocp-1142.svc.cluster.local:8080/jackson/jackson-dataformat-csv-2.16.2.redhat-00001.jar'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   99k  100   99k    0     0  2694k      0 --:--:-- --:--:-- --:--:-- 2694k
--> b22a4b7e1cab

So based on the constructed hashes, what comes first on the classpath is used adn as a result we haven't deterministic class path, which results into issues like DBZ-8651.

Build the connect pluings in a way which is deterministic. Either the URL of the artifact server has to be the same in all the runs or we need to build the container image of the connect plugin ourselves and don't rely on Strimzi with this.