-
Bug
-
Resolution: Done
-
Major
-
7.0.1.Final
The test is ran twice but it seems to fail often the execution of (at least by perception, it could even be always).
(Please be aware of the <REDACTED> in the following)
[ERROR] testInstallationEquivalence(org.wildfly.test.manual.management.ProvisioningConsistencyTestCase) Time elapsed: 0.101 s <<< FAILURE! java.lang.AssertionError: <REDACTED>/jboss-as/testsuite/integration/manualmode-expansion/target/wildfly-from-channel/standalone at org.junit.Assert.fail(Assert.java:89) at org.junit.Assert.assertTrue(Assert.java:42) at org.wildfly.test.distribution.validation.ProvisioningConsistencyBaseTest$1.preVisitDirectory(ProvisioningConsistencyBaseTest.java:106) at org.wildfly.test.distribution.validation.ProvisioningConsistencyBaseTest$1.preVisitDirectory(ProvisioningConsistencyBaseTest.java:80) at java.base/java.nio.file.Files.walkFileTree(Files.java:2732) at java.base/java.nio.file.Files.walkFileTree(Files.java:2797) at org.wildfly.test.distribution.validation.ProvisioningConsistencyBaseTest.testInstallationEquivalence(ProvisioningConsistencyBaseTest.java:80) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
Failing version: https://github.com/wildfly/wildfly/blob/main/testsuite/integration/manualmode-expansion/src/test/java/org/wildfly/test/manual/management/ProvisioningConsistencyTestCase.java
Passing version: https://github.com/wildfly/wildfly/blob/main/testsuite/integration/manualmode/src/test/java/org/wildfly/test/manual/management/ProvisioningConsistencyTestCase.java
(for convenience a link to https://github.com/wildfly/wildfly/blob/main/testsuite/shared/src/main/java/org/wildfly/test/distribution/validation/ProvisioningConsistencyBaseTest.java)
[JBTM-3848] Investigate ProvisioningConsistencyTestCase>ProvisioningConsistencyBaseTest.testInstallationEquivalence:80 CI failures
./build.sh -Djboss.dist=pwd/dist/target/wildfly-32.0.0.Beta1-SNAPSHOT clean verify -f testsuite/integration/basic/pom.xml -Dtest=SecurityCommandsTestCase
Puts cli-marker into dist/target/wildfly-32.0.0.Beta1-SNAPSHOT/bin/
If the ProvisioningConsistencyTestCase is going to use this to determine equivalence, that would account for the test failing though it would then need to be considered why that would not fail on the WildFly CI (it may still be that this file is not included when testing for equivalence - I have pushed a commit reverting the removal of -fae).
Tom Jenkinson
added a comment - - edited ./build.sh -Djboss.dist=pwd/dist/target/wildfly-32.0.0.Beta1-SNAPSHOT clean verify -f testsuite/integration/basic/pom.xml -Dtest=SecurityCommandsTestCase
Puts cli-marker into dist/target/wildfly-32.0.0.Beta1-SNAPSHOT/bin/
If the ProvisioningConsistencyTestCase is going to use this to determine equivalence, that would account for the test failing though it would then need to be considered why that would not fail on the WildFly CI (it may still be that this file is not included when testing for equivalence - I have pushed a commit reverting the removal of -fae). For https://github.com/wildfly/wildfly.git
git reset --hard upstream/main
git clean -fdx
./build.sh clean install -DskipTests
cd dist/target
git init .
git add wildfly-32.0.0.Beta1-SNAPSHOT
gedit .git/info/exclude
(adding in
checkstyle-cachefile
checkstyle-checker.xml
checkstyle-header.txt
checkstyle-result.xml
checkstyle-suppressions.xml
maven-archiver/
maven-status/
site/
test-classes/
verifier/
wildfly-32.0.0.Beta1-SNAPSHOT.jar
xml-validation/
}
git commit -m Initial
git status
cd ../../
./build.sh -Djboss.dist=`pwd`/dist/target/wildfly-32.0.0.Beta1-SNAPSHOT clean verify -f testsuite/integration/manualmode-expansion/pom.xml -Dtest=ProvisioningConsistencyTestCase
Works on it's own so I guess some other test has changed jboss.dist at some point which is causing the later problem. I don't think it's the one in manualmode though because I can run that and then run manualmode-expansion fine
Tom Jenkinson
added a comment - - edited For https://github.com/wildfly/wildfly.git
git reset --hard upstream/main
git clean -fdx
./build.sh clean install -DskipTests
cd dist/target
git init .
git add wildfly-32.0.0.Beta1-SNAPSHOT
gedit .git/info/exclude
(adding in
checkstyle-cachefile
checkstyle-checker.xml
checkstyle-header.txt
checkstyle-result.xml
checkstyle-suppressions.xml
maven-archiver/
maven-status/
site/
test-classes/
verifier/
wildfly-32.0.0.Beta1-SNAPSHOT.jar
xml-validation/
}
git commit -m Initial
git status
cd ../../
./build.sh -Djboss.dist=`pwd`/dist/target/wildfly-32.0.0.Beta1-SNAPSHOT clean verify -f testsuite/integration/manualmode-expansion/pom.xml -Dtest=ProvisioningConsistencyTestCase
Works on it's own so I guess some other test has changed jboss.dist at some point which is causing the later problem. I don't think it's the one in manualmode though because I can run that and then run manualmode-expansion fine Tom Jenkinson
added a comment - By running a build of WildFly and then this test (it failed, I did need to run it twice but) there are differences in the dist/target/wildfly* version compared to testsuite/integration/manualmode-expansion/target/wildfly-from-channel/.
I am going to try to do this just with WildFly checkout now.
Tom Jenkinson
added a comment - By running a build of WildFly and then this test (it failed, I did need to run it twice but) there are differences in the dist/target/wildfly* version compared to testsuite/integration/manualmode-expansion/target/wildfly-from-channel/.
I am going to try to do this just with WildFly checkout now. 
I think on CI it is output in this folder that will indicate which tests were running that caused the dist/target server to be used: testsuite/integration/elytron-oidc-client/target/surefire-reports/