Uploaded image for project: 'JBoss Transaction Manager'
  1. JBoss Transaction Manager
  2. JBTM-3465

Narayana CI processing may be speed-up

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Done
    • Major
    • None
    • 5.11.1.Final
    • Testing
    • Hide
      This JIRA has been closed as our CI was improved with:
      - Jenkins slave agents sharing the maven folder m2
      - Creating isolated volumes for each slave agent org.jboss.narayana, org.wildfly and workspace
      - Creating a cron job to clean docker volumes every hour

      The main problem was identified in the performance bottleneck that Docker presents when it comes to writing in containers (i.e. not using volumes or mount points).

      The main result of the improvements applied to our CI is that builds are now (at least) 50% faster than what they used to be.
      Show
      This JIRA has been closed as our CI was improved with: - Jenkins slave agents sharing the maven folder m2 - Creating isolated volumes for each slave agent org.jboss.narayana, org.wildfly and workspace - Creating a cron job to clean docker volumes every hour The main problem was identified in the performance bottleneck that Docker presents when it comes to writing in containers (i.e. not using volumes or mount points). The main result of the improvements applied to our CI is that builds are now (at least) 50% faster than what they used to be.

    Description

      Currently the code managing the Narayana CI testing is based on the shell/bat script running the tests.

      The processing of all scripts contain mostly the same processing. Clone Narayana project, clone WildFly project, build Narayana, build WildFly with updated Narayana version, run the tests.
      The whole processing of simple few tests could take up to 2 hours just because of all the recompilation and downloading artifacts all the time. E.g. WildFly compilation can be ~40 minutes, Narayana compilation ~1 hour.
      It would be convenient to speed-up the testing as we could see the PR tests faster, the responses could be more more agile and up-to-that the CI system could not be pushed to limits with CPU and disk usage.

      I gathered few suggestions what can be done (there could be better solutions at the end)

      • do not running JACOCO axis in narayana CI job
        • a create separate job as we did for AS_TESTS and run once a week could be created
      • on Narayana compilation we could compile only a particular module? (a PoC in LRA https://github.com/jbosstm/narayana/blob/5.11.1.Final/scripts/hudson/narayana.sh#L509)
      • Downloading artifacts takes the longest time of execution. Having a job which generates a ‘zip’ and is copied as a whole chunk to local ~/.m2/repository as job setup?
      • XTS and RTS speed-up
        • For axes which compile whole WildFly we could be using only WildFly NIghtly build + updating artifacts from Narayana build directly in /module directory?

      Attachments

        Activity

          People

            jfinelli@redhat.com Manuel Finelli
            ochaloup@redhat.com Ondrej Chaloupka (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: