Uploaded image for project: 'Tools (JBoss Tools)'
  1. Tools (JBoss Tools)
  2. JBIDE-14017

cache downloaded zips fetched as part of building aggregate site for reuse across Jenkins slaves (rather than in each slave's workspace)



    • Bug
    • Resolution: Won't Do
    • Major
    • 4.1.0.Beta1
    • 4.1.0.Alpha2
    • build, updatesite
    • None


      (2013-04-12 12:19:29) mistria: nickboldt: aggregate could compare logs
      (2013-04-12 12:19:35) nickboldt: mistria: which logs?
      (2013-04-12 12:20:58) mistria: nickboldt: the one we generate when using "fetch-zips"
      (2013-04-12 12:22:40) nickboldt: so you want the aggregate job to fetch the zips BEFORE it builds the update site to determine if it should build the update site?
      (2013-04-12 12:24:05) nickboldt: whole point of a "should I build?" check is that it happens before the build... eg., check SVN, check git, or run jbosstools-composite-install_master to check the installed IUs installed from the composite to determine if contents or versions have changed
      (2013-04-12 12:24:46) nickboldt: not adverse to the idea of splitting the "fetch zips" step out of the "build aggregate" job, but ... then we have 2 more jobs to worry about
      (2013-04-12 12:24:53) mistria: nickboldt: actually, the new version of the fetch mojo won't re-fetch zips if it finds out the the signature of the to-be-downloaded zip is the same as the one it has in cache
      (2013-04-12 12:25:05) nickboldt: cache in workspace?
      (2013-04-12 12:25:06) mistria: nickboldt: so no change, not fetch
      (2013-04-12 12:25:23) nickboldt: or in static jenkins share folder?
      (2013-04-12 12:25:23) mistria: nickboldt: in aggregate/site/zips
      (2013-04-12 12:25:36) nickboldt: great, so it's one fetch per slave
      (2013-04-12 12:26:14) nickboldt: would be better to cache that outside the workspace, if building in Jenkins
      (2013-04-12 12:26:16) mistria: nickboldt: right, we should provide a way to override this directory and put in in a shared folder
      (2013-04-12 12:27:12) mistria: nickboldt: there is not system property for that yet... https://github.com/jbosstools/jbosstools-maven-plugins/blob/master/tycho-plugins/repository-utils/src/main/java/org/jboss/tools/tycho/sitegenerator/FetchZipsFromRepo.java#L60
      (2013-04-12 12:28:21) nickboldt: Old way: https://github.com/jbosstools/jbosstools-build-sites/blob/jbosstools-4.1.0.Alpha2x/aggregate/build.xml#L510 and https://github.com/jbosstools/jbosstools-build-sites/blob/jbosstools-4.1.0.Alpha2x/aggregate/build.xml#L33 (but we still dumped to ${output.dir}/zips, not to {WORKINGDIR}/zips )
      (2013-04-12 12:28:58) nickboldt: mistria: should allow caching files in Jenkins's static file tree for use across slaves

      So, I guess all we need to do is expose outputFolder to be overridden when -Phudson is enabled.

      We could even set the correct path in the parent pom's hudson profile so that it need not be set in all the various jobs.


        Issue Links



              mistria@redhat.com Mickael Istria
              nickboldt Nick Boldt
              0 Vote for this issue
              3 Start watching this issue