Uploaded image for project: 'CPE Infrastructure'
  1. CPE Infrastructure
  2. CPE-2657

Further improve full compose speed

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • None
    • False
    • None
    • False
    • Testable

      https://pagure.io/releng/issue/12463

      • Describe the issue
        We've massively improved the time a full compose takes over the last few years; it's now down to 2:13:45 for the latest Rawhide compose. This ticket is really just to hold my notes on what's needed to make it any faster.

      We now basically have two significant chains:

      1. pkgset (35:20) -> ostree (28:03) -> ostree_installer (1:05:02). total: 2:08:25
      2. pkgset (35:20) -> gather (39:36) -> createrepo (7:07) -> kiwibuild (46:43). total: 2:08:46

      The additional ~5 minutes are small things that happen before pkgset and after the image build phase, and between the gather/createrepo/buildinstall phase and the image build phase.

      gather->createrepo run in parallel with buildinstall which takes 27:58, so if we saved more than ~20 minutes there, buildinstall would become the bottleneck. kiwibuild runs in parallel with createiso, osbuild, live_media and imagebuild, which take 6:30, 18:40, 31:51, and 41:42 respectively, so 5 minutes saving on kiwi would make imagebuild the bottleneck, 15 mins on kiwi and 10 mins on imagebuild would make live_media the bottleneck.

      Overall seems like the place to attack is pkgset, if we still want to try and shave off more time from this. Getting significant gains anywhere else at all needs us to improve multiple things, at least two or three.

      • When do you need this? (YYYY/MM/DD)
        No particular time.
      • When is this no longer needed or useful? (YYYY/MM/DD)
        When Konflux eats Pungi in the glorious future!
      • If we cannot complete your request, what is the impact?
        Composes take longer.

              Unassigned Unassigned
              rh-ee-mkonecny Michal Konecny
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: