Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-62498

provisioning machines with large AWS instance types take 15 minutes

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 4.18.0
    • RHCOS
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • 3
    • Important
    • None
    • None
    • None
    • None
    • Ready to Pick
    • 1
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      It takes ~15 minutes to scale up a large AWS machine but only ~5 minutes for a smaller instance type.
      the Autoscaler will initiate a scale up of a machineset but it will take ~15 minutes to scale up a c5.24xlarge or r5.24xlarge instance.

      Scaling up a c5.4xlarge takes 5 minutes.

      Tested manually scaling up the machinesets and got the same results.

      test results
      ~~~
      The vm will get created in AWS in about a minute and ssh is possible.
      Watching the journalctl (journalctl -f) it is observed that the "Fetching ostree" is taking around ~12 minutes for the large instancetypes but on ~3 minutes for the c5.4xlarge

      Tested pulling the ostree image and the pull completes in 5-15 seconds

      After the "Fetching ostree" completes the vm reboots and in about 1-2 minutes shows up as node in the cluster.

      used the same ami image, ami-078e26f293629fe91, when testing the different instance types
      Tried different volumetypes io1, io2 and gp3
      used the same disk size for all, 500gb

      Need to understand why the provisioning of larger instance types take longer that the small instance types.
      It is causing issue with pod deployments when the auto scale up of the machinesets takes a long time
      ~~~

              Unassigned Unassigned
              rhn-support-dseals Daniel Seals
              None
              None
              Michael Nguyen Michael Nguyen
              None
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: