-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
openshift-4.20
-
None
-
Quality / Stability / Reliability
-
False
-
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
After completing the MicroShift 4.20 performance tests, I saw that the pod start up times were much slower than those seen in 4.19.
Version-Release number of selected component (if applicable):
4.20-rc3
How reproducible:
Cold start times are very reproducible, warm start times ~50/50.
Steps to Reproduce:
1. Ansible role for microshift-start uses ready.sh to measure microshift start up times. Run script to capture. 2. Note pod start up times. 3.
Actual results:
=== Cold Boot (boot0.txt) Metrics === Directory | Service (s) | Kubeconfig (s) | Non-Storage (s) | All Pods (s) -------------------------------------------------------------------------- 420-rc3-00 | 51.689 | 53 | 113 | 171 420-rc3-01 | 51.9 | 52 | 116 | 138 420-rc3-02 | 51.522 | 52 | 111 | 159 420-rc3-03 | 51.316 | 51 | 112 | 136 420-rc3-04 | 52.469 | 53 | 112 | 161 420-rc3-05 | 52.014 | 53 | 134 | 185 420-rc3-06 | 51.79 | 52 | 111 | 138 420-rc3-07 | 51.364 | 52 | 113 | 134 420-rc3-08 | 52.473 | 53 | 112 | 137 420-rc3-09 | 51.969 | 52 | 112 | 136 420-rc3-10 | 51.535 | 52 | 110 | 121 420-rc3-11 | 52.364 | 53 | 114 | 156 420-rc3-12 | 52.028 | 52 | 108 | 157 -------------------------------------------------------------------------- Average | 51.9 | 52 | 114 | 148=== Warm Boot (boot1.txt) Metrics === Directory | Service (s) | Kubeconfig (s) | Non-Storage (s) | All Pods (s) -------------------------------------------------------------------------- 420-rc3-00 | 23.4 | 24 | 34 | 116 420-rc3-01 | 23.485 | 24 | 34 | 89 420-rc3-02 | 23.578 | 24 | 35 | 82 420-rc3-03 | 23.438 | 24 | 34 | 110 420-rc3-04 | 23.644 | 25 | 67 | 108 420-rc3-05 | 23.246 | 23 | 66 | 89 420-rc3-06 | 23.42 | 24 | 35 | 79 420-rc3-07 | 23.323 | 24 | 66 | 106 420-rc3-08 | 23.549 | 24 | 34 | 77 420-rc3-09 | 23.164 | 24 | 67 | 108 420-rc3-10 | 23.454 | 24 | 35 | 84 420-rc3-11 | 23.37 | 23 | 66 | 94 420-rc3-12 | 23.443 | 24 | 35 | 93 -------------------------------------------------------------------------- Average | 23.4 | 24 | 47 | 95
Expected results:
=== Cold Boot (boot0.txt) Metrics === Directory | Service (s) | Kubeconfig (s) | Non-Storage (s) | All Pods (s) -------------------------------------------------------------------------- 420 | 51.689 | 53 | ~ 75 | ~ 115 === Warm Boot (boot1.txt) Metrics === Directory | Service (s) | Kubeconfig (s) | Non-Storage (s) | All Pods (s) -------------------------------------------------------------------------- 420-rc3-00 | 23.4 | 24 | ~ 35 | ~ 85
Additional info:
Large stddev on warm start times