Jio has escalated the issue [1] of intermittent image upload failures on their many clusters. The Glance, Cinder and Ceph team members are working on finding out the exact root cause, recently we have been able to reproduce this issue in our lab. We were able to narrow down the issue with number of glance api workers configured in glance-api conf file, when we set it to 1 instead of 12 then we are not seeing this issue. All history and detail description can be found in BZ [2], the customer has requested to provide some benchmarking data on how many parallel image upload we can run with RBD as backend for Glance and Cinder and workers set to 12. It will help us and customer to make the decision on changing design e.g. changing their workflow or changing glance backend to Cinder.

Just to be more specific, please include tests with smaller images like cirros and big images of 30 GB size.

[1]https://issues.redhat.com/browse/RHOSPPRIO-456

[2]https://bugzilla.redhat.com/show_bug.cgi?id=2333295