-
Bug
-
Resolution: Done
-
Normal
-
quay-v3.10.0
-
False
-
None
-
False
-
Quay Enterprise
-
-
When Ceph/RADOS driver is used, uploading of layers over 5 GB in size breaks down due to Quay doing a one-off copy of the assembled layer which usually results in a timeout on Boto side. To address this, we limit the maximum chunk size for the upload to something that is user configurable. If not specified, 32 MB chunk size is used (the same as Docker uses in its own documentation).
The following code change seems to resolve the problem:
... class RadosGWStorage(_CloudStorage): def __init__( self, context, hostname, is_secure, storage_path, access_key, secret_key, bucket_name, port=None, maximum_chunk_size_mb=None, ): upload_params = {} connect_kwargs = { "endpoint_url": _build_endpoint_url(hostname, port=port, is_secure=is_secure), } super(RadosGWStorage, self).__init__( context, boto3.session.Session, connect_kwargs, upload_params, storage_path, bucket_name, access_key, secret_key, ) chunk_size = ( maximum_chunk_size_mb if maximum_chunk_size_mb is not None else 32 ) # 32mb default, as used in Docker registry:2 self.maximum_chunk_size = chunk_size * 1024 * 1024 ...
The RHOCSStorage class for Noobaa still inherits everything from the Ceph class.