-
Bug
-
Resolution: Done
-
Blocker
-
quay-v3.4.0
-
False
-
False
-
Undefined
-
-
0
Description:
This is an issue found when deploy Quay with TNG Operator on OCP(enabled FIPS), after deployment complete successfully, login quay hit 500 error page, checked quay POD losg, get error message "*ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS", see logs attached.
Note: This issue block testing quay V3.4.0 on OCP with FIPS enabled.
Check OCP to make sure FIPS was enabled:
[root@ip-10-0-1-60 centos]# oc get node
NAME STATUS ROLES AGE VERSION
ip-10-0-137-104.us-east-2.compute.internal Ready master 5h54m v1.19.0+9f84db3
ip-10-0-140-218.us-east-2.compute.internal Ready worker 5h44m v1.19.0+9f84db3
ip-10-0-147-130.us-east-2.compute.internal Ready worker 5h47m v1.19.0+9f84db3
ip-10-0-181-88.us-east-2.compute.internal Ready worker 5h44m v1.19.0+9f84db3
ip-10-0-190-141.us-east-2.compute.internal Ready master 5h54m v1.19.0+9f84db3
ip-10-0-191-137.us-east-2.compute.internal Ready worker 5h44m v1.19.0+9f84db3
ip-10-0-192-189.us-east-2.compute.internal Ready,SchedulingDisabled master 5h54m v1.19.0+9f84db3
ip-10-0-198-131.us-east-2.compute.internal Ready worker 5h45m v1.19.0+9f84db3
[root@ip-10-0-1-60 centos]# oc debug no/ip-10-0-140-218.us-east-2.compute.internal
Starting pod/ip-10-0-140-218us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.140.218
If you don't see a command prompt, try pressing enter.
sh-4.4# chroot /host
sh-4.4# fips-mode-setup --checkfips-mode-setup --check^C
sh-4.4# fips-mode-setup --check
FIPS mode is enabled.
Quay Image:
lizhang@lzha-mac quay3.4 % oc get pod NAME READY STATUS RESTARTS AGE quayfips-clair-54d55666f9-tb5cw 1/1 Running 0 8m36s quayfips-clair-postgres-55cfc4568b-q6dg9 1/1 Running 1 11m quayfips-quay-app-79557d4f89-b9qxp 1/1 Running 0 103s quayfips-quay-config-editor-549d85d975-jpcwb 1/1 Running 0 8m36s quayfips-quay-database-6c5cdf4986-5xmbl 1/1 Running 0 11m quayfips-quay-mirror-6757d68f45-kl6pl 1/1 Running 0 8m36s quayfips-quay-postgres-init-jbtvz 0/1 Completed 0 10m quayfips-quay-redis-568977d465-fd97c 1/1 Running 0 10m lizhang@lzha-mac quay3.4 % oc get pod quayfips-quay-app-79557d4f89-b9qxp -o json | jq '.spec.containers[0].image' "registry.redhat.io/quay/quay@sha256:12434420fa3ca1df4654bf6d08c8980b1209b13fbfe8336fd9b1ef6f04cc4c75"
Quay POD Logs:
1561 gunicorn-web stdout | 2020-11-13 09:06:14,567 [280] [ERROR] [gunicorn.error] Error handling request /api/v1/user/ 1562 gunicorn-web stdout | Traceback (most recent call last): 1563 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 55, in handle 1564 gunicorn-web stdout | self.handle_request(listener_name, req, client, addr) 1565 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/ggevent.py", line 143, in handle_request 1566 gunicorn-web stdout | super().handle_request(listener_name, req, sock, addr) 1567 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 106, in handle_request 1568 gunicorn-web stdout | respiter = self.wsgi(environ, resp.start_response) 1569 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2463, in __call__ 1570 gunicorn-web stdout | return self.wsgi_app(environ, start_response) 1571 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/werkzeug/middleware/proxy_fix.py", line 232, in __call__ 1572 gunicorn-web stdout | return self.app(environ, start_response) 1573 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2449, in wsgi_app 1574 gunicorn-web stdout | response = self.handle_exception(e) 1575 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask_restful/__init__.py", line 269, in error_router 1576 gunicorn-web stdout | return original_handler(e) 1577 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1866, in handle_exception 1578 gunicorn-web stdout | reraise(exc_type, exc_value, tb) 1579 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask/_compat.py", line 38, in reraise 1580 gunicorn-web stdout | raise value.with_traceback(tb) 1581 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2446, in wsgi_app 1582 gunicorn-web stdout | response = self.full_dispatch_request() 1583 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1951, in full_dispatch_request 1584 gunicorn-web stdout | rv = self.handle_user_exception(e) 1585 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask_restful/__init__.py", line 269, in error_router 1586 gunicorn-web stdout | return original_handler(e) 1587 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1820, in handle_user_exception 1588 gunicorn-web stdout | reraise(exc_type, exc_value, tb) 1589 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask/_compat.py", line 38, in reraise 1590 gunicorn-web stdout | raise value.with_traceback(tb) 1591 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1949, in full_dispatch_request 1592 gunicorn-web stdout | rv = self.dispatch_request() 1593 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1935, in dispatch_request 1594 gunicorn-web stdout | return self.view_functions[rule.endpoint](**req.view_args) 1595 gunicorn-web stdout | File "/quay-registry/endpoints/decorators.py", line 197, in wrapper 1596 gunicorn-web stdout | return func(*args, **kwargs) 1597 gunicorn-web stdout | File "/quay-registry/auth/decorators.py", line 65, in wrapper 1598 gunicorn-web stdout | return func(*args, **kwargs) 1599 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask_restful/utils/cors.py", line 35, in wrapped_function 1600 gunicorn-web stdout | resp = make_response(f(*args, **kwargs)) 1601 gunicorn-web stdout | File "/quay-registry/endpoints/csrf.py", line 73, in wrapper 1602 gunicorn-web stdout | resp = func(*args, **kwargs) 1603 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask_restful/__init__.py", line 458, in wrapper 1604 gunicorn-web stdout | resp = resource(*args, **kwargs) 1605 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask/views.py", line 89, in view 1606 gunicorn-web stdout | return self.dispatch_request(*args, **kwargs) 1607 gunicorn-web stdout | File "/usr/local/lib/python3.8/site-packages/flask_restful/__init__.py", line 573, in dispatch_request 1608 gunicorn-web stdout | resp = meth(*args, **kwargs) 1609 gunicorn-web stdout | File "/quay-registry/endpoints/decorators.py", line 145, in wrapper 1610 gunicorn-web stdout | return func(*args, **kwargs) 1611 gunicorn-web stdout | File "/quay-registry/endpoints/decorators.py", line 120, in wrapper 1612 gunicorn-web stdout | return func(*args, **kwargs) 1613 gunicorn-web stdout | File "/quay-registry/endpoints/api/__init__.py", line 416, in wrapped 1614 gunicorn-web stdout | return func(self, *args, **kwargs) 1615 gunicorn-web stdout | File "/quay-registry/endpoints/api/user.py", line 497, in post 1616 gunicorn-web stdout | return user_view(new_user), 200, headers 1617 gunicorn-web stdout | File "/quay-registry/endpoints/api/user.py", line 158, in user_view 1618 gunicorn-web stdout | "avatar": avatar.get_data_for_user(user), 1619 gunicorn-web stdout | File "/quay-registry/avatars/avatars.py", line 85, in get_data_for_user 1620 gunicorn-web stdout | return self.get_data(user.username, user.email, "robot" if user.robot else "user") 1621 gunicorn-web stdout | File "/quay-registry/avatars/avatars.py", line 110, in get_data 1622 gunicorn-web stdout | hash_value = hashlib.md5(username_email_or_id.strip().lower().encode("utf-8")).hexdigest() 1623 gunicorn-web stdout | ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS 1624 gunicorn-web stdout | 2020-11-13 09:06:14,570 [280] [INFO] [gunicorn.access] - - [13/Nov/2020:09:06:14 +0000] "POST /api/v1/user/ HTTP/1.0" 500 0 "-" "- " 1625 nginx stdout | 10.131.0.8 () - - [13/Nov/2020:09:06:14 +0000] "POST /api/v1/user/ HTTP/2.0" 500 141 "https://quayfips-quay-quay.apps.lzha1115.qe.devclust er.openshift.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36" (0.348 166 7 0.348)
Steps:
- On OCP 4.6 with FIPS enable ,deploy Quay with TNG Operator with using AWS S3 as external registry storage
- Create new Quay super user
- Login quay with super user
Expected Results:
Login Quay successfully with super user.
Actual Results:
Login quay was failed and hit quay 500 error page.