-
Bug
-
Resolution: Obsolete
-
Major
-
None
-
quay-3.7
-
False
-
None
-
False
-
Quay Enterprise
-
-
-
0
In certain occasions, the CSRF token gets randomly rotated or not sent at all. This causes Quay's session to terminate completely and the client needs to log back in to continue working. We don't see much in the logs apart from the fact that Quay is complaining about the CSRF token being missing or invalid:
~/cases/03354939# grep -i "csrf" -rn . ./quay1.log:188:gunicorn-web stdout | 2022-11-06 08:25:45,481 [218] [ERROR] [endpoints.csrf] CSRF Failure. Session token (_csrf_token) was and request token (_csrf_token) was 5zQMBHwRot3EfBPVBMsaVCP2WfjJWkHNCWBpFkUoWFeJuDHBEJFyws9th7cj8lNe ./quay1.log:189:gunicorn-web stdout | 2022-11-06 08:25:45,483 [218] [ERROR] [util.http] Error 403: CSRF token was invalid or missing.; Arguments: {'url': 'https://usquay.ocpnonprod.me.alinma.internal/api/v1/signin', 'status_code': 403, 'message': 'CSRF token was invalid or missing.'} ./quay3.log:2531:gunicorn-web stdout | 2022-11-06 08:53:14,899 [220] [ERROR] [endpoints.csrf] CSRF Failure. Session token (_csrf_token) was and request token (_csrf_token) was x8Jb5g-lwmSWQq79Am_CUI0M2ok5llw-YftD4d72HqgPLH2zZw6yHiIx3f1HRic_ ./quay3.log:2532:gunicorn-web stdout | 2022-11-06 08:53:14,902 [220] [ERROR] [util.http] Error 403: CSRF token was invalid or missing.; Arguments: {'url': 'https://usquay.ocpnonprod.me.alinma.internal/api/v1/signin', 'status_code': 403, 'message': 'CSRF token was invalid or missing.'}
It looks like the token is not sent by the browser at all which is strange. Client tried in several browsers in incognito mode and all have the exact same error. Timins when this happens are completely random, sometimes the error shows after a couple of minutes, sometimes it takes longer than that. During testing, the only thing we noticed is that this behaviour happens only if more Quay instances are run behind the load balancer. If only one instance is run, then this issue doesn't happen. Quay is deployed on VMs.
We cannot replicate this issue locally. Can you please check the issue?
