Originally, the bug appeared to be limited to situations where a secret was repeatedly created/updated/deleted. On further investigation, the problem is seen intermittently when a single secret is created/updated when the E2E tests are run. The problem has been seen by 2 separate team members while running the test locally. The problem does not seem to be based on a timing issue as extending the timeouts in the test has not resolved the issue.

This issue is related to:

• The test described here: https://jira.coreos.com/browse/APPSVC-126
• Which is implemented here: https://github.com/redhat-developer/service-binding-operator/pull/167
• In response to the feature request made here: https://github.com/redhat-developer/service-binding-operator/issues/69

In the course of extending the existing E2E tests we observed that this pattern is initially successful, but when repeated eventually fails:

• Create a secret for a backing service operator
• Create a backing Service CR
• Create a deployment for an app that requires the backing service operator
• Create a Service Binding Request to bind the app to the backing service
• Verify that the app is successfully bound to the backing service
• Modify the binding-request secret, the action of modification should trigger the Service Binding Operator to reconcile and reset the secret to its original values
• Delete the service binding request, the secrets, and the app deployment
• Repeat

A workaround for the issue is to use a unique secret for each iteration of the test. This is the case even though the secret is created and deleted in each loop iteration in the test. The error does not seem to be a timing issue.

We have not been able to recreate this issue manually.

To recreate this error:

• Using the code in this pull request - https://github.com/redhat-developer/service-binding-operator/pull/167
• Comment out this line: https://github.com/redhat-developer/service-binding-operator/pull/167/files#diff-bb4f63f20e11707845fafdd125e8951cR312
• Running the test will cause the error to occur.

While the test is running, this shell command displays the values of the secret. When successful, the command should return an initial value of "user," then a value of "bogus," and then a value of "user" again after the Service Binding Operator resets the value. For example:

2e-service-binding-request5
user
e2e-service-binding-request5
user
e2e-service-binding-request5
user
e2e-service-binding-request5
bogus
e2e-service-binding-request5
bogus
e2e-service-binding-request5
bogus
e2e-service-binding-request5
bogus
e2e-service-binding-request5
bogus
e2e-service-binding-request5
user
e2e-service-binding-request5
user
e2e-service-binding-request5
user

The shell command is:

while true; 
do TEMP=`oc get secret | grep e2e-service-binding-request | sed 's/Opaque.*//'`; 
echo $TEMP;  
oc get secret $TEMP -o=yaml | grep DATABASE_SECRET_USER | sed 's/  DATABASE_SECRET_USER: //' | base64 -d; 
echo "";
done