Zync requires zync-que redeploy for routes to be created

Sometimes, after deploying 3scale to OpenShift, none of the routes will get created until the zync-que pod is redeployed. This is because Zync relies on some services – notably apicast-(staging|production) and system-(master|provider|developer) services – to have been created in the namespace by the time the zync-que container starts running.

Here's an explanation through lines of code of Zync:

• Zync includes this config file: https://github.com/3scale/zync/blob/912f913df4ff70ba8c02cc10451875e75a39888c/config/openshift.yml
• It becomes part of the Rails app config: https://github.com/3scale/zync/blob/912f913df4ff70ba8c02cc10451875e75a39888c/config/application.rb#L84
• Which then determines whether the K8s integration is enabled: https://github.com/3scale/zync/blob/912f913df4ff70ba8c02cc10451875e75a39888c/app/jobs/process_entry_job.rb#L95

As it can be seen from the config file, if zync-que container starts before any of those services get created, the zync-que container will not have at least one of the corresponding environment variables set and therefore the K8s integration will be disabled.

The above has been confirmed with evidences. Unfortunately it is not easy to reproduce as it depends on chance.

One way to see this is that the order the operator does things may sometimes cause Zync to fail creating the routes, requiring (to fix it) a re-deployment of the zync-que pod possibly followed by a forced re-sync of proxy and provider domains by manually running in any of the system pods a rake task zync:resync:domains. Another way of seeing is that Zync should not rely on those environment variables to figure out whether it's running on OpenShift or not in the first place.

See also https://issues.redhat.com/browse/THREESCALE-6835?focusedCommentId=20687163&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-20687163