On CRW CI we are running E2E tests related to GitHub. They requires registering GitHub oAuth application with a correct callback URL which depends on IP of slave where tests are running.
Another solution: setup persistent IP on jenkins slave https://docs.openstack.org/newton/user-guide/cli-manage-ip-addresses.html