Restricting virtualservice by adding namespace/hostname in custom ingress-gateway not working

The ServiceMesh Gateway resource doesn't seem to be behaving as expected.
Configuration:

OpenShift Route:
abc.app.example.com => istio-system/ingress-gateway
xyz.app.example.com => istio-system/ingress-gateway

Virtual Services:
1. Namespace: abc, hosts: abc.app.example.com
2. Namespace: xyz, hosts: xyz.app.example.com

now configure a Gateway resource. When a Gateway is created with either of the following host combinations it works:
1. *
2. abc/, xyz/

If Gateway is configured with only non-wildcard hosts, the routes stops working:
xyz/xyz.app.example.com
abc/abc.app.example.com

Curiously, if the Gateway configuration contains a dummy wildcard host (outside of the specific virtual service namespace) + the specific hosts, it starts working again:
istio-system/*
xyz/xyz.app.example.com
abc/abc.app.example.com

Also works:
xyz/*
abc/abc.app.example.com

Upstream Doc:
https://istio.io/v1.6/docs/reference/config/networking/gateway/#Server