-
Bug
-
Resolution: Done-Errata
-
Undefined
-
None
-
4.13.z, 4.12.z, 4.14.0
-
None
-
No
-
Rejected
-
False
-
Description of problem
The service DNS resolution test with Windows workloads has been been observed to fail on ovn-kubernetes-master-e2e-aws-ovn-windows which uses the ipi-aws-ovn-hybrid-toggle workflow. This test does not fail when the ipi-aws-ovn-hybrid-toggle workflow is not used.
How reproducible: 80-90% of the time
Steps to Reproduce
1. Bring up an OCP 4.14 cluster with OVN networking on AWS
2. Convert it to use OVN hybrid networking
3. Install WMCO from source
a. Check out the WMCO repo and run:
b. OPERATOR_IMAGE=quay.io/aravindh/wmco:latest hack/olm.sh -k $private-kye.pem
c. Bring up a Windows node: hack/machineset apply
d. Bring up a Windows workload. Please ensure host and container image matches.
e. Exec into a Windows Pod and try to curl the service DNS enpoint.
Actual results:
Endpoint is not reachable
Expected results:
Endpoint should be reachable
this error shows up in CI as
=== RUN TestWMCO/network/East_West_Networking/ip-10-0-182-245.ec2.internal/service_DNS_resolution W0517 18:22:22.520679 14795 warnings.go:70] would violate PodSecurity "restricted:latest": runAsNonRoot != true (pod or container "win-curler-ec2amaz-4127ogk-dns-test" must set securityContext.runAsNonRoot=true) network_test.go:162: Error Trace: /go/src/github.com/openshift/windows-machine-config-operator/test/e2e/network_test.go:162 Error: Received unexpected error: job &Job{ObjectMeta:{win-curler-ec2amaz-4127ogk-dns-test-job-c88hn win-curler-ec2amaz-4127ogk-dns-test-job- wmco-test 40749324-1534-4d79-b2de-111566cd5afe 51274 1 2023-05-17 18:22:22 +0000 UTC <nil> <nil> map[controller-uid:40749324-1534-4d79-b2de-111566cd5afe job-name:win-curler-ec2amaz-4127ogk-dns-test-job-c88hn] map[batch.kubernetes.io/job-tracking:] [] [] [{e2e.test Update batch/v1 2023-05-17 18:22:22 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{}},"f:spec":{"f:backoffLimit":{},"f:completionMode":{},"f:completions":{},"f:parallelism":{},"f:suspend":{},"f:template":{"f:spec":{"f:affinity":{".":{},"f:nodeAffinity":{".":{},"f:requiredDuringSchedulingIgnoredDuringExecution":{}}},"f:containers":{"k:{\"name\":\"win-curler-ec2amaz-4127ogk-dns-test\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:nodeSelector":{},"f:os":{".":{},"f:name":{}},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{},"f:tolerations":{}}}}} } {kube-controller-manager Update batch/v1 2023-05-17 18:22:26 +0000 UTC FieldsV1 {"f:status":{"f:active":{},"f:ready":{},"f:startTime":{},"f:uncountedTerminatedPods":{}}} status}]},Spec:JobSpec{Parallelism:*1,Completions:*1,ActiveDeadlineSeconds:nil,Selector:&v1.LabelSelector{MatchLabels:map[string]string{controller-uid: 40749324-1534-4d79-b2de-111566cd5afe,},MatchExpressions:[]LabelSelectorRequirement{},},ManualSelector:nil,Template:{{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[controller-uid:40749324-1534-4d79-b2de-111566cd5afe job-name:win-curler-ec2amaz-4127ogk-dns-test-job-c88hn] map[] [] [] []} {[] [] [{win-curler-ec2amaz-4127ogk-dns-test mcr.microsoft.com/powershell:lts-nanoserver-1809 [pwsh.exe -command for (($i =0), ($j = 0); $i -lt 60; $i++) { $response = Invoke-Webrequest -UseBasicParsing -Uri win-webserver-ec2amaz-4127ogk-deployment-7mfs7-qxb8h.wmco-test.svc.cluster.local; $code = $response.StatusCode; echo "GET returned code $code";If ($code -eq 200) {exit 0}; Start-Sleep -s 10;}; exit 1] [] [] [] [] {map[] map[] []} [] [] nil nil nil nil /dev/termination-log File IfNotPresent nil false false false}] [] Never 0xc0008323b8 <nil> ClusterFirst map[kubernetes.io/os:windows] <nil> false false false <nil> PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,} [] &Affinity{NodeAffinity:&NodeAffinity{RequiredDuringSchedulingIgnoredDuringExecution:&NodeSelector{NodeSelectorTerms:[]NodeSelectorTerm{NodeSelectorTerm{MatchExpressions:[]NodeSelectorRequirement{},MatchFields:[]NodeSelectorRequirement{NodeSelectorRequirement{Key:metadata.name,Operator:In,Values:[ip-10-0-182-245.ec2.internal],},},},},},PreferredDuringSchedulingIgnoredDuringExecution:[]PreferredSchedulingTerm{},},PodAffinity:nil,PodAntiAffinity:nil,} default-scheduler [{os Windows NoSchedule <nil>}] [] <nil> nil [] <nil> <nil> <nil> map[] [] <nil> &PodOS{Name:windows,} <nil> [] []}},BackoffLimit:*6,TTLSecondsAfterFinished:nil,CompletionMode:*NonIndexed,Suspend:*false,PodFailurePolicy:nil,},Status:JobStatus{Conditions:[]JobCondition{},StartTime:2023-05-17 18:22:22 +0000 UTC,CompletionTime:<nil>,Active:1,Succeeded:0,Failed:0,CompletedIndexes:,UncountedTerminatedPods:&UncountedTerminatedPods{Succeeded:[],Failed:[],},Ready:*1,},} timed out: [{{ } {win-curler-ec2amaz-4127ogk-dns-test-job-c88hn-nfjdr.1760015908cc78ff wmco-test 7e8adb22-db7f-4370-99c0-e355abdf4bd6 51256 0 2023-05-17 18:22:22 +0000 UTC <nil> <nil> map[] map[] [] [] [{kube-scheduler Update events.k8s.io/v1 2023-05-17 18:22:22 +0000 UTC FieldsV1 {"f:action":{},"f:eventTime":{},"f:note":{},"f:reason":{},"f:regarding":{},"f:reportingController":{},"f:reportingInstance":{},"f:type":{}} }]} {Pod wmco-test win-curler-ec2amaz-4127ogk-dns-test-job-c88hn-nfjdr 4bbd38a4-0a07-4f6b-b9f4-12b88030c60e v1 51251 } Scheduled Successfully assigned wmco-test/win-curler-ec2amaz-4127ogk-dns-test-job-c88hn-nfjdr to ip-10-0-182-245.ec2.internal { } 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC 0 Normal 2023-05-17 18:22:22.547892 +0000 UTC nil Binding nil default-scheduler default-scheduler-ip-10-0-164-150} {{ } {win-curler-ec2amaz-4127ogk-dns-test-job-c88hn-nfjdr.1760015964aad38c wmco-test ac895667-581b-437e-ab5b-e4d0206f8e8b 51263 0 2023-05-17 18:22:23 +0000 UTC <nil> <nil> map[] map[] [] [] [{kubelet.exe Update v1 2023-05-17 18:22:23 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod wmco-test win-curler-ec2amaz-4127ogk-dns-test-job-c88hn-nfjdr 4bbd38a4-0a07-4f6b-b9f4-12b88030c60e v1 51252 spec.containers{win-curler-ec2amaz-4127ogk-dns-test}} Pulled Container image "mcr.microsoft.com/powershell:lts-nanoserver-1809" already present on machine {kubelet ip-10-0-182-245.ec2.internal} 2023-05-17 18:22:24 +0000 UTC 2023-05-17 18:22:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil nil } {{ } {win-curler-ec2amaz-4127ogk-dns-test-job-c88hn-nfjdr.17600159681d7254 wmco-test f9b86c3c-ccd4-4cb3-b009-4aeeabd09ec5 51264 0 2023-05-17 18:22:24 +0000 UTC <nil> <nil> map[] map[] [] [] [{kubelet.exe Update v1 2023-05-17 18:22:24 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod wmco-test win-curler-ec2amaz-4127ogk-dns-test-job-c88hn-nfjdr 4bbd38a4-0a07-4f6b-b9f4-12b88030c60e v1 51252 spec.containers{win-curler-ec2amaz-4127ogk-dns-test}} Created Created container win-curler-ec2amaz-4127ogk-dns-test {kubelet ip-10-0-182-245.ec2.internal} 2023-05-17 18:22:24 +0000 UTC 2023-05-17 18:22:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil nil } {{ } {win-curler-ec2amaz-4127ogk-dns-test-job-c88hn-nfjdr.1760015996a7abbc wmco-test a1adcbc0-7eed-4a30-b90d-8ff2baca87c9 51268 0 2023-05-17 18:22:24 +0000 UTC <nil> <nil> map[] map[] [] [] [{kubelet.exe Update v1 2023-05-17 18:22:24 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]} {Pod wmco-test win-curler-ec2amaz-4127ogk-dns-test-job-c88hn-nfjdr 4bbd38a4-0a07-4f6b-b9f4-12b88030c60e v1 51252 spec.containers{win-curler-ec2amaz-4127ogk-dns-test}} Started Started container win-curler-ec2amaz-4127ogk-dns-test {kubelet ip-10-0-182-245.ec2.internal} 2023-05-17 18:22:24 +0000 UTC 2023-05-17 18:22:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil nil }]
- blocks
-
OCPBUGS-15368 Service DNS resolutions fails in Windows Pods on converted OVN hybrid clusters
- Closed
- is cloned by
-
OCPBUGS-15368 Service DNS resolutions fails in Windows Pods on converted OVN hybrid clusters
- Closed
- links to
-
RHEA-2023:5006 rpm