-
Bug
-
Resolution: Done
-
Major
-
Logging 5.5.0
-
False
-
None
-
False
-
NEW
-
VERIFIED
-
-
Log Storage - Sprint 222, Log Storage - Sprint 223, Log Storage - Sprint 224, Log Storage - Sprint 225, Log Storage - Sprint 226, Log Storage - Sprint 227, Log Storage - Sprint 228
Unable to query Loki logs through logcli with time out error. Also, see network/DNS errors on Loki Components.
Gateway:
level=error name=lokistack-gateway ts=2022-07-26T13:51:37.314957328Z caller=openshift.go:146 tenant=application msg="unable to auto discover OpenShift OAuth endpoints: unable to send request to oauth server: Get \"https://172.30.0.1/.well-known/oauth-authorization-server\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" level=error name=lokistack-gateway ts=2022-07-26T13:51:37.317124791Z caller=openshift.go:146 tenant=infrastructure msg="unable to auto discover OpenShift OAuth endpoints: unable to send request to oauth server: Get \"https://172.30.0.1/.well-known/oauth-authorization-server\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" level=error name=lokistack-gateway ts=2022-07-26T13:51:37.317263053Z caller=openshift.go:146 tenant=audit msg="unable to auto discover OpenShift OAuth endpoints: unable to send request to oauth server: Get \"https://172.30.0.1/.well-known/oauth-authorization-server\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Ingester:
level=error ts=2022-07-26T14:37:04.802619358Z caller=flush.go:146 org_id=audit msg="failed to flush user" err="store put chunk: RequestError: send request failed\ncaused by: Put \"https://s3.us-east-1.amazonaws.com/vsphere-kbharti-loki/audit/b2a68385d7921ab0%3A1823ac0e361%3A1823acfb388%3A9b702641\": dial tcp 52.217.72.254:443: i/o timeout" level=error ts=2022-07-26T14:37:04.840920722Z caller=flush.go:146 org_id=audit msg="failed to flush user" err="store put chunk: RequestError: send request failed\ncaused by: Put \"https://s3.us-east-1.amazonaws.com/vsphere-kbharti-loki/audit/1a706534f6a6528f%3A1823ac054ba%3A1823acde5c5%3A5ab455b6\": dial tcp 52.217.72.254:443: i/o timeout" level=error ts=2022-07-26T14:37:04.864713696Z caller=flush.go:146 org_id=infrastructure msg="failed to flush user" err="store put chunk: RequestError: send request failed\ncaused by: Put \"https://s3.us-east-1.amazonaws.com/vsphere-kbharti-loki/infrastructure/a821a114b092d15d%3A1823a9a14e3%3A1823ad735ce%3Aa8c94853\": dial tcp 52.217.72.254:443: i/o timeout" level=error ts=2022-07-26T14:37:04.906220475Z caller=flush.go:146 org_id=infrastructure msg="failed to flush user" err="store put chunk: RequestError: send request failed\ncaused by: Put \"https://s3.us-east-1.amazonaws.com/vsphere-kbharti-loki/infrastructure/6f43c796dd089202%3A1823a99c188%3A1823ac2b736%3A247e00fe\": dial tcp 52.217.72.254:443: i/o timeout"
Querier:
level=warn ts=2022-07-26T13:51:51.057805228Z caller=dns_resolver.go:225 msg="failed DNS A record lookup" err="lookup lokistack-dev-query-frontend-grpc.openshift-logging.svc.cluster.local on 172.30.0.10:53: no such host" level=warn ts=2022-07-26T13:51:54.070748127Z caller=dns_resolver.go:225 msg="failed DNS A record lookup" err="lookup lokistack-dev-query-frontend-grpc.openshift-logging.svc.cluster.local on 172.30.0.10:53: no such host" level=info ts=2022-07-26T13:51:57.082108286Z caller=worker.go:209 msg="adding connection" addr=10.128.2.14:9095
Cluster Profile: vsphere7 https-proxy cluster
OCP version: 4.11
Steps to reproduce:
1) Deploy Loki and ClusterLogging Operators.
2) Create LokiStack CR
spec: managementState: Managed replicationFactor: 1 rules: enabled: true namespaceSelector: matchLabels: openshift.io/cluster-monitoring: 'true' selector: matchLabels: openshift.io/cluster-monitoring: 'true' size: 1x.extra-small storage: schemas: - effectiveDate: '2020-10-11' version: v11 secret: name: test type: s3 storageClassName: thin tenants: mode: openshift-logging
How reproducible: Always
- causes
-
LOG-3292 Loki Controller manager in CrashLoop due to failure to list *v1.Proxy
- Closed
- links to
- mentioned on
(1 mentioned on)