-
Bug
-
Resolution: Not a Bug
-
Blocker
-
None
-
Logging 5.6.z, Logging 5.5.z
Description of problem:
Audit logs are missing on CW/default ES end.
Version-Release number of selected component (if applicable):
ROSA hypershift/RedHat Hosted cluster
rosa version: 1.2.11
How reproducible:
Always
Steps to Reproduce:
- Create ROSA hypershift cluster on us-west-2 region
- Deploy 5.6.0 CLO and ES operatorS
- Set up CLF to forward logs to default ES/Cloudwatch
- Create ClusterLogging instance with either vector or fluentd as collector
Actual results:
Infra and application logs found on CW/Default ES.
Audit logs are missing
Expected results:
Infra, Audit and application logs found on CW/Default ES.
Additional info:
CLF: http://pastebin.test.redhat.com/1091100
fluent.conf: http://pastebin.test.redhat.com/1091101
vector.toml: http://pastebin.test.redhat.com/1091102
$ rosa version 1.2.11 There is a newer release version, please consider updating: https://github.com/openshift/rosa/releases/tag/v1.2.13$ rosa list cluster ID NAME STATE 21kebqrhpssasq3qaiahtrjfp7qvv882 kbharti-loghs1 ready $ rosa describe cluster -c 21kebqrhpssasq3qaiahtrjfp7qvv882 Name: kbharti-loghs1 ID: 21kebqrhpssasq3qaiahtrjfp7qvv882 External ID: 1180f035-822b-4d22-9070-f39742db0f36 Control Plane: Red Hat hosted OpenShift Version: 4.12.0 Channel Group: stable DNS: kbharti-loghs1.xo93.s3.devshift.org AWS Account: 301721915996 API URL: https://api.hcp.kbharti-loghs1.xo93.s3.devshift.org:443Console URL: https://console-openshift-console.apps.kbharti-loghs1.rosa.kbharti-loghs1.xo93.s3.devshift.orgRegion: us-west-2 Multi-AZ: falseNodes: - Compute (workers): 6 Network: - Type: OVNKubernetes - Service CIDR: 172.30.0.0/16 - Machine CIDR: 10.0.0.0/16 - Pod CIDR: 10.128.0.0/14 - Host Prefix: /23 STS Role ARN: arn:aws:iam::301721915996:role/ManagedOpenShift-Installer-Role Support Role ARN: arn:aws:iam::301721915996:role/ManagedOpenShift-Support-Role Instance IAM Roles: - Control plane: arn:aws:iam::301721915996:role/ManagedOpenShift-ControlPlane-Role - Worker: arn:aws:iam::301721915996:role/ManagedOpenShift-Worker-Role Operator IAM Roles: - arn:aws:iam::301721915996:role/kbharti-loghs1-j9o8-openshift-ingress-operator-cloud-credentials - arn:aws:iam::301721915996:role/kbharti-loghs1-j9o8-kube-system-kube-controller-manager - arn:aws:iam::301721915996:role/kbharti-loghs1-j9o8-kube-system-capa-controller-manager - arn:aws:iam::301721915996:role/kbharti-loghs1-j9o8-openshift-image-registry-installer-cloud-cre - arn:aws:iam::301721915996:role/kbharti-loghs1-j9o8-openshift-cloud-credential-operator-cloud-cr - arn:aws:iam::301721915996:role/kbharti-loghs1-j9o8-openshift-cloud-network-config-controller-cl - arn:aws:iam::301721915996:role/kbharti-loghs1-j9o8-kube-system-control-plane-operator - arn:aws:iam::301721915996:role/kbharti-loghs1-j9o8-kube-system-kms-provider - arn:aws:iam::301721915996:role/kbharti-loghs1-j9o8-openshift-cluster-csi-drivers-ebs-cloud-cred - arn:aws:iam::301721915996:role/kbharti-loghs1-j9o8-openshift-machine-api-aws-cloud-credentials State: ready Private: No Created: Feb 2 2023 03:54:23 UTC Details Page: https://qaprodauth.console.redhat.com/openshift/details/s/2LANjV6sYh8Wzy57baD3PibTng3OIDC Endpoint URL: https://d3gt1gce2zmg3d.cloudfront.net/21kebqrhpssasq3qaiahtrjfp7qvv882$ oc get csv -n openshift-logging NAME DISPLAY VERSION REPLACES PHASE cluster-logging.v5.6.0 Red Hat OpenShift Logging 5.6.0 Succeeded elasticsearch-operator.v5.6.0 OpenShift Elasticsearch Operator 5.6.0 Succeeded kbharti@kbharti-mac ~ % oc get pods -n openshift-logging NAME READY STATUS RESTARTS AGE cluster-logging-operator-6cd5987b95-9n6s7 1/1 Running 0 5h49m collector-4wf7v 2/2 Running 0 15s collector-9dkfq 2/2 Running 0 15s collector-bz6hn 2/2 Running 0 15s collector-cvsrq 2/2 Running 0 15s collector-km58g 2/2 Running 0 15s collector-vlg2p 2/2 Running 0 15s elasticsearch-cdm-nox9qnx3-1-5c545879d6-gbrg5 2/2 Running 0 5h45m elasticsearch-im-app-27922200-fvdvj 0/1 Completed 0 9m14s elasticsearch-im-audit-27922200-sbmms 0/1 Completed 0 9m14s elasticsearch-im-infra-27922200-ft6cf 0/1 Completed 0 9m14s kibana-5bfd68b475-w9kz9 2/2 Running 0 5h45m CW logstreams: $ ~ % aws --output json logs describe-log-groups | jq { "logGroups": [ { "logGroupName": "kbharti-rosahs-log.application", "creationTime": 1675311905699, "metricFilterCount": 0, "arn": "arn:aws:logs:us-west-2:301721915996:log-group:kbharti-rosahs-log.application:*", "storedBytes": 0 } , { "logGroupName": "kbharti-rosahs-log.infrastructure", "creationTime": 1675311839160, "metricFilterCount": 0, "arn": "arn:aws:logs:us-west-2:301721915996:log-group:kbharti-rosahs-log.infrastructure:*", "storedBytes": 0 } ] } ES indices sh-4.4$ indices Thu Feb 2 10:11:21 UTC 2023 health status index uuid pri rep docs.count docs.deleted store.size pri.store.size green open app-000005 0gOIpxKQRDO7F9nG7VvGaQ 1 0 18002 0 5 5 green open .security GK2OoahnS_OWaiiQ_iHZuw 1 0 6 0 0 0 green open .kibana_1 GCf8U3tFSwG8lRtWy6ITGQ 1 0 0 0 0 0 green open app-000004 YqkvYx8sRjKPeGCQBtas1A 1 0 14360 0 4 4 green open app-000006 xiljwSqQS5-6MV98ov2Q1A 1 0 6302 0 1 1 green open app-000002 jODEi6CGQaCivMVO2KFOzQ 1 0 14388 0 4 4 green open audit-000001 8D2jIqyaTDuq7cc_tYWToA 1 0 0 0 0 0 green open app-000003 MJn8m6vBT6Webs1unr6ZQQ 1 0 14382 0 4 4 green open infra-000001 w8IyQEPzRj6KSk6UC3mamA 1 0 435117 0 292 292 green open app-000001 9bbJTCOiSSmHuiZhiF9Stg 1 0 16516 0 4 4
- is related to
-
LOG-2257 [B1-F5] CLO testing on HyperShift-based clusters (for customers)
-
- Closed
-