-
Bug
-
Resolution: Not a Bug
-
Undefined
-
None
-
4.14
-
Quality / Stability / Reliability
-
False
-
-
None
-
None
-
No
-
None
-
Proposed
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
the kube-apiserver CrashLoopBackOff after I enabled auditWebhook in the hypershift hosted cluster. oc logs kube-apiserver-cd8d754c6-cv5gm audit-logs tail: cannot open '/var/log/kube-apiserver/audit.log' for reading: No such file or directory tail: '/var/log/kube-apiserver/audit.log' has appeared; following new file
Version-Release number of selected component (if applicable):
4.14.0-0.nightly-2023-09-26-124507
How reproducible:
Always
Steps to Reproduce:
1) On Management Cluster, create a Logging collection httpserver
1.1) deploy cluster-logging operator
1.2) create ClusterLogging resource
cat <<EOF|oc create -f -
apiVersion: "logging.openshift.io/v1"
kind: "ClusterLogging"
metadata:
name: "instance"
namespace: openshift-logging
spec:
managementState: "Managed"
logStore:
type: "elasticsearch"
elasticsearch:
nodeCount: 1
resources:
limits:
memory: 2Gi
requests:
cpu: 200m
memory: 2Gi
storage: {}
redundancyPolicy: "ZeroRedundancy"
visualization:
type: "kibana"
kibana:
replicas: 1
collection:
type: "vector"
EOF
1.3) create CLF resource
cat <<EOF | oc apply -f -
apiVersion: logging.openshift.io/v1
kind: ClusterLogForwarder
metadata:
name: instance
namespace: openshift-logging
spec:
inputs:
- name: input-http
receiver:
http:
format: kubeAPIAudit
receiverPort:
name: httpserver
port: 443
targetPort: 8443
pipelines:
- name: to-default
inputRefs:
- input-http
outputRefs:
- default
EOF
1.4) make sure the collection pod are running and service httpserver is created
$oc get pods -n openshift-logging
NAME READY STATUS RESTARTS AGE
collector-6nk2j 1/1 Running 0 5h27m
collector-zx6tt 1/1 Running 0 5h27m
$ oc get svc -n openshift-logging
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
httpserver ClusterIP 172.30.96.79 <none> 443/TCP 5h27m
2) Enable audit webhook on kube-apiserver,openshift-apiserver,openshift-oauth-apiserver and openshift-ovn-kubernetes pods
2.1) create auditWebhook secret
cat <<EOF| tee httpserver.kubeconfig
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: $(oc extract cm/collector-trusted-ca-bundle --confirm| cat ca-bundle.crt |base64 -w0)
server: https://httpserver.openshift-logging.svc:443
name: httpserver-openshift-logging
contexts:
- context:
cluster: httpserver-openshift-logging
name: httpserver
current-context: httpserver
kind: Config
preferences: {}
EOF
oc create secret generic httpserver-secret --from-file=kubeconfig=httpserver.kubeconfig -n clusters-hypershift-ci-23507
2.2) enable auditWebhook in hostedcontrolplanes.
oc -n clusters patch hostedclusters.hypershift.openshift.io hypershift-ci-23507 -p='{"spec": {"auditWebhook": {"name":"httpserver-secret"}}}' --type=merge
2.3) Make sure the pods kube-apiserver,openshift-apiserver,openshift-oauth-apiserver and openshift-ovn-kubernetes are restarted and running
3) Make sure auditWebhook can sent logs to Logging collection httpserver
Actual results:
the kube-apiserver CrashLoopBackOff after I enabled auditwebhook in hypershift hosted cluster.
Expected results:
Hypershift-operator can enable auditWebhook for hosted cluster.
Additional info:
That may be a blocker to https://issues.redhat.com/browse/SDE-3223