Description of problem:
cacheless catalogsources is not ready error log message: | time="2025-09-01T06:19:33Z" level=info msg="starting pprof endpoint" address="localhost:6060" time="2025-09-01T06:19:33Z" level=fatal msg="mkdir /tmp/opm-serve-cache-3951864160: read-only file system" reason: Error
Version-Release number of selected component (if applicable):
4.20.0-0.nightly-2025-08-29-001439
How reproducible:
always
Steps to Reproduce:
1. create catsrc
xzha@xzha1-mac OCP-81995 % cat catsrc.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
name: catalog-binless
namespace: test-1
spec:
displayName: Catalog without bins!
grpcPodConfig:
extractContent:
catalogDir: /configs
securityContextConfig: restricted
image: quay.io/openshifttest/nginxolm-operator-index:nginxolm81995-binless
publisher: Keenon
sourceType: grpc
updateStrategy:
registryPoll:
interval: 60m
xzha@xzha1-mac OCP-81995 % oc get pod
NAME READY STATUS RESTARTS AGE
catalog-binless-f6fvz 0/1 CrashLoopBackOff 1 (7s ago) 12s
xzha@xzha1-mac OCP-81995 % oc get pod catalog-binless-f6fvz -o yaml
apiVersion: v1
kind: Pod
metadata:
..
creationTimestamp: "2025-09-01T06:19:28Z"
generateName: catalog-binless-
generation: 1
labels:
olm.catalogSource: catalog-binless
olm.managed: "true"
olm.pod-spec-hash: 4xC2IGPywNZKugGZ8tgUOBi3cWPzqDkvJeFRxH
name: catalog-binless-f6fvz
namespace: test-1
ownerReferences:
- apiVersion: operators.coreos.com/v1alpha1
blockOwnerDeletion: false
controller: true
kind: CatalogSource
name: catalog-binless
uid: 524b7a23-caed-42e0-9a04-6f40e1163206
resourceVersion: "131807"
uid: b336aba6-4bad-4e7d-b72e-f06784187b50
spec:
containers:
- args:
- serve
- /extracted-catalog/catalog
command:
- /bin/opm
image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2421a51b40b6a1bd600507e04c24e8c8e870b3092b0dc9c3c9be6d29375e4c4c
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
- grpc_health_probe
- -addr=:50051
failureThreshold: 3
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
name: registry-server
ports:
- containerPort: 50051
name: grpc
protocol: TCP
readinessProbe:
exec:
command:
- grpc_health_probe
- -addr=:50051
failureThreshold: 3
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
requests:
cpu: 10m
memory: 50Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 1000770000
startupProbe:
exec:
command:
- grpc_health_probe
- -addr=:50051
failureThreshold: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: FallbackToLogsOnError
volumeMounts:
- mountPath: /extracted-catalog
name: catalog-content
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-gr52q
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
imagePullSecrets:
- name: catalog-binless-dockercfg-vgjrn
initContainers:
- args:
- /bin/copy-content
- /utilities/copy-content
command:
- cp
image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d717dce83f4a92a53a2607d623a1640efaba1e838cf9446e33a145e315b796d1
imagePullPolicy: IfNotPresent
name: extract-utilities
resources: {}
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 1000770000
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: FallbackToLogsOnError
volumeMounts:
- mountPath: /utilities
name: utilities
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-gr52q
readOnly: true
- args:
- --catalog.from=/configs
- --catalog.to=/extracted-catalog/catalog
command:
- /utilities/copy-content
image: quay.io/openshifttest/nginxolm-operator-index:nginxolm81995-binless
imagePullPolicy: Always
name: extract-content
resources: {}
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 1000770000
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: FallbackToLogsOnError
volumeMounts:
- mountPath: /utilities
name: utilities
- mountPath: /extracted-catalog
name: catalog-content
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-gr52q
readOnly: true
nodeName: ip-10-0-20-92.us-east-2.compute.internal
nodeSelector:
kubernetes.io/os: linux
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 1000770000
seLinuxOptions:
level: s0:c28,c7
seccompProfile:
type: RuntimeDefault
serviceAccount: catalog-binless
serviceAccountName: catalog-binless
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
- effect: NoSchedule
key: node.kubernetes.io/memory-pressure
operator: Exists
volumes:
- emptyDir: {}
name: utilities
- emptyDir: {}
name: catalog-content
- name: kube-api-access-gr52q
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
- configMap:
items:
- key: service-ca.crt
path: service-ca.crt
name: openshift-service-ca.crt
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2025-09-01T06:19:30Z"
status: "True"
type: PodReadyToStartContainers
- lastProbeTime: null
lastTransitionTime: "2025-09-01T06:19:32Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2025-09-01T06:19:28Z"
message: 'containers with unready status: [registry-server]'
reason: ContainersNotReady
status: "False"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2025-09-01T06:19:28Z"
message: 'containers with unready status: [registry-server]'
reason: ContainersNotReady
status: "False"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2025-09-01T06:19:28Z"
status: "True"
type: PodScheduled
containerStatuses:
- allocatedResources:
cpu: 10m
memory: 50Mi
containerID: cri-o://c712ba795fbf1d084e66c63bc467409b6487f7b8f8bf7bf9aaa2782b0f1d15b1
image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2421a51b40b6a1bd600507e04c24e8c8e870b3092b0dc9c3c9be6d29375e4c4c
imageID: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2421a51b40b6a1bd600507e04c24e8c8e870b3092b0dc9c3c9be6d29375e4c4c
lastState:
terminated:
containerID: cri-o://c712ba795fbf1d084e66c63bc467409b6487f7b8f8bf7bf9aaa2782b0f1d15b1
exitCode: 1
finishedAt: "2025-09-01T06:19:33Z"
message: |
time="2025-09-01T06:19:33Z" level=info msg="starting pprof endpoint" address="localhost:6060"
time="2025-09-01T06:19:33Z" level=fatal msg="mkdir /tmp/opm-serve-cache-3951864160: read-only file system"
reason: Error
startedAt: "2025-09-01T06:19:33Z"
name: registry-server
ready: false
resources:
requests:
cpu: 10m
memory: 50Mi
restartCount: 1
started: false
state:
waiting:
message: back-off 10s restarting failed container=registry-server pod=catalog-binless-f6fvz_test-1(b336aba6-4bad-4e7d-b72e-f06784187b50)
reason: CrashLoopBackOff
user:
linux:
gid: 0
supplementalGroups:
- 0
- 1000770000
uid: 1000770000
volumeMounts:
- mountPath: /extracted-catalog
name: catalog-content
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-gr52q
readOnly: true
recursiveReadOnly: Disabled
hostIP: 10.0.20.92
hostIPs:
- ip: 10.0.20.92
initContainerStatuses:
- containerID: cri-o://2dc94a3b6abca72fe0af761b32924c73640cdbfa1cd9c0a5ddf25d07b966b041
image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d717dce83f4a92a53a2607d623a1640efaba1e838cf9446e33a145e315b796d1
imageID: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d717dce83f4a92a53a2607d623a1640efaba1e838cf9446e33a145e315b796d1
lastState: {}
name: extract-utilities
ready: true
resources: {}
restartCount: 0
started: false
state:
terminated:
containerID: cri-o://2dc94a3b6abca72fe0af761b32924c73640cdbfa1cd9c0a5ddf25d07b966b041
exitCode: 0
finishedAt: "2025-09-01T06:19:29Z"
reason: Completed
startedAt: "2025-09-01T06:19:29Z"
user:
linux:
gid: 0
supplementalGroups:
- 0
- 1000770000
uid: 1000770000
volumeMounts:
- mountPath: /utilities
name: utilities
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-gr52q
readOnly: true
recursiveReadOnly: Disabled
- containerID: cri-o://14e99a3cf4895fc6726233bec0d4dbd6fc90daf0f2a360df091851800435ad45
image: quay.io/openshifttest/nginxolm-operator-index:nginxolm81995-binless
imageID: quay.io/openshifttest/nginxolm-operator-index@sha256:c5d9895e57303dc16a9f903ad2973b8367fbdcc3593b5d81eee908a1f3f14035
lastState: {}
name: extract-content
ready: true
resources: {}
restartCount: 0
started: false
state:
terminated:
containerID: cri-o://14e99a3cf4895fc6726233bec0d4dbd6fc90daf0f2a360df091851800435ad45
exitCode: 0
finishedAt: "2025-09-01T06:19:32Z"
reason: Completed
startedAt: "2025-09-01T06:19:32Z"
user:
linux:
gid: 0
supplementalGroups:
- 0
- 1000770000
uid: 1000770000
volumeMounts:
- mountPath: /utilities
name: utilities
- mountPath: /extracted-catalog
name: catalog-content
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-gr52q
readOnly: true
recursiveReadOnly: Disabled
phase: Running
podIP: 10.131.0.76
podIPs:
- ip: 10.131.0.76
qosClass: Burstable
startTime: "2025-09-01T06:19:28Z"
2.
3.
Actual results:
message: | time="2025-09-01T06:19:33Z" level=info msg="starting pprof endpoint" address="localhost:6060" time="2025-09-01T06:19:33Z" level=fatal msg="mkdir /tmp/opm-serve-cache-3951864160: read-only file system"
Expected results:
catsrc is ready
Additional info:
- clones
-
OCPBUGS-61085 status of cacheless catalogsource is TRANSIENT_FAILURE
-
- Verified
-