-- Logs begin at Wed 2022-11-16 11:49:03 UTC, end at Wed 2022-11-16 17:07:21 UTC. -- Nov 16 11:49:03.009543 localhost kernel: Linux version 4.18.0-305.62.1.el8_4.x86_64 (mockbuild@x86-vm-07.build.eng.bos.redhat.com) (gcc version 8.4.1 20200928 (Red Hat 8.4.1-1) (GCC)) #1 SMP Thu Aug 11 12:07:27 EDT 2022 Nov 16 11:49:03.009564 localhost kernel: Command line: BOOT_IMAGE=(hd0,gpt3)/ostree/rhcos-dfe3636c0ecb6ffa92f58233688e78d3a8f6fec44b763bfd21d40bdea12ba3ff/vmlinuz-4.18.0-305.62.1.el8_4.x86_64 random.trust_cpu=on console=tty0 console=ttyS0,115200n8 ignition.firstboot ostree=/ostree/boot.1/rhcos/dfe3636c0ecb6ffa92f58233688e78d3a8f6fec44b763bfd21d40bdea12ba3ff/0 ignition.platform.id=azure Nov 16 11:49:03.009576 localhost kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' Nov 16 11:49:03.009582 localhost kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' Nov 16 11:49:03.009588 localhost kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' Nov 16 11:49:03.009594 localhost kernel: x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 Nov 16 11:49:03.009601 localhost kernel: x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. Nov 16 11:49:03.009607 localhost kernel: BIOS-provided physical RAM map: Nov 16 11:49:03.009613 localhost kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable Nov 16 11:49:03.009621 localhost kernel: BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved Nov 16 11:49:03.009627 localhost kernel: BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved Nov 16 11:49:03.009633 localhost kernel: BIOS-e820: [mem 0x0000000000100000-0x000000003ffeffff] usable Nov 16 11:49:03.009639 localhost kernel: BIOS-e820: [mem 0x000000003fff0000-0x000000003fffefff] ACPI data Nov 16 11:49:03.009645 localhost kernel: BIOS-e820: [mem 0x000000003ffff000-0x000000003fffffff] ACPI NVS Nov 16 11:49:03.009652 localhost kernel: BIOS-e820: [mem 0x0000000100000000-0x00000004bfffffff] usable Nov 16 11:49:03.009661 localhost kernel: NX (Execute Disable) protection: active Nov 16 11:49:03.009667 localhost kernel: SMBIOS 2.3 present. Nov 16 11:49:03.009673 localhost kernel: DMI: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008 12/07/2018 Nov 16 11:49:03.009680 localhost kernel: Hypervisor detected: Microsoft Hyper-V Nov 16 11:49:03.009686 localhost kernel: Hyper-V: features 0x2e7f, hints 0x60c2c, misc 0xed7b2 Nov 16 11:49:03.009707 localhost kernel: Hyper-V Host Build:20348-10.0-1-0.1272 Nov 16 11:49:03.009713 localhost kernel: Hyper-V: LAPIC Timer Frequency: 0x30d40 Nov 16 11:49:03.009735 localhost kernel: tsc: Marking TSC unstable due to running on Hyper-V Nov 16 11:49:03.009740 localhost kernel: Hyper-V: Using hypercall for remote TLB flush Nov 16 11:49:03.009746 localhost kernel: clocksource: hyperv_clocksource_tsc_page: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns Nov 16 11:49:03.009752 localhost kernel: tsc: Detected 2294.686 MHz processor Nov 16 11:49:03.009757 localhost kernel: e820: update [mem 0x00000000-0x00000fff] usable ==> reserved Nov 16 11:49:03.009766 localhost kernel: e820: remove [mem 0x000a0000-0x000fffff] usable Nov 16 11:49:03.009772 localhost kernel: last_pfn = 0x4c0000 max_arch_pfn = 0x400000000 Nov 16 11:49:03.009777 localhost kernel: MTRR default type: uncachable Nov 16 11:49:03.009783 localhost kernel: MTRR fixed ranges enabled: Nov 16 11:49:03.009788 localhost kernel: 00000-9FFFF write-back Nov 16 11:49:03.009794 localhost kernel: A0000-DFFFF uncachable Nov 16 11:49:03.009799 localhost kernel: E0000-FFFFF write-back Nov 16 11:49:03.009805 localhost kernel: MTRR variable ranges enabled: Nov 16 11:49:03.009810 localhost kernel: 0 base 000000000000 mask 3FFFC0000000 write-back Nov 16 11:49:03.009816 localhost kernel: 1 base 000100000000 mask 3FF000000000 write-back Nov 16 11:49:03.009821 localhost kernel: 2 disabled Nov 16 11:49:03.009827 localhost kernel: 3 disabled Nov 16 11:49:03.009835 localhost kernel: 4 disabled Nov 16 11:49:03.009844 localhost kernel: 5 disabled Nov 16 11:49:03.009850 localhost kernel: 6 disabled Nov 16 11:49:03.009856 localhost kernel: 7 disabled Nov 16 11:49:03.009862 localhost kernel: x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT Nov 16 11:49:03.009885 localhost kernel: e820: update [mem 0x40000000-0xffffffff] usable ==> reserved Nov 16 11:49:03.009891 localhost kernel: last_pfn = 0x3fff0 max_arch_pfn = 0x400000000 Nov 16 11:49:03.009915 localhost kernel: found SMP MP-table at [mem 0x000ff780-0x000ff78f] Nov 16 11:49:03.009936 localhost kernel: Using GB pages for direct mapping Nov 16 11:49:03.009945 localhost kernel: BRK [0x35c002000, 0x35c002fff] PGTABLE Nov 16 11:49:03.009951 localhost kernel: BRK [0x35c003000, 0x35c003fff] PGTABLE Nov 16 11:49:03.009957 localhost kernel: BRK [0x35c004000, 0x35c004fff] PGTABLE Nov 16 11:49:03.009963 localhost kernel: BRK [0x35c005000, 0x35c005fff] PGTABLE Nov 16 11:49:03.009969 localhost kernel: BRK [0x35c006000, 0x35c006fff] PGTABLE Nov 16 11:49:03.009975 localhost kernel: RAMDISK: [mem 0x2d1c3000-0x328d9fff] Nov 16 11:49:03.009981 localhost kernel: ACPI: Early table checksum verification disabled Nov 16 11:49:03.009987 localhost kernel: ACPI: RSDP 0x00000000000F5C00 000014 (v00 ACPIAM) Nov 16 11:49:03.009993 localhost kernel: ACPI: RSDT 0x000000003FFF0000 000040 (v01 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:49:03.010016 localhost kernel: ACPI: FACP 0x000000003FFF0200 000081 (v02 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:49:03.010022 localhost kernel: ACPI: DSDT 0x000000003FFF1D24 003CD5 (v01 MSFTVM MSFTVM02 00000002 INTL 02002026) Nov 16 11:49:03.010028 localhost kernel: ACPI: FACS 0x000000003FFFF000 000040 Nov 16 11:49:03.010038 localhost kernel: ACPI: WAET 0x000000003FFF1A80 000028 (v01 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:49:03.010044 localhost kernel: ACPI: SLIC 0x000000003FFF1AC0 000176 (v01 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:49:03.010050 localhost kernel: ACPI: OEM0 0x000000003FFF1CC0 000064 (v01 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:49:03.010057 localhost kernel: ACPI: SRAT 0x000000003FFF0800 0001A0 (v02 VRTUAL MICROSFT 00000001 MSFT 00000001) Nov 16 11:49:03.010063 localhost kernel: ACPI: APIC 0x000000003FFF0300 000092 (v01 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:49:03.010069 localhost kernel: ACPI: OEMB 0x000000003FFFF040 000064 (v01 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:49:03.010075 localhost kernel: ACPI: Local APIC address 0xfee00000 Nov 16 11:49:03.010081 localhost kernel: SRAT: PXM 0 -> APIC 0x00 -> Node 0 Nov 16 11:49:03.010088 localhost kernel: SRAT: PXM 0 -> APIC 0x01 -> Node 0 Nov 16 11:49:03.010094 localhost kernel: SRAT: PXM 0 -> APIC 0x02 -> Node 0 Nov 16 11:49:03.010103 localhost kernel: SRAT: PXM 0 -> APIC 0x03 -> Node 0 Nov 16 11:49:03.010109 localhost kernel: SRAT: PXM 0 -> APIC 0x04 -> Node 0 Nov 16 11:49:03.010115 localhost kernel: SRAT: PXM 0 -> APIC 0x05 -> Node 0 Nov 16 11:49:03.010121 localhost kernel: SRAT: PXM 0 -> APIC 0x06 -> Node 0 Nov 16 11:49:03.010127 localhost kernel: SRAT: PXM 0 -> APIC 0x07 -> Node 0 Nov 16 11:49:03.010134 localhost kernel: ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x3fffffff] hotplug Nov 16 11:49:03.010140 localhost kernel: ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0x4bfffffff] hotplug Nov 16 11:49:03.010146 localhost kernel: ACPI: SRAT: Node 0 PXM 0 [mem 0x4c0200000-0xfdfffffff] hotplug Nov 16 11:49:03.010153 localhost kernel: ACPI: SRAT: Node 0 PXM 0 [mem 0x1000000000-0xffffffffff] hotplug Nov 16 11:49:03.010159 localhost kernel: ACPI: SRAT: Node 0 PXM 0 [mem 0x10000200000-0x1ffffffffff] hotplug Nov 16 11:49:03.010165 localhost kernel: ACPI: SRAT: Node 0 PXM 0 [mem 0x20000200000-0x3ffffffffff] hotplug Nov 16 11:49:03.010172 localhost kernel: NUMA: Node 0 [mem 0x00000000-0x3fffffff] + [mem 0x100000000-0x4bfffffff] -> [mem 0x00000000-0x4bfffffff] Nov 16 11:49:03.010181 localhost kernel: NODE_DATA(0) allocated [mem 0x4bffd6000-0x4bfffffff] Nov 16 11:49:03.010187 localhost kernel: Zone ranges: Nov 16 11:49:03.010193 localhost kernel: DMA [mem 0x0000000000001000-0x0000000000ffffff] Nov 16 11:49:03.010200 localhost kernel: DMA32 [mem 0x0000000001000000-0x00000000ffffffff] Nov 16 11:49:03.010206 localhost kernel: Normal [mem 0x0000000100000000-0x00000004bfffffff] Nov 16 11:49:03.010224 localhost kernel: Device empty Nov 16 11:49:03.010231 localhost kernel: Movable zone start for each node Nov 16 11:49:03.010237 localhost kernel: Early memory node ranges Nov 16 11:49:03.010243 localhost kernel: node 0: [mem 0x0000000000001000-0x000000000009efff] Nov 16 11:49:03.010250 localhost kernel: node 0: [mem 0x0000000000100000-0x000000003ffeffff] Nov 16 11:49:03.010256 localhost kernel: node 0: [mem 0x0000000100000000-0x00000004bfffffff] Nov 16 11:49:03.010262 localhost kernel: Zeroed struct page in unavailable ranges: 114 pages Nov 16 11:49:03.010285 localhost kernel: Initmem setup node 0 [mem 0x0000000000001000-0x00000004bfffffff] Nov 16 11:49:03.010294 localhost kernel: On node 0 totalpages: 4194190 Nov 16 11:49:03.010300 localhost kernel: DMA zone: 64 pages used for memmap Nov 16 11:49:03.010306 localhost kernel: DMA zone: 21 pages reserved Nov 16 11:49:03.010312 localhost kernel: DMA zone: 3998 pages, LIFO batch:0 Nov 16 11:49:03.010318 localhost kernel: DMA32 zone: 4032 pages used for memmap Nov 16 11:49:03.010324 localhost kernel: DMA32 zone: 258032 pages, LIFO batch:63 Nov 16 11:49:03.010330 localhost kernel: Normal zone: 61440 pages used for memmap Nov 16 11:49:03.010336 localhost kernel: Normal zone: 3932160 pages, LIFO batch:63 Nov 16 11:49:03.010342 localhost kernel: ACPI: PM-Timer IO Port: 0x408 Nov 16 11:49:03.010348 localhost kernel: ACPI: Local APIC address 0xfee00000 Nov 16 11:49:03.010354 localhost kernel: ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) Nov 16 11:49:03.010360 localhost kernel: IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 Nov 16 11:49:03.010366 localhost kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) Nov 16 11:49:03.010375 localhost kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) Nov 16 11:49:03.010381 localhost kernel: ACPI: IRQ0 used by override. Nov 16 11:49:03.010387 localhost kernel: ACPI: IRQ9 used by override. Nov 16 11:49:03.010393 localhost kernel: Using ACPI (MADT) for SMP configuration information Nov 16 11:49:03.010399 localhost kernel: smpboot: Allowing 8 CPUs, 0 hotplug CPUs Nov 16 11:49:03.010405 localhost kernel: PM: Registered nosave memory: [mem 0x00000000-0x00000fff] Nov 16 11:49:03.010411 localhost kernel: PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] Nov 16 11:49:03.010417 localhost kernel: PM: Registered nosave memory: [mem 0x000a0000-0x000dffff] Nov 16 11:49:03.010423 localhost kernel: PM: Registered nosave memory: [mem 0x000e0000-0x000fffff] Nov 16 11:49:03.010429 localhost kernel: PM: Registered nosave memory: [mem 0x3fff0000-0x3fffefff] Nov 16 11:49:03.010445 localhost kernel: PM: Registered nosave memory: [mem 0x3ffff000-0x3fffffff] Nov 16 11:49:03.010452 localhost kernel: PM: Registered nosave memory: [mem 0x40000000-0xffffffff] Nov 16 11:49:03.010458 localhost kernel: [mem 0x40000000-0xffffffff] available for PCI devices Nov 16 11:49:03.010464 localhost kernel: Booting paravirtualized kernel on Hyper-V Nov 16 11:49:03.010471 localhost kernel: clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns Nov 16 11:49:03.010480 localhost kernel: setup_percpu: NR_CPUS:8192 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1 Nov 16 11:49:03.010486 localhost kernel: percpu: Embedded 54 pages/cpu s184320 r8192 d28672 u262144 Nov 16 11:49:03.010492 localhost kernel: pcpu-alloc: s184320 r8192 d28672 u262144 alloc=1*2097152 Nov 16 11:49:03.010499 localhost kernel: pcpu-alloc: [0] 0 1 2 3 4 5 6 7 Nov 16 11:49:03.010505 localhost kernel: Hyper-V: PV spinlocks enabled Nov 16 11:49:03.010511 localhost kernel: PV qspinlock hash table entries: 256 (order: 0, 4096 bytes) Nov 16 11:49:03.010517 localhost kernel: Built 1 zonelists, mobility grouping on. Total pages: 4128633 Nov 16 11:49:03.010523 localhost kernel: Policy zone: Normal Nov 16 11:49:03.010531 localhost kernel: Kernel command line: BOOT_IMAGE=(hd0,gpt3)/ostree/rhcos-dfe3636c0ecb6ffa92f58233688e78d3a8f6fec44b763bfd21d40bdea12ba3ff/vmlinuz-4.18.0-305.62.1.el8_4.x86_64 random.trust_cpu=on console=tty0 console=ttyS0,115200n8 ignition.firstboot ostree=/ostree/boot.1/rhcos/dfe3636c0ecb6ffa92f58233688e78d3a8f6fec44b763bfd21d40bdea12ba3ff/0 ignition.platform.id=azure Nov 16 11:49:03.010538 localhost kernel: Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. Nov 16 11:49:03.010545 localhost kernel: Memory: 1024392K/16776760K available (12293K kernel code, 2225K rwdata, 7716K rodata, 2480K init, 14040K bss, 464416K reserved, 0K cma-reserved) Nov 16 11:49:03.010553 localhost kernel: SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1 Nov 16 11:49:03.010575 localhost kernel: Kernel/User page tables isolation: enabled Nov 16 11:49:03.010581 localhost kernel: ftrace: allocating 38610 entries in 151 pages Nov 16 11:49:03.010587 localhost kernel: ftrace: allocated 151 pages with 5 groups Nov 16 11:49:03.010594 localhost kernel: rcu: Hierarchical RCU implementation. Nov 16 11:49:03.010601 localhost kernel: rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=8. Nov 16 11:49:03.010607 localhost kernel: rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. Nov 16 11:49:03.010613 localhost kernel: rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8 Nov 16 11:49:03.010619 localhost kernel: NR_IRQS: 524544, nr_irqs: 488, preallocated irqs: 16 Nov 16 11:49:03.010625 localhost kernel: random: crng done (trusting CPU's manufacturer) Nov 16 11:49:03.010631 localhost kernel: Console: colour VGA+ 80x25 Nov 16 11:49:03.010640 localhost kernel: printk: console [tty0] enabled Nov 16 11:49:03.010645 localhost kernel: printk: console [ttyS0] enabled Nov 16 11:49:03.010651 localhost kernel: ACPI: Core revision 20200925 Nov 16 11:49:03.010657 localhost kernel: APIC: Switch to symmetric I/O mode setup Nov 16 11:49:03.010663 localhost kernel: Hyper-V: Using IPI hypercalls Nov 16 11:49:03.010670 localhost kernel: Hyper-V: Using enlightened APIC (xapic mode) Nov 16 11:49:03.010676 localhost kernel: ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 Nov 16 11:49:03.010682 localhost kernel: Calibrating delay loop (skipped), value calculated using timer frequency.. 4589.37 BogoMIPS (lpj=2294686) Nov 16 11:49:03.010689 localhost kernel: pid_max: default: 32768 minimum: 301 Nov 16 11:49:03.010695 localhost kernel: Security Framework initialized Nov 16 11:49:03.010717 localhost kernel: Yama: becoming mindful. Nov 16 11:49:03.010725 localhost kernel: SELinux: Initializing. Nov 16 11:49:03.010731 localhost kernel: Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes) Nov 16 11:49:03.010737 localhost kernel: Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes) Nov 16 11:49:03.010744 localhost kernel: Mount-cache hash table entries: 32768 (order: 6, 262144 bytes) Nov 16 11:49:03.010750 localhost kernel: Mountpoint-cache hash table entries: 32768 (order: 6, 262144 bytes) Nov 16 11:49:03.010758 localhost kernel: Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8 Nov 16 11:49:03.010764 localhost kernel: Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4 Nov 16 11:49:03.010770 localhost kernel: FEATURE SPEC_CTRL Not Present Nov 16 11:49:03.010777 localhost kernel: FEATURE IBPB_SUPPORT Not Present Nov 16 11:49:03.010783 localhost kernel: Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization Nov 16 11:49:03.010789 localhost kernel: Spectre V2 : Mitigation: Full generic retpoline Nov 16 11:49:03.010795 localhost kernel: Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch Nov 16 11:49:03.010801 localhost kernel: Speculative Store Bypass: Vulnerable Nov 16 11:49:03.010807 localhost kernel: TAA: Mitigation: Clear CPU buffers Nov 16 11:49:03.010815 localhost kernel: MDS: Mitigation: Clear CPU buffers Nov 16 11:49:03.010821 localhost kernel: Freeing SMP alternatives memory: 32K Nov 16 11:49:03.010827 localhost kernel: smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz (family: 0x6, model: 0x4f, stepping: 0x1) Nov 16 11:49:03.010834 localhost kernel: Performance Events: unsupported p6 CPU model 79 no PMU driver, software events only. Nov 16 11:49:03.010840 localhost kernel: rcu: Hierarchical SRCU implementation. Nov 16 11:49:03.010846 localhost kernel: NMI watchdog: Perf NMI watchdog permanently disabled Nov 16 11:49:03.010852 localhost kernel: smp: Bringing up secondary CPUs ... Nov 16 11:49:03.010858 localhost kernel: x86: Booting SMP configuration: Nov 16 11:49:03.010864 localhost kernel: .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 Nov 16 11:49:03.010870 localhost kernel: smp: Brought up 1 node, 8 CPUs Nov 16 11:49:03.010878 localhost kernel: smpboot: Max logical packages: 1 Nov 16 11:49:03.010884 localhost kernel: smpboot: Total of 8 processors activated (36714.97 BogoMIPS) Nov 16 11:49:03.010890 localhost kernel: node 0 deferred pages initialised in 16ms Nov 16 11:49:03.010896 localhost kernel: devtmpfs: initialized Nov 16 11:49:03.010902 localhost kernel: x86/mm: Memory block size: 128MB Nov 16 11:49:03.010908 localhost kernel: PM: Registering ACPI NVS region [mem 0x3ffff000-0x3fffffff] (4096 bytes) Nov 16 11:49:03.010914 localhost kernel: clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns Nov 16 11:49:03.010921 localhost kernel: futex hash table entries: 2048 (order: 5, 131072 bytes) Nov 16 11:49:03.010927 localhost kernel: pinctrl core: initialized pinctrl subsystem Nov 16 11:49:03.010935 localhost kernel: NET: Registered protocol family 16 Nov 16 11:49:03.010941 localhost kernel: DMA: preallocated 2048 KiB GFP_KERNEL pool for atomic allocations Nov 16 11:49:03.010947 localhost kernel: DMA: preallocated 2048 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations Nov 16 11:49:03.010954 localhost kernel: DMA: preallocated 2048 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations Nov 16 11:49:03.010960 localhost kernel: audit: initializing netlink subsys (disabled) Nov 16 11:49:03.010966 localhost kernel: audit: type=2000 audit(1668599336.092:1): state=initialized audit_enabled=0 res=1 Nov 16 11:49:03.010972 localhost kernel: cpuidle: using governor menu Nov 16 11:49:03.010978 localhost kernel: ACPI: bus type PCI registered Nov 16 11:49:03.010984 localhost kernel: acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 Nov 16 11:49:03.010992 localhost kernel: PCI: Using configuration type 1 for base access Nov 16 11:49:03.010999 localhost kernel: mtrr: your CPUs had inconsistent variable MTRR settings Nov 16 11:49:03.011005 localhost kernel: mtrr: probably your BIOS does not setup all CPUs. Nov 16 11:49:03.011011 localhost kernel: mtrr: corrected configuration. Nov 16 11:49:03.011017 localhost kernel: HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages Nov 16 11:49:03.011040 localhost kernel: HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages Nov 16 11:49:03.011046 localhost kernel: cryptd: max_cpu_qlen set to 1000 Nov 16 11:49:03.011069 localhost kernel: ACPI: Added _OSI(Module Device) Nov 16 11:49:03.011076 localhost kernel: ACPI: Added _OSI(Processor Device) Nov 16 11:49:03.011082 localhost kernel: ACPI: Added _OSI(3.0 _SCP Extensions) Nov 16 11:49:03.011091 localhost kernel: ACPI: Added _OSI(Processor Aggregator Device) Nov 16 11:49:03.011098 localhost kernel: ACPI: Added _OSI(Linux-Dell-Video) Nov 16 11:49:03.011104 localhost kernel: ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio) Nov 16 11:49:03.011111 localhost kernel: ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics) Nov 16 11:49:03.011117 localhost kernel: ACPI: 1 ACPI AML tables successfully acquired and loaded Nov 16 11:49:03.011123 localhost kernel: ACPI: Interpreter enabled Nov 16 11:49:03.011130 localhost kernel: ACPI: (supports S0 S5) Nov 16 11:49:03.011136 localhost kernel: ACPI: Using IOAPIC for interrupt routing Nov 16 11:49:03.011160 localhost kernel: PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug Nov 16 11:49:03.011170 localhost kernel: ACPI: Enabled 1 GPEs in block 00 to 0F Nov 16 11:49:03.011176 localhost kernel: ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) Nov 16 11:49:03.011374 localhost kernel: acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI EDR HPX-Type3] Nov 16 11:49:03.011555 localhost kernel: acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. Nov 16 11:49:03.011580 localhost kernel: PCI host bridge to bus 0000:00 Nov 16 11:49:03.011699 localhost kernel: pci_bus 0000:00: root bus resource [mem 0xfe0000000-0xfffffffff window] Nov 16 11:49:03.011803 localhost kernel: pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] Nov 16 11:49:03.012011 localhost kernel: pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] Nov 16 11:49:03.012189 localhost kernel: pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] Nov 16 11:49:03.012348 localhost kernel: pci_bus 0000:00: root bus resource [mem 0x40000000-0xfffbffff window] Nov 16 11:49:03.012459 localhost kernel: pci_bus 0000:00: root bus resource [bus 00-ff] Nov 16 11:49:03.012625 localhost kernel: pci 0000:00:00.0: [8086:7192] type 00 class 0x060000 Nov 16 11:49:03.012800 localhost kernel: pci 0000:00:07.0: [8086:7110] type 00 class 0x060100 Nov 16 11:49:03.012939 localhost kernel: pci 0000:00:07.1: [8086:7111] type 00 class 0x010180 Nov 16 11:49:03.013074 localhost kernel: pci 0000:00:07.1: reg 0x20: [io 0xffa0-0xffaf] Nov 16 11:49:03.013188 localhost kernel: pci 0000:00:07.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] Nov 16 11:49:03.013356 localhost kernel: pci 0000:00:07.1: legacy IDE quirk: reg 0x14: [io 0x03f6] Nov 16 11:49:03.013487 localhost kernel: pci 0000:00:07.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] Nov 16 11:49:03.013739 localhost kernel: pci 0000:00:07.1: legacy IDE quirk: reg 0x1c: [io 0x0376] Nov 16 11:49:03.013886 localhost kernel: pci 0000:00:07.3: [8086:7113] type 00 class 0x068000 Nov 16 11:49:03.013905 localhost kernel: * Found PM-Timer Bug on the chipset. Due to workarounds for a bug, * this clock source is slow. Consider trying other clock sources Nov 16 11:49:03.014033 localhost kernel: pci 0000:00:07.3: quirk: [io 0x0400-0x043f] claimed by PIIX4 ACPI Nov 16 11:49:03.014186 localhost kernel: pci 0000:00:08.0: [1414:5353] type 00 class 0x030000 Nov 16 11:49:03.014328 localhost kernel: pci 0000:00:08.0: reg 0x10: [mem 0xf8000000-0xfbffffff] Nov 16 11:49:03.014340 localhost kernel: ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 7 9 10 *11 12 14 15) Nov 16 11:49:03.014347 localhost kernel: ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 7 9 10 11 12 14 15) *0, disabled. Nov 16 11:49:03.014353 localhost kernel: ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 7 9 10 11 12 14 15) *0, disabled. Nov 16 11:49:03.014360 localhost kernel: ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 7 9 10 11 12 14 15) *0, disabled. Nov 16 11:49:03.014371 localhost kernel: iommu: Default domain type: Passthrough Nov 16 11:49:03.014522 localhost kernel: pci 0000:00:08.0: vgaarb: setting as boot VGA device Nov 16 11:49:03.014644 localhost kernel: pci 0000:00:08.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none Nov 16 11:49:03.014770 localhost kernel: pci 0000:00:08.0: vgaarb: bridge control possible Nov 16 11:49:03.014780 localhost kernel: vgaarb: loaded Nov 16 11:49:03.014787 localhost kernel: SCSI subsystem initialized Nov 16 11:49:03.014795 localhost kernel: ACPI: bus type USB registered Nov 16 11:49:03.014802 localhost kernel: usbcore: registered new interface driver usbfs Nov 16 11:49:03.014812 localhost kernel: usbcore: registered new interface driver hub Nov 16 11:49:03.014819 localhost kernel: usbcore: registered new device driver usb Nov 16 11:49:03.014826 localhost kernel: pps_core: LinuxPPS API ver. 1 registered Nov 16 11:49:03.014833 localhost kernel: pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti Nov 16 11:49:03.014840 localhost kernel: PTP clock support registered Nov 16 11:49:03.014847 localhost kernel: EDAC MC: Ver: 3.0.0 Nov 16 11:49:03.014853 localhost kernel: PCI: Using ACPI for IRQ routing Nov 16 11:49:03.014860 localhost kernel: PCI: pci_cache_line_size set to 64 bytes Nov 16 11:49:03.014867 localhost kernel: e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff] Nov 16 11:49:03.014874 localhost kernel: e820: reserve RAM buffer [mem 0x3fff0000-0x3fffffff] Nov 16 11:49:03.014883 localhost kernel: NetLabel: Initializing Nov 16 11:49:03.014890 localhost kernel: NetLabel: domain hash size = 128 Nov 16 11:49:03.014897 localhost kernel: NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO Nov 16 11:49:03.014904 localhost kernel: NetLabel: unlabeled traffic allowed by default Nov 16 11:49:03.014911 localhost kernel: clocksource: Switched to clocksource hyperv_clocksource_tsc_page Nov 16 11:49:03.014917 localhost kernel: VFS: Disk quotas dquot_6.6.0 Nov 16 11:49:03.014924 localhost kernel: VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) Nov 16 11:49:03.014931 localhost kernel: pnp: PnP ACPI init Nov 16 11:49:03.015083 localhost kernel: pnp 00:00: Plug and Play ACPI device, IDs PNP0b00 (active) Nov 16 11:49:03.015253 localhost kernel: pnp 00:01: Plug and Play ACPI device, IDs PNP0303 PNP030b (active) Nov 16 11:49:03.015381 localhost kernel: pnp 00:02: Plug and Play ACPI device, IDs PNP0f03 PNP0f13 (active) Nov 16 11:49:03.015505 localhost kernel: pnp 00:03: [dma 0 disabled] Nov 16 11:49:03.015642 localhost kernel: pnp 00:03: Plug and Play ACPI device, IDs PNP0501 (active) Nov 16 11:49:03.015764 localhost kernel: pnp 00:04: [dma 0 disabled] Nov 16 11:49:03.015890 localhost kernel: pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active) Nov 16 11:49:03.016019 localhost kernel: pnp 00:05: [dma 2] Nov 16 11:49:03.016145 localhost kernel: pnp 00:05: Plug and Play ACPI device, IDs PNP0700 (active) Nov 16 11:49:03.016292 localhost kernel: system 00:06: [io 0x01e0-0x01ef] has been reserved Nov 16 11:49:03.016414 localhost kernel: system 00:06: [io 0x0160-0x016f] has been reserved Nov 16 11:49:03.016561 localhost kernel: system 00:06: [io 0x0278-0x027f] has been reserved Nov 16 11:49:03.016666 localhost kernel: system 00:06: [io 0x0378-0x037f] has been reserved Nov 16 11:49:03.016822 localhost kernel: system 00:06: [io 0x0678-0x067f] has been reserved Nov 16 11:49:03.016935 localhost kernel: system 00:06: [io 0x0778-0x077f] has been reserved Nov 16 11:49:03.017057 localhost kernel: system 00:06: [io 0x04d0-0x04d1] has been reserved Nov 16 11:49:03.017254 localhost kernel: system 00:06: Plug and Play ACPI device, IDs PNP0c02 (active) Nov 16 11:49:03.017392 localhost kernel: system 00:07: [io 0x0400-0x043f] has been reserved Nov 16 11:49:03.017509 localhost kernel: system 00:07: [io 0x0370-0x0371] has been reserved Nov 16 11:49:03.017633 localhost kernel: system 00:07: [io 0x0440-0x044f] has been reserved Nov 16 11:49:03.017776 localhost kernel: system 00:07: [mem 0xfec00000-0xfec00fff] could not be reserved Nov 16 11:49:03.017945 localhost kernel: system 00:07: [mem 0xfee00000-0xfee00fff] has been reserved Nov 16 11:49:03.018085 localhost kernel: system 00:07: Plug and Play ACPI device, IDs PNP0c02 (active) Nov 16 11:49:03.018224 localhost kernel: system 00:08: [mem 0x00000000-0x0009ffff] could not be reserved Nov 16 11:49:03.018391 localhost kernel: system 00:08: [mem 0x000c0000-0x000dffff] could not be reserved Nov 16 11:49:03.018521 localhost kernel: system 00:08: [mem 0x000e0000-0x000fffff] could not be reserved Nov 16 11:49:03.018899 localhost kernel: system 00:08: [mem 0x00100000-0x3fffffff] could not be reserved Nov 16 11:49:03.019069 localhost kernel: system 00:08: [mem 0xfffc0000-0xffffffff] has been reserved Nov 16 11:49:03.019184 localhost kernel: system 00:08: Plug and Play ACPI device, IDs PNP0c01 (active) Nov 16 11:49:03.019195 localhost kernel: pnp: PnP ACPI: found 9 devices Nov 16 11:49:03.019202 localhost kernel: clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns Nov 16 11:49:03.019350 localhost kernel: pci_bus 0000:00: resource 4 [mem 0xfe0000000-0xfffffffff window] Nov 16 11:49:03.019545 localhost kernel: pci_bus 0000:00: resource 5 [io 0x0000-0x0cf7 window] Nov 16 11:49:03.019667 localhost kernel: pci_bus 0000:00: resource 6 [io 0x0d00-0xffff window] Nov 16 11:49:03.019794 localhost kernel: pci_bus 0000:00: resource 7 [mem 0x000a0000-0x000bffff window] Nov 16 11:49:03.019907 localhost kernel: pci_bus 0000:00: resource 8 [mem 0x40000000-0xfffbffff window] Nov 16 11:49:03.019916 localhost kernel: NET: Registered protocol family 2 Nov 16 11:49:03.019924 localhost kernel: tcp_listen_portaddr_hash hash table entries: 8192 (order: 5, 131072 bytes) Nov 16 11:49:03.019930 localhost kernel: TCP established hash table entries: 131072 (order: 8, 1048576 bytes) Nov 16 11:49:03.019937 localhost kernel: TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) Nov 16 11:49:03.019944 localhost kernel: TCP: Hash tables configured (established 131072 bind 65536) Nov 16 11:49:03.019950 localhost kernel: MPTCP token hash table entries: 16384 (order: 6, 393216 bytes) Nov 16 11:49:03.019957 localhost kernel: UDP hash table entries: 8192 (order: 6, 262144 bytes) Nov 16 11:49:03.019963 localhost kernel: UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes) Nov 16 11:49:03.019974 localhost kernel: NET: Registered protocol family 1 Nov 16 11:49:03.019980 localhost kernel: NET: Registered protocol family 44 Nov 16 11:49:03.020114 localhost kernel: pci 0000:00:00.0: Limiting direct PCI/PCI transfers Nov 16 11:49:03.020277 localhost kernel: pci 0000:00:08.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] Nov 16 11:49:03.020429 localhost kernel: pci 0000:00:08.0: pci_fixup_video+0x0/0x110 took 10280 usecs Nov 16 11:49:03.020442 localhost kernel: PCI: CLS 0 bytes, default 64 Nov 16 11:49:03.020449 localhost kernel: Unpacking initramfs... Nov 16 11:49:03.020456 localhost kernel: Freeing initrd memory: 89180K Nov 16 11:49:03.020468 localhost kernel: PCI-DMA: Using software bounce buffering for IO (SWIOTLB) Nov 16 11:49:03.020482 localhost kernel: software IO TLB: mapped [mem 0x3bff0000-0x3fff0000] (64MB) Nov 16 11:49:03.020489 localhost kernel: Initialise system trusted keyrings Nov 16 11:49:03.020496 localhost kernel: Key type blacklist registered Nov 16 11:49:03.020503 localhost kernel: workingset: timestamp_bits=36 max_order=22 bucket_order=0 Nov 16 11:49:03.020509 localhost kernel: zbud: loaded Nov 16 11:49:03.020516 localhost kernel: pstore: using deflate compression Nov 16 11:49:03.020523 localhost kernel: Platform Keyring initialized Nov 16 11:49:03.020530 localhost kernel: NET: Registered protocol family 38 Nov 16 11:49:03.020536 localhost kernel: Key type asymmetric registered Nov 16 11:49:03.020547 localhost kernel: Asymmetric key parser 'x509' registered Nov 16 11:49:03.020554 localhost kernel: Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) Nov 16 11:49:03.020561 localhost kernel: io scheduler mq-deadline registered Nov 16 11:49:03.020567 localhost kernel: io scheduler kyber registered Nov 16 11:49:03.020574 localhost kernel: io scheduler bfq registered Nov 16 11:49:03.020581 localhost kernel: atomic64_test: passed for x86-64 platform with CX8 and with SSE Nov 16 11:49:03.020588 localhost kernel: shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 Nov 16 11:49:03.020595 localhost kernel: intel_idle: Please enable MWAIT in BIOS SETUP Nov 16 11:49:03.020602 localhost kernel: input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 Nov 16 11:49:03.020611 localhost kernel: ACPI: Power Button [PWRF] Nov 16 11:49:03.020618 localhost kernel: Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled Nov 16 11:49:03.020625 localhost kernel: 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A Nov 16 11:49:03.020632 localhost kernel: 00:04: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A Nov 16 11:49:03.020639 localhost kernel: Non-volatile memory driver v1.3 Nov 16 11:49:03.020645 localhost kernel: rdac: device handler registered Nov 16 11:49:03.020652 localhost kernel: hp_sw: device handler registered Nov 16 11:49:03.020659 localhost kernel: emc: device handler registered Nov 16 11:49:03.020666 localhost kernel: alua: device handler registered Nov 16 11:49:03.020673 localhost kernel: libphy: Fixed MDIO Bus: probed Nov 16 11:49:03.020694 localhost kernel: ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver Nov 16 11:49:03.020704 localhost kernel: ehci-pci: EHCI PCI platform driver Nov 16 11:49:03.020711 localhost kernel: ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver Nov 16 11:49:03.020721 localhost kernel: ohci-pci: OHCI PCI platform driver Nov 16 11:49:03.020728 localhost kernel: uhci_hcd: USB Universal Host Controller Interface driver Nov 16 11:49:03.020735 localhost kernel: usbcore: registered new interface driver usbserial_generic Nov 16 11:49:03.020742 localhost kernel: usbserial: USB Serial support registered for generic Nov 16 11:49:03.020749 localhost kernel: i8042: PNP: PS/2 Controller [PNP0303:PS2K,PNP0f03:PS2M] at 0x60,0x64 irq 1,12 Nov 16 11:49:03.020756 localhost kernel: serio: i8042 KBD port at 0x60,0x64 irq 1 Nov 16 11:49:03.020763 localhost kernel: serio: i8042 AUX port at 0x60,0x64 irq 12 Nov 16 11:49:03.020771 localhost kernel: mousedev: PS/2 mouse device common for all mice Nov 16 11:49:03.020898 localhost kernel: rtc_cmos 00:00: RTC can wake from S4 Nov 16 11:49:03.021058 localhost kernel: rtc_cmos 00:00: registered as rtc0 Nov 16 11:49:03.021173 localhost kernel: rtc_cmos 00:00: alarms up to one month, 114 bytes nvram Nov 16 11:49:03.021183 localhost kernel: intel_pstate: CPU model not supported Nov 16 11:49:03.021191 localhost kernel: input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 Nov 16 11:49:03.021198 localhost kernel: hid: raw HID events driver (C) Jiri Kosina Nov 16 11:49:03.021205 localhost kernel: usbcore: registered new interface driver usbhid Nov 16 11:49:03.021228 localhost kernel: usbhid: USB HID core driver Nov 16 11:49:03.021235 localhost kernel: drop_monitor: Initializing network drop monitor service Nov 16 11:49:03.021243 localhost kernel: Initializing XFRM netlink socket Nov 16 11:49:03.021250 localhost kernel: NET: Registered protocol family 10 Nov 16 11:49:03.021263 localhost kernel: Segment Routing with IPv6 Nov 16 11:49:03.021271 localhost kernel: NET: Registered protocol family 17 Nov 16 11:49:03.021278 localhost kernel: mpls_gso: MPLS GSO support Nov 16 11:49:03.021285 localhost kernel: AVX2 version of gcm_enc/dec engaged. Nov 16 11:49:03.021292 localhost kernel: AES CTR mode by8 optimization enabled Nov 16 11:49:03.021455 localhost kernel: psmouse serio1: trackpoint: failed to get extended button data, assuming 3 buttons Nov 16 11:49:03.021468 localhost kernel: registered taskstats version 1 Nov 16 11:49:03.021475 localhost kernel: Loading compiled-in X.509 certificates Nov 16 11:49:03.021486 localhost kernel: Loaded X.509 cert 'Red Hat Enterprise Linux kernel signing key: 50d7cbcde40d8e71bbdeeb056f556f1e4cb31eba' Nov 16 11:49:03.021496 localhost kernel: Loaded X.509 cert 'Red Hat Enterprise Linux Driver Update Program (key 3): bf57f3e87362bc7229d9f465321773dfd1f77a80' Nov 16 11:49:03.021504 localhost kernel: Loaded X.509 cert 'Red Hat Enterprise Linux kpatch signing key: 4d38fd864ebe18c5f0b72e3852e2014c3a676fc8' Nov 16 11:49:03.021510 localhost kernel: zswap: loaded using pool lzo/zbud Nov 16 11:49:03.021518 localhost kernel: page_owner is disabled Nov 16 11:49:03.021525 localhost kernel: Key type big_key registered Nov 16 11:49:03.021532 localhost kernel: Key type encrypted registered Nov 16 11:49:03.021539 localhost kernel: ima: No TPM chip found, activating TPM-bypass! Nov 16 11:49:03.021546 localhost kernel: ima: Allocated hash algorithm: sha256 Nov 16 11:49:03.021555 localhost kernel: ima: No architecture policies found Nov 16 11:49:03.021563 localhost kernel: evm: Initialising EVM extended attributes: Nov 16 11:49:03.021570 localhost kernel: evm: security.selinux Nov 16 11:49:03.021577 localhost kernel: evm: security.ima Nov 16 11:49:03.021584 localhost kernel: evm: security.capability Nov 16 11:49:03.021591 localhost kernel: evm: HMAC attrs: 0x1 Nov 16 11:49:03.021743 localhost kernel: rtc_cmos 00:00: setting system clock to 2022-11-16 11:49:02 UTC (1668599342) Nov 16 11:49:03.021755 localhost kernel: Unstable clock detected, switching default tracing clock to "global" If you want to keep using the local clock, then add: "trace_clock=local" on the kernel command line Nov 16 11:49:03.021763 localhost kernel: Freeing unused decrypted memory: 2036K Nov 16 11:49:03.021773 localhost kernel: Freeing unused kernel memory: 2480K Nov 16 11:49:03.021780 localhost kernel: Write protecting the kernel read-only data: 22528k Nov 16 11:49:03.021787 localhost kernel: Freeing unused kernel memory: 2012K Nov 16 11:49:03.021795 localhost kernel: Freeing unused kernel memory: 476K Nov 16 11:49:03.021828 localhost systemd-journald[374]: Missed 10 kernel messages Nov 16 11:49:03.021846 localhost kernel: fuse: init (API version 7.31) Nov 16 11:49:03.021854 localhost systemd-journald[374]: Journal started Nov 16 11:49:03.021898 localhost systemd-journald[374]: Runtime journal (/run/log/journal/81c3f50d0967412d96a4f56848942d9f) is 8.0M, max 801.1M, 793.1M free. Nov 16 11:49:03.018004 localhost systemd-modules-load[388]: Inserted module 'fuse' Nov 16 11:49:03.018070 localhost systemd-modules-load[388]: Module 'msr' is builtin Nov 16 11:49:03.049232 localhost systemd-journald[374]: Missed 2 kernel messages Nov 16 11:49:03.049232 localhost kernel: Loading iSCSI transport class v2.0-870. Nov 16 11:49:03.383311 localhost systemd[1]: systemd-vconsole-setup.service: Succeeded. Nov 16 11:49:03.383632 localhost systemd[1]: Started Setup Virtual Console. Nov 16 11:49:03.401946 localhost systemd[1]: Started Create Static Device Nodes in /dev. Nov 16 11:49:03.415498 localhost systemd[1]: Starting dracut ask for additional cmdline parameters... Nov 16 11:49:03.427192 localhost systemd[1]: Starting Apply Kernel Variables... Nov 16 11:49:03.449607 localhost systemd[1]: Started dracut ask for additional cmdline parameters. Nov 16 11:49:03.463054 localhost systemd[1]: Starting dracut cmdline hook... Nov 16 11:49:03.481728 localhost dracut-cmdline[423]: dracut-410.84.202210040010-0 dracut-049-137.git20220131.el8_4.1 Nov 16 11:49:03.492396 localhost dracut-cmdline[423]: Using kernel command line parameters: ip=auto rd.driver.pre=dm_multipath BOOT_IMAGE=(hd0,gpt3)/ostree/rhcos-dfe3636c0ecb6ffa92f58233688e78d3a8f6fec44b763bfd21d40bdea12ba3ff/vmlinuz-4.18.0-305.62.1.el8_4.x86_64 random.trust_cpu=on console=tty0 console=ttyS0,115200n8 ignition.firstboot ostree=/ostree/boot.1/rhcos/dfe3636c0ecb6ffa92f58233688e78d3a8f6fec44b763bfd21d40bdea12ba3ff/0 ignition.platform.id=azure Nov 16 11:49:03.497266 localhost systemd[1]: Started Apply Kernel Variables. Nov 16 11:49:03.610244 localhost systemd-journald[374]: Missed 9 kernel messages Nov 16 11:49:03.610244 localhost kernel: iscsi: registered transport (tcp) Nov 16 11:49:03.656835 localhost kernel: iscsi: registered transport (qla4xxx) Nov 16 11:49:03.656879 localhost kernel: QLogic iSCSI HBA Driver Nov 16 11:49:03.674240 localhost kernel: libcxgbi:libcxgbi_init_module: Chelsio iSCSI driver library libcxgbi v0.9.1-ko (Apr. 2015) Nov 16 11:49:03.716797 localhost kernel: Chelsio T4-T6 iSCSI Driver cxgb4i v0.9.5-ko (Apr. 2015) Nov 16 11:49:03.716831 localhost kernel: iscsi: registered transport (cxgb4i) Nov 16 11:49:03.730238 localhost kernel: cnic: QLogic cnicDriver v2.5.22 (July 20, 2015) Nov 16 11:49:03.749001 localhost kernel: QLogic NetXtreme II iSCSI Driver bnx2i v2.7.10.1 (Jul 16, 2014) Nov 16 11:49:03.749056 localhost kernel: iscsi: registered transport (bnx2i) Nov 16 11:49:03.771645 localhost kernel: iscsi: registered transport (be2iscsi) Nov 16 11:49:03.771696 localhost kernel: In beiscsi_module_init, tt=00000000d9bc477b Nov 16 11:49:03.978127 localhost systemd[1]: Started dracut cmdline hook. Nov 16 11:49:03.990378 localhost systemd[1]: Starting dracut pre-udev hook... Nov 16 11:49:04.053969 localhost systemd-journald[374]: Missed 2 kernel messages Nov 16 11:49:04.053969 localhost kernel: device-mapper: uevent: version 1.0.3 Nov 16 11:49:04.054063 localhost kernel: device-mapper: ioctl: 4.43.0-ioctl (2020-10-01) initialised: dm-devel@redhat.com Nov 16 11:49:04.121116 localhost systemd[1]: Started dracut pre-udev hook. Nov 16 11:49:04.132074 localhost systemd[1]: Starting udev Kernel Device Manager... Nov 16 11:49:04.219351 localhost systemd[1]: Started udev Kernel Device Manager. Nov 16 11:49:04.232912 localhost systemd[1]: Starting dracut pre-trigger hook... Nov 16 11:49:04.252728 localhost dracut-pre-trigger[593]: rd.md=0: removing MD RAID activation Nov 16 11:49:04.460812 localhost systemd[1]: Started dracut pre-trigger hook. Nov 16 11:49:04.473447 localhost systemd[1]: Starting udev Coldplug all Devices... Nov 16 11:49:04.560647 localhost systemd[1]: Mounting Kernel Configuration File System... Nov 16 11:49:04.570800 localhost systemd[1]: Mounted Kernel Configuration File System. Nov 16 11:49:04.660800 localhost systemd[1]: Started udev Coldplug all Devices. Nov 16 11:49:04.674699 localhost systemd[1]: Starting udev Wait for Complete Device Initialization... Nov 16 11:49:04.702256 localhost systemd-journald[374]: Missed 11 kernel messages Nov 16 11:49:04.702256 localhost kernel: hv_vmbus: Vmbus version:4.0 Nov 16 11:49:04.709228 localhost kernel: libata version 3.00 loaded. Nov 16 11:49:04.717652 localhost systemd-udevd[651]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:49:04.728939 localhost kernel: ata_piix 0000:00:07.1: version 2.13 Nov 16 11:49:04.729158 localhost systemd-journald[374]: Missed 1 kernel messages Nov 16 11:49:04.729158 localhost kernel: ata_piix 0000:00:07.1: Hyper-V Virtual Machine detected, ATA device ignore set Nov 16 11:49:04.748221 localhost kernel: hv_vmbus: registering driver hyperv_keyboard Nov 16 11:49:04.748256 localhost kernel: scsi host0: ata_piix Nov 16 11:49:04.755731 localhost kernel: scsi host1: ata_piix Nov 16 11:49:04.756024 localhost kernel: ata1: PATA max UDMA/33 cmd 0x1f0 ctl 0x3f6 bmdma 0xffa0 irq 14 Nov 16 11:49:04.756057 localhost kernel: hv_vmbus: registering driver hid_hyperv Nov 16 11:49:04.764231 localhost kernel: ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0xffa8 irq 15 Nov 16 11:49:04.779237 localhost kernel: input: Microsoft Vmbus HID-compliant Mouse as /devices/0006:045E:0621.0001/input/input4 Nov 16 11:49:04.800652 localhost kernel: hid-generic 0006:045E:0621.0001: input: HID v0.01 Mouse [Microsoft Vmbus HID-compliant Mouse] on Nov 16 11:49:04.800960 localhost kernel: hv_vmbus: registering driver hv_storvsc Nov 16 11:49:04.808238 localhost kernel: hv_vmbus: registering driver hv_netvsc Nov 16 11:49:04.808276 localhost kernel: scsi host2: storvsc_host_t Nov 16 11:49:04.808334 localhost kernel: scsi host3: storvsc_host_t Nov 16 11:49:04.809228 localhost kernel: scsi 3:0:1:0: Direct-Access Msft Virtual Disk 1.0 PQ: 0 ANSI: 5 Nov 16 11:49:04.817208 localhost kernel: scsi host4: storvsc_host_t Nov 16 11:49:04.817287 localhost kernel: scsi host5: storvsc_host_t Nov 16 11:49:04.823610 localhost kernel: scsi 2:0:0:0: Direct-Access Msft Virtual Disk 1.0 PQ: 0 ANSI: 5 Nov 16 11:49:04.854239 localhost kernel: scsi 3:0:1:0: Attached scsi generic sg0 type 0 Nov 16 11:49:04.854500 localhost kernel: scsi 2:0:0:0: Attached scsi generic sg1 type 0 Nov 16 11:49:04.862272 localhost kernel: sd 3:0:1:0: [sda] 67108864 512-byte logical blocks: (34.4 GB/32.0 GiB) Nov 16 11:49:04.862552 localhost kernel: sd 3:0:1:0: [sda] 4096-byte physical blocks Nov 16 11:49:04.862779 localhost kernel: sd 2:0:0:0: [sdb] 536870912 512-byte logical blocks: (275 GB/256 GiB) Nov 16 11:49:04.862971 localhost kernel: sd 2:0:0:0: [sdb] 4096-byte physical blocks Nov 16 11:49:04.863154 localhost kernel: sd 3:0:1:0: [sda] Write Protect is off Nov 16 11:49:04.863433 localhost kernel: sd 3:0:1:0: [sda] Mode Sense: 0f 00 10 00 Nov 16 11:49:04.863618 localhost kernel: sd 2:0:0:0: [sdb] Write Protect is off Nov 16 11:49:04.863798 localhost kernel: sd 2:0:0:0: [sdb] Mode Sense: 0f 00 10 00 Nov 16 11:49:04.863984 localhost kernel: sd 3:0:1:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA Nov 16 11:49:04.864166 localhost kernel: sd 2:0:0:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA Nov 16 11:49:04.880353 localhost kernel: sda: sda1 Nov 16 11:49:04.881233 localhost kernel: sd 3:0:1:0: [sda] Attached SCSI disk Nov 16 11:49:04.906296 localhost systemd-udevd[645]: Using default interface naming scheme 'rhel-8.0'. Nov 16 11:49:04.921341 localhost systemd-journald[374]: Missed 1 kernel messages Nov 16 11:49:04.921341 localhost kernel: ata1.01: host indicates ignore ATA devices, ignored Nov 16 11:49:04.924233 localhost kernel: ata1.00: host indicates ignore ATA devices, ignored Nov 16 11:49:04.948254 localhost kernel: GPT:Primary header thinks Alt. header is not at the end of the disk. Nov 16 11:49:04.948313 localhost kernel: GPT:33554431 != 536870911 Nov 16 11:49:04.948356 localhost kernel: GPT:Alternate GPT header not at the end of the disk. Nov 16 11:49:04.948370 localhost kernel: GPT:33554431 != 536870911 Nov 16 11:49:04.948381 localhost kernel: GPT: Use GNU Parted to correct GPT errors. Nov 16 11:49:04.948417 localhost kernel: sdb: sdb1 sdb2 sdb3 sdb4 Nov 16 11:49:04.949246 localhost kernel: sd 2:0:0:0: [sdb] Attached SCSI disk Nov 16 11:49:05.233480 localhost kernel: ata2.01: NODEV after polling detection Nov 16 11:49:05.419793 localhost kernel: ata2.00: ATAPI: Virtual CD, , max MWDMA2 Nov 16 11:49:05.433745 localhost kernel: scsi 1:0:0:0: CD-ROM Msft Virtual CD/ROM 1.0 PQ: 0 ANSI: 5 Nov 16 11:49:05.444301 localhost kernel: scsi 1:0:0:0: Attached scsi generic sg2 type 5 Nov 16 11:49:05.469797 localhost kernel: sr 1:0:0:0: [sr0] scsi3-mmc drive: 0x/0x tray Nov 16 11:49:05.470054 localhost kernel: cdrom: Uniform CD-ROM driver Revision: 3.20 Nov 16 11:49:05.477277 localhost kernel: sr 1:0:0:0: Attached scsi CD-ROM sr0 Nov 16 11:49:05.633765 localhost systemd-udevd[645]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:49:05.705246 localhost systemd-journald[374]: Missed 1 kernel messages Nov 16 11:49:05.705246 localhost kernel: psmouse serio1: trackpoint: IBM TrackPoint firmware: 0x01, buttons: 3/3 Nov 16 11:49:05.716291 localhost kernel: input: TPPS/2 IBM TrackPoint as /devices/platform/i8042/serio1/input/input3 Nov 16 11:49:05.736314 localhost kernel: input: AT Translated Set 2 keyboard as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A03:00/device:07/VMBUS:01/d34b2567-b9b6-42b9-8778-0a4ec0b955bf/serio2/input/input5 Nov 16 11:49:05.936199 localhost systemd[1]: Found device Virtual_Disk boot. Nov 16 11:49:05.948208 localhost systemd[1]: Starting Generate New UUID For Boot Disk GPT... Nov 16 11:49:06.000638 localhost systemd[1]: Started udev Wait for Complete Device Initialization. Nov 16 11:49:06.031057 localhost systemd[1]: Starting Device-Mapper Multipath Device Controller... Nov 16 11:49:06.052416 localhost systemd[1]: Started Device-Mapper Multipath Device Controller. Nov 16 11:49:06.066899 localhost systemd[1]: Starting Open-iSCSI... Nov 16 11:49:06.079164 localhost coreos-gpt-setup[725]: Randomizing disk GUID Nov 16 11:49:06.101776 localhost multipathd[736]: --------start up-------- Nov 16 11:49:06.101776 localhost multipathd[736]: read /etc/multipath.conf Nov 16 11:49:06.147111 localhost systemd-journald[374]: Missed 7 kernel messages Nov 16 11:49:06.147111 localhost kernel: GPT:Primary header thinks Alt. header is not at the end of the disk. Nov 16 11:49:06.147195 localhost kernel: GPT:33554431 != 536870911 Nov 16 11:49:06.147254 localhost systemd-journald[374]: Missed 1 kernel messages Nov 16 11:49:06.147254 localhost kernel: GPT:Alternate GPT header not at the end of the disk. Nov 16 11:49:06.147279 localhost kernel: GPT:33554431 != 536870911 Nov 16 11:49:06.147290 localhost kernel: GPT: Use GNU Parted to correct GPT errors. Nov 16 11:49:06.147301 localhost kernel: sdb: sdb1 sdb2 sdb3 sdb4 Nov 16 11:49:06.147319 localhost multipathd[736]: /etc/multipath.conf does not exist, blacklisting all devices. Nov 16 11:49:06.147319 localhost multipathd[736]: You can run "/sbin/mpathconf --enable" to create Nov 16 11:49:06.147319 localhost multipathd[736]: /etc/multipath.conf. See man mpathconf(8) for more details Nov 16 11:49:06.147319 localhost multipathd[736]: path checkers start up Nov 16 11:49:06.147319 localhost multipathd[736]: /etc/multipath.conf does not exist, blacklisting all devices. Nov 16 11:49:06.147319 localhost multipathd[736]: You can run "/sbin/mpathconf --enable" to create Nov 16 11:49:06.147319 localhost multipathd[736]: /etc/multipath.conf. See man mpathconf(8) for more details Nov 16 11:49:06.213558 localhost iscsid[739]: iscsid: can't open InitiatorName configuration file /etc/iscsi/initiatorname.iscsi Nov 16 11:49:06.213558 localhost iscsid[739]: iscsid: Warning: InitiatorName file /etc/iscsi/initiatorname.iscsi does not exist or does not contain a properly formatted InitiatorName. If using software iscsi (iscsi_tcp or ib_iser) or partial offload (bnx2i or cxgbi iscsi), you may not be able to log into or discover targets. Please create a file /etc/iscsi/initiatorname.iscsi that contains a sting with the format: InitiatorName=iqn.yyyy-mm.[:identifier]. Nov 16 11:49:06.213558 localhost iscsid[739]: Example: InitiatorName=iqn.2001-04.com.redhat:fc6. Nov 16 11:49:06.213558 localhost iscsid[739]: If using hardware iscsi like qla4xxx this message can be ignored. Nov 16 11:49:06.213558 localhost iscsid[739]: iscsid: can't open InitiatorAlias configuration file /etc/iscsi/initiatorname.iscsi Nov 16 11:49:06.135161 localhost systemd[1]: Started Open-iSCSI. Nov 16 11:49:07.214886 localhost coreos-gpt-setup[725]: The operation has completed successfully. Nov 16 11:49:07.223827 localhost systemd-journald[374]: Missed 14 kernel messages Nov 16 11:49:07.223827 localhost kernel: sdb: sdb1 sdb2 sdb3 sdb4 Nov 16 11:49:07.317186 localhost systemd[1]: Started Generate New UUID For Boot Disk GPT. Nov 16 11:49:07.378915 localhost systemd[1]: Starting Ignition OSTree: Regenerate Filesystem UUID (boot)... Nov 16 11:49:07.395191 localhost systemd[1]: Reached target Local File Systems (Pre). Nov 16 11:49:07.408048 localhost systemd[1]: Reached target Local File Systems. Nov 16 11:49:07.420343 localhost systemd[1]: Starting Create Volatile Files and Directories... Nov 16 11:49:07.432393 localhost ignition-ostree-firstboot-uuid[768]: e2fsck 1.45.6 (20-Mar-2020) Nov 16 11:49:07.456481 localhost ignition-ostree-firstboot-uuid[768]: Pass 1: Checking inodes, blocks, and sizes Nov 16 11:49:07.467138 localhost ignition-ostree-firstboot-uuid[768]: Pass 2: Checking directory structure Nov 16 11:49:07.476356 localhost ignition-ostree-firstboot-uuid[768]: Pass 3: Checking directory connectivity Nov 16 11:49:07.486964 localhost ignition-ostree-firstboot-uuid[768]: Pass 4: Checking reference counts Nov 16 11:49:07.486964 localhost ignition-ostree-firstboot-uuid[768]: Pass 5: Checking group summary information Nov 16 11:49:07.490392 localhost systemd[1]: Started Create Volatile Files and Directories. Nov 16 11:49:07.519849 localhost systemd[1]: Reached target System Initialization. Nov 16 11:49:07.530905 localhost systemd[1]: Reached target Basic System. Nov 16 11:49:07.551060 localhost ignition-ostree-firstboot-uuid[768]: boot: 321/98304 files (0.3% non-contiguous), 126961/393216 blocks Nov 16 11:49:07.748883 localhost ignition-ostree-firstboot-uuid[768]: tune2fs 1.45.6 (20-Mar-2020) Nov 16 11:49:07.816115 localhost ignition-ostree-firstboot-uuid[768]: Regenerated UUID for /dev/disk/by-label/boot Nov 16 11:49:07.816972 localhost systemd[1]: Started Ignition OSTree: Regenerate Filesystem UUID (boot). Nov 16 11:49:07.841998 localhost systemd[1]: Starting CoreOS Ignition User Config Setup... Nov 16 11:49:07.942323 localhost systemd-journald[374]: Missed 20 kernel messages Nov 16 11:49:07.942323 localhost kernel: EXT4-fs (sdb3): mounted filesystem with ordered data mode. Opts: (null) Nov 16 11:49:07.942401 localhost coreos-ignition-setup-user[791]: File /mnt/boot_partition/ignition/config.ign does not exist.. Skipping copy Nov 16 11:49:07.960013 localhost systemd[1]: Started CoreOS Ignition User Config Setup. Nov 16 11:49:07.971746 localhost systemd[1]: Starting Ignition (fetch-offline)... Nov 16 11:49:08.001600 localhost ignition[804]: Ignition 2.13.0 Nov 16 11:49:08.001613 localhost ignition[804]: Stage: fetch-offline Nov 16 11:49:08.001696 localhost ignition[804]: reading system config file "/usr/lib/ignition/base.d/00-core.ign" Nov 16 11:49:08.001716 localhost ignition[804]: parsing config with SHA512: ff6a5153be363997e4d5d3ea8cc4048373a457c48c4a5b134a08a30aacd167c1e0f099f0bdf1e24c99ad180628cd02b767b863b5fe3a8fce3fe1886847eb8e2e Nov 16 11:49:08.003290 localhost ignition[804]: no config dir at "/usr/lib/ignition/base.platform.d/azure" Nov 16 11:49:08.003423 localhost ignition[804]: parsed url from cmdline: "" Nov 16 11:49:08.003427 localhost ignition[804]: no config URL provided Nov 16 11:49:08.003434 localhost ignition[804]: reading system config file "/usr/lib/ignition/user.ign" Nov 16 11:49:08.003444 localhost ignition[804]: no config at "/usr/lib/ignition/user.ign" Nov 16 11:49:08.106154 localhost ignition[804]: opening config device: "/dev/sr0" Nov 16 11:49:08.116157 localhost ignition[804]: getting drive status for "/dev/sr0" Nov 16 11:49:08.117289 localhost ignition[804]: drive status: OK Nov 16 11:49:08.118404 localhost ignition[804]: mounting config device Nov 16 11:49:08.118415 localhost ignition[804]: op(1): [started] mounting "/dev/sr0" at "/tmp/ignition-azure2173572060" Nov 16 11:49:08.208854 localhost systemd-journald[374]: Missed 12 kernel messages Nov 16 11:49:08.208854 localhost kernel: UDF-fs: INFO Mounting volume 'UDF Volume', timestamp 2022/11/17 00:00 (1000) Nov 16 11:49:08.210036 localhost ignition[804]: op(1): [finished] mounting "/dev/sr0" at "/tmp/ignition-azure2173572060" Nov 16 11:49:08.210048 localhost ignition[804]: checking for config drive Nov 16 11:49:08.217131 localhost systemd[1]: tmp-ignition\x2dazure2173572060.mount: Succeeded. Nov 16 11:49:08.213708 localhost ignition[804]: reading config Nov 16 11:49:08.216792 localhost ignition[804]: op(2): [started] unmounting "/dev/sr0" at "/tmp/ignition-azure2173572060" Nov 16 11:49:08.250064 localhost ignition[804]: op(2): [finished] unmounting "/dev/sr0" at "/tmp/ignition-azure2173572060" Nov 16 11:49:08.250114 localhost ignition[804]: parsing config with SHA512: a436d0050df027d17e950f5b1e8d5c15e356b1548f0a2e9ed7675c0053e3275bd661edce42cbc03717bb5ca72f6afd3b8aa6bf0c53a639a2125df5b4817d1fe4 Nov 16 11:49:08.262145 localhost ignition[804]: Adding "root-ca" to list of CAs Nov 16 11:49:08.262174 localhost ignition[804]: failed to fetch config: resource requires networking Nov 16 11:49:08.264268 localhost systemd[1]: Started Ignition (fetch-offline). Nov 16 11:49:08.262298 localhost ignition[804]: Ignition finished successfully Nov 16 11:49:08.294264 localhost systemd[1]: Starting CoreOS Enable Network... Nov 16 11:49:08.505447 localhost systemd[1]: Started CoreOS Enable Network. Nov 16 11:49:08.518108 localhost systemd[1]: Starting Copy CoreOS Firstboot Networking Config... Nov 16 11:49:08.529832 localhost systemd[1]: Starting Ignition (fetch)... Nov 16 11:49:08.552276 localhost systemd-journald[374]: Missed 12 kernel messages Nov 16 11:49:08.552276 localhost kernel: EXT4-fs (sdb3): mounted filesystem with ordered data mode. Opts: (null) Nov 16 11:49:08.554703 localhost coreos-copy-firstboot-network[834]: info: no files to copy from /mnt/boot_partition/coreos-firstboot-network; skipping Nov 16 11:49:08.561611 localhost ignition[838]: Ignition 2.13.0 Nov 16 11:49:08.561620 localhost ignition[838]: Stage: fetch Nov 16 11:49:08.561887 localhost ignition[838]: reading system config file "/usr/lib/ignition/base.d/00-core.ign" Nov 16 11:49:08.561914 localhost ignition[838]: parsing config with SHA512: ff6a5153be363997e4d5d3ea8cc4048373a457c48c4a5b134a08a30aacd167c1e0f099f0bdf1e24c99ad180628cd02b767b863b5fe3a8fce3fe1886847eb8e2e Nov 16 11:49:08.563280 localhost ignition[838]: no config dir at "/usr/lib/ignition/base.platform.d/azure" Nov 16 11:49:08.563422 localhost ignition[838]: parsed url from cmdline: "" Nov 16 11:49:08.563427 localhost ignition[838]: no config URL provided Nov 16 11:49:08.563434 localhost ignition[838]: reading system config file "/usr/lib/ignition/user.ign" Nov 16 11:49:08.563444 localhost ignition[838]: no config at "/usr/lib/ignition/user.ign" Nov 16 11:49:08.630357 localhost systemd[1]: Started Copy CoreOS Firstboot Networking Config. Nov 16 11:49:08.644405 localhost systemd[1]: Starting dracut initqueue hook... Nov 16 11:49:08.684291 localhost ignition[838]: opening config device: "/dev/sr0" Nov 16 11:49:08.694889 localhost ignition[838]: getting drive status for "/dev/sr0" Nov 16 11:49:08.696137 localhost ignition[838]: drive status: OK Nov 16 11:49:08.697083 localhost ignition[838]: mounting config device Nov 16 11:49:08.697134 localhost ignition[838]: op(1): [started] mounting "/dev/sr0" at "/tmp/ignition-azure746648515" Nov 16 11:49:08.762552 localhost systemd-journald[374]: Missed 12 kernel messages Nov 16 11:49:08.762552 localhost kernel: UDF-fs: INFO Mounting volume 'UDF Volume', timestamp 2022/11/17 00:00 (1000) Nov 16 11:49:08.763357 localhost ignition[838]: op(1): [finished] mounting "/dev/sr0" at "/tmp/ignition-azure746648515" Nov 16 11:49:08.763368 localhost ignition[838]: checking for config drive Nov 16 11:49:08.770059 localhost systemd[1]: tmp-ignition\x2dazure746648515.mount: Succeeded. Nov 16 11:49:08.766501 localhost ignition[838]: reading config Nov 16 11:49:08.769730 localhost ignition[838]: op(2): [started] unmounting "/dev/sr0" at "/tmp/ignition-azure746648515" Nov 16 11:49:08.803871 localhost ignition[838]: op(2): [finished] unmounting "/dev/sr0" at "/tmp/ignition-azure746648515" Nov 16 11:49:08.805152 localhost ignition[838]: parsing config with SHA512: a436d0050df027d17e950f5b1e8d5c15e356b1548f0a2e9ed7675c0053e3275bd661edce42cbc03717bb5ca72f6afd3b8aa6bf0c53a639a2125df5b4817d1fe4 Nov 16 11:49:08.820180 localhost ignition[838]: Adding "root-ca" to list of CAs Nov 16 11:49:08.820326 localhost ignition[838]: GET https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:22623/config/worker: attempt #1 Nov 16 11:49:08.840886 localhost ignition[838]: GET error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:22623/config/worker": dial tcp: lookup api-int.ropatil1611az.qe.azure.devcluster.openshift.com on [::1]:53: read udp [::1]:41764->[::1]:53: read: connection refused Nov 16 11:49:09.037287 localhost NetworkManager[866]: [1668599349.0372] NetworkManager (version 1.30.0-16.el8_4) is starting... (for the first time) Nov 16 11:49:09.037306 localhost NetworkManager[866]: [1668599349.0372] Read config: /etc/NetworkManager/NetworkManager.conf Nov 16 11:49:09.064899 localhost systemd-journald[374]: Missed 9 kernel messages Nov 16 11:49:09.064899 localhost kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 11:49:09.041345 localhost ignition[838]: GET https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:22623/config/worker: attempt #2 Nov 16 11:49:09.087400 localhost systemd-journald[374]: Missed 1 kernel messages Nov 16 11:49:09.087400 localhost kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 11:49:09.038964 localhost NetworkManager[866]: [1668599349.0389] auth[0x557c599cbea0]: create auth-manager: D-Bus connection not available. Polkit is disabled and only root will be authorized. Nov 16 11:49:09.110392 localhost systemd-journald[374]: Missed 1 kernel messages Nov 16 11:49:09.110392 localhost kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 11:49:09.042168 localhost ignition[838]: GET error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:22623/config/worker": dial tcp: lookup api-int.ropatil1611az.qe.azure.devcluster.openshift.com on [::1]:53: read udp [::1]:49196->[::1]:53: read: connection refused Nov 16 11:49:09.040124 localhost NetworkManager[866]: [1668599349.0400] manager[0x557c599fa020]: monitoring kernel firmware directory '/lib/firmware'. Nov 16 11:49:09.040288 localhost NetworkManager[866]: [1668599349.0402] hostname: hostname: hostnamed not used as proxy creation failed with: Could not connect: No such file or directory Nov 16 11:49:09.040459 localhost NetworkManager[866]: [1668599349.0404] dns-mgr[0x557c599e91d0]: init: dns=default,systemd-resolved rc-manager=symlink Nov 16 11:49:09.054960 localhost NetworkManager[866]: [1668599349.0549] Loaded device plugin: NMTeamFactory (/usr/lib64/NetworkManager/1.30.0-16.el8_4/libnm-device-plugin-team.so) Nov 16 11:49:09.054974 localhost NetworkManager[866]: [1668599349.0549] manager: rfkill: Wi-Fi enabled by radio killswitch; enabled by state file Nov 16 11:49:09.054982 localhost NetworkManager[866]: [1668599349.0549] manager: rfkill: WWAN enabled by radio killswitch; enabled by state file Nov 16 11:49:09.054989 localhost NetworkManager[866]: [1668599349.0549] manager: Networking is enabled by state file Nov 16 11:49:09.055003 localhost NetworkManager[866]: [1668599349.0549] dhcp-init: Using DHCP client 'internal' Nov 16 11:49:09.055331 localhost NetworkManager[866]: [1668599349.0553] ifcfg-rh: dbus: don't use D-Bus for com.redhat.ifcfgrh1 service Nov 16 11:49:09.055342 localhost NetworkManager[866]: [1668599349.0553] settings: Loaded settings plugin: ifcfg-rh ("/usr/lib64/NetworkManager/1.30.0-16.el8_4/libnm-settings-plugin-ifcfg-rh.so") Nov 16 11:49:09.055359 localhost NetworkManager[866]: [1668599349.0553] settings: Loaded settings plugin: keyfile (internal) Nov 16 11:49:09.056502 localhost NetworkManager[866]: [1668599349.0564] device (lo): carrier: link connected Nov 16 11:49:09.056531 localhost NetworkManager[866]: [1668599349.0565] manager: (lo): new Generic device (/org/freedesktop/NetworkManager/Devices/1) Nov 16 11:49:09.056952 localhost NetworkManager[866]: [1668599349.0569] manager: (eth0): new Ethernet device (/org/freedesktop/NetworkManager/Devices/2) Nov 16 11:49:09.057086 localhost NetworkManager[866]: [1668599349.0570] device (eth0): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external') Nov 16 11:49:09.118087 localhost NetworkManager[866]: [1668599349.1180] device (eth0): carrier: link connected Nov 16 11:49:09.118693 localhost NetworkManager[866]: [1668599349.1186] sleep-monitor-sd: failed to acquire D-Bus proxy: Could not connect: No such file or directory Nov 16 11:49:09.118830 localhost NetworkManager[866]: [1668599349.1188] device (eth0): state change: unavailable -> disconnected (reason 'none', sys-iface-state: 'managed') Nov 16 11:49:09.119108 localhost NetworkManager[866]: [1668599349.1190] policy: auto-activating connection 'Wired Connection' (9cfed2c1-311f-4178-b0ee-c9d8230e7568) Nov 16 11:49:09.119406 localhost NetworkManager[866]: [1668599349.1193] device (eth0): Activation: starting connection 'Wired Connection' (9cfed2c1-311f-4178-b0ee-c9d8230e7568) Nov 16 11:49:09.119428 localhost NetworkManager[866]: [1668599349.1194] device (eth0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed') Nov 16 11:49:09.119452 localhost NetworkManager[866]: [1668599349.1194] manager: NetworkManager state is now CONNECTING Nov 16 11:49:09.119483 localhost NetworkManager[866]: [1668599349.1194] device (eth0): state change: prepare -> config (reason 'none', sys-iface-state: 'managed') Nov 16 11:49:09.442373 localhost ignition[838]: GET https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:22623/config/worker: attempt #3 Nov 16 11:49:09.119665 localhost NetworkManager[866]: [1668599349.1196] device (eth0): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed') Nov 16 11:49:09.443107 localhost ignition[838]: GET error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:22623/config/worker": dial tcp: lookup api-int.ropatil1611az.qe.azure.devcluster.openshift.com on [::1]:53: read udp [::1]:35060->[::1]:53: read: connection refused Nov 16 11:49:09.119784 localhost NetworkManager[866]: [1668599349.1197] dhcp4 (eth0): activation: beginning transaction (timeout in 90 seconds) Nov 16 11:49:09.170608 localhost NetworkManager[866]: [1668599349.1705] dhcp4 (eth0): state changed unknown -> bound, address=10.0.1.4 Nov 16 11:49:09.198835 localhost NetworkManager[866]: [1668599349.1988] device (eth0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'managed') Nov 16 11:49:09.198896 localhost NetworkManager[866]: [1668599349.1988] device (eth0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'managed') Nov 16 11:49:09.198925 localhost NetworkManager[866]: [1668599349.1989] device (eth0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'managed') Nov 16 11:49:09.199162 localhost NetworkManager[866]: [1668599349.1991] manager: NetworkManager state is now CONNECTED_LOCAL Nov 16 11:49:09.199534 localhost NetworkManager[866]: [1668599349.1995] manager: NetworkManager state is now CONNECTED_SITE Nov 16 11:49:09.199563 localhost NetworkManager[866]: [1668599349.1995] policy: set 'Wired Connection' (eth0) as default for IPv4 routing and DNS Nov 16 11:49:09.200013 localhost NetworkManager[866]: [1668599349.1999] device (eth0): Activation: successful, device activated. Nov 16 11:49:09.200066 localhost NetworkManager[866]: [1668599349.2000] manager: NetworkManager state is now CONNECTED_GLOBAL Nov 16 11:49:09.200400 localhost NetworkManager[866]: [1668599349.2003] manager: startup complete Nov 16 11:49:09.200413 localhost NetworkManager[866]: [1668599349.2004] quitting now that startup is complete Nov 16 11:49:09.536369 localhost NetworkManager[866]: [1668599349.5363] dhcp4 (eth0): canceled DHCP transaction Nov 16 11:49:09.536397 localhost NetworkManager[866]: [1668599349.5363] dhcp4 (eth0): state changed bound -> done Nov 16 11:49:09.536425 localhost NetworkManager[866]: [1668599349.5364] device (eth0): DHCPv4: trying to acquire a new lease within 180 seconds Nov 16 11:49:09.536475 localhost NetworkManager[866]: [1668599349.5364] manager: NetworkManager state is now CONNECTED_SITE Nov 16 11:49:09.536582 localhost NetworkManager[866]: [1668599349.5365] exiting (success) Nov 16 11:49:09.570062 localhost systemd[1]: Started dracut initqueue hook. Nov 16 11:49:09.614061 localhost systemd[1]: Reached target Remote File Systems (Pre). Nov 16 11:49:09.647143 localhost systemd[1]: Reached target Remote File Systems. Nov 16 11:49:09.679836 localhost systemd[1]: Starting dracut pre-mount hook... Nov 16 11:49:09.712146 localhost systemd[1]: Started dracut pre-mount hook. Nov 16 11:49:10.243533 localhost ignition[838]: GET https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:22623/config/worker: attempt #4 Nov 16 11:49:10.244665 localhost ignition[838]: GET error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:22623/config/worker": dial tcp: lookup api-int.ropatil1611az.qe.azure.devcluster.openshift.com on [::1]:53: read udp [::1]:39299->[::1]:53: read: connection refused Nov 16 11:49:11.845793 localhost ignition[838]: GET https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:22623/config/worker: attempt #5 Nov 16 11:49:11.846697 localhost ignition[838]: GET error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:22623/config/worker": dial tcp: lookup api-int.ropatil1611az.qe.azure.devcluster.openshift.com on [::1]:53: read udp [::1]:49896->[::1]:53: read: connection refused Nov 16 11:49:15.049724 localhost ignition[838]: GET https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:22623/config/worker: attempt #6 Nov 16 11:49:15.114120 localhost ignition[838]: GET result: OK Nov 16 11:49:15.126640 localhost ignition[838]: fetched referenced config at https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:22623/config/worker with SHA512: 0acccd3fb0fe9bb0f337359320d6de93c9f7ab93c75d5f49edb1e6c2b7d967bcffef3a18601979297dc010dabc216e81d03c82221f10b600a99ecb02728a65a6 Nov 16 11:49:15.151864 localhost ignition[838]: Adding "root-ca" to list of CAs Nov 16 11:49:15.152540 localhost ignition[838]: Adding "root-ca" to list of CAs Nov 16 11:49:15.162892 localhost ignition[838]: fetch: fetch complete Nov 16 11:49:15.160777 localhost ignition[838]: fetched base config from "system" Nov 16 11:49:15.162899 localhost ignition[838]: fetch: fetch passed Nov 16 11:49:15.160806 localhost ignition[838]: fetched user config from "azure" Nov 16 11:49:15.163017 localhost ignition[838]: Ignition finished successfully Nov 16 11:49:15.160821 localhost ignition[838]: fetched base config from "system" Nov 16 11:49:15.160833 localhost ignition[838]: fetched user config from "azure" Nov 16 11:49:15.160850 localhost ignition[838]: fetched referenced user config from "/config/worker" Nov 16 11:49:15.165753 localhost systemd[1]: Started Ignition (fetch). Nov 16 11:49:15.193028 localhost systemd[1]: Starting Ignition OSTree: Detect Partition Transposition... Nov 16 11:49:15.213490 localhost systemd[1]: Starting Ignition (kargs)... Nov 16 11:49:15.246232 localhost ignition[894]: Ignition 2.13.0 Nov 16 11:49:15.235672 localhost systemd[1]: Starting RHCOS Check For Legacy LUKS Configuration... Nov 16 11:49:15.246241 localhost ignition[894]: Stage: kargs Nov 16 11:49:15.258560 localhost systemd[1]: Started Ignition OSTree: Detect Partition Transposition. Nov 16 11:49:15.246386 localhost ignition[894]: reading system config file "/usr/lib/ignition/base.d/00-core.ign" Nov 16 11:49:15.246408 localhost ignition[894]: parsing config with SHA512: ff6a5153be363997e4d5d3ea8cc4048373a457c48c4a5b134a08a30aacd167c1e0f099f0bdf1e24c99ad180628cd02b767b863b5fe3a8fce3fe1886847eb8e2e Nov 16 11:49:15.247782 localhost ignition[894]: no config dir at "/usr/lib/ignition/base.platform.d/azure" Nov 16 11:49:15.261562 localhost ignition[894]: Adding "root-ca" to list of CAs Nov 16 11:49:15.299333 localhost systemd[1]: Started RHCOS Check For Legacy LUKS Configuration. Nov 16 11:49:15.263249 localhost ignition[894]: kargs: kargs passed Nov 16 11:49:15.263309 localhost ignition[894]: Ignition finished successfully Nov 16 11:49:15.322724 localhost systemd[1]: Started Ignition (kargs). Nov 16 11:49:15.338480 localhost systemd[1]: Starting Check for FIPS mode... Nov 16 11:49:15.531149 localhost rhcos-fips[916]: Found /etc/ignition-machine-config-encapsulated.json in Ignition config Nov 16 11:49:15.542389 localhost rhcos-fips[916]: FIPS mode not requested Nov 16 11:49:15.563084 localhost systemd[1]: Started Check for FIPS mode. Nov 16 11:49:15.574340 localhost systemd[1]: Starting Afterburn (Check In - from the initramfs)... Nov 16 11:49:15.586598 localhost systemd[1]: Starting Ignition (disks)... Nov 16 11:49:15.616258 localhost ignition[940]: Ignition 2.13.0 Nov 16 11:49:15.616282 localhost ignition[940]: Stage: disks Nov 16 11:49:15.616496 localhost ignition[940]: reading system config file "/usr/lib/ignition/base.d/00-core.ign" Nov 16 11:49:15.634354 localhost systemd[1]: Started Ignition (disks). Nov 16 11:49:15.616520 localhost ignition[940]: parsing config with SHA512: ff6a5153be363997e4d5d3ea8cc4048373a457c48c4a5b134a08a30aacd167c1e0f099f0bdf1e24c99ad180628cd02b767b863b5fe3a8fce3fe1886847eb8e2e Nov 16 11:49:15.617861 localhost ignition[940]: no config dir at "/usr/lib/ignition/base.platform.d/azure" Nov 16 11:49:15.631131 localhost ignition[940]: Adding "root-ca" to list of CAs Nov 16 11:49:15.632655 localhost ignition[940]: disks: disks passed Nov 16 11:49:15.662810 localhost systemd[1]: Starting Ignition OSTree: Regenerate Filesystem UUID (root)... Nov 16 11:49:15.632714 localhost ignition[940]: Ignition finished successfully Nov 16 11:49:15.685786 localhost systemd[1]: Reached target Initrd Root Device. Nov 16 11:49:16.367806 localhost ignition-ostree-firstboot-uuid[953]: Clearing log and setting UUID Nov 16 11:49:16.367806 localhost ignition-ostree-firstboot-uuid[953]: writing all SBs Nov 16 11:49:16.367806 localhost ignition-ostree-firstboot-uuid[953]: new UUID = 566864e3-acbe-4961-8488-233c4e33ce96 Nov 16 11:49:16.436097 localhost ignition-ostree-firstboot-uuid[953]: Regenerated UUID for /dev/disk/by-label/root Nov 16 11:49:16.436994 localhost systemd[1]: Started Ignition OSTree: Regenerate Filesystem UUID (root). Nov 16 11:49:16.462860 localhost systemd[1]: Starting Ignition OSTree: Mount (firstboot) /sysroot... Nov 16 11:49:16.471137 localhost ignition-ostree-mount-sysroot[960]: Mounting /dev/disk/by-label/root (/dev/sdb4) to /sysroot Nov 16 11:49:16.599230 localhost systemd-journald[374]: Missed 102 kernel messages Nov 16 11:49:16.599230 localhost kernel: SGI XFS with ACLs, security attributes, quota, no debug enabled Nov 16 11:49:16.608288 localhost kernel: XFS (sdb4): Mounting V5 Filesystem Nov 16 11:49:16.737310 localhost kernel: XFS (sdb4): Ending clean mount Nov 16 11:49:16.761316 localhost kernel: XFS (sdb4): Quotacheck needed: Please wait. Nov 16 11:49:17.807324 localhost kernel: XFS (sdb4): Quotacheck: Done. Nov 16 11:49:17.809560 localhost systemd[1]: Started Ignition OSTree: Mount (firstboot) /sysroot. Nov 16 11:49:17.825238 localhost systemd[1]: Starting Ignition OSTree: Grow root filesystem... Nov 16 11:49:18.287698 localhost ignition-ostree-growfs[978]: CHANGED: partition=4 start=1050624 old: size=6850560 end=7901184 new: size=535820255 end=536870879 Nov 16 11:49:22.260709 localhost ignition-ostree-growfs[978]: meta-data=/dev/sdb4 isize=512 agcount=4, agsize=214080 blks Nov 16 11:49:22.260709 localhost ignition-ostree-growfs[978]: = sectsz=512 attr=2, projid32bit=1 Nov 16 11:49:22.260709 localhost ignition-ostree-growfs[978]: = crc=1 finobt=1, sparse=1, rmapbt=0 Nov 16 11:49:22.260709 localhost ignition-ostree-growfs[978]: = reflink=1 Nov 16 11:49:22.260709 localhost ignition-ostree-growfs[978]: data = bsize=4096 blocks=856320, imaxpct=25 Nov 16 11:49:22.260709 localhost ignition-ostree-growfs[978]: = sunit=0 swidth=0 blks Nov 16 11:49:22.260709 localhost ignition-ostree-growfs[978]: naming =version 2 bsize=4096 ascii-ci=0, ftype=1 Nov 16 11:49:22.260709 localhost ignition-ostree-growfs[978]: log =internal log bsize=4096 blocks=2560, version=2 Nov 16 11:49:22.260709 localhost ignition-ostree-growfs[978]: = sectsz=512 sunit=0 blks, lazy-count=1 Nov 16 11:49:22.260709 localhost ignition-ostree-growfs[978]: realtime =none extsz=4096 blocks=0, rtextents=0 Nov 16 11:49:22.260709 localhost ignition-ostree-growfs[978]: data blocks changed from 856320 to 66977531 Nov 16 11:49:22.263312 localhost systemd[1]: Started Ignition OSTree: Grow root filesystem. Nov 16 11:49:22.320296 localhost systemd[1]: Starting OSTree Prepare OS/... Nov 16 11:49:22.365913 localhost ostree-prepare-root[1034]: preparing sysroot at /sysroot Nov 16 11:49:22.366059 localhost ostree-prepare-root[1034]: Resolved OSTree target to: /sysroot/ostree/deploy/rhcos/deploy/3362fb42bacf40676ca4d24dfd9938b7f9067c0619fef495c91175429819918f.0 Nov 16 11:49:22.379121 localhost ostree-prepare-root[1034]: filesystem at /sysroot currently writable: 1 Nov 16 11:49:22.379126 localhost ostree-prepare-root[1034]: sysroot.readonly configuration value: 1 Nov 16 11:49:22.419583 localhost systemd[1]: sysroot-ostree-deploy-rhcos-var.mount: Succeeded. Nov 16 11:49:22.419592 localhost systemd[1]: sysroot-ostree-deploy-rhcos-deploy-3362fb42bacf40676ca4d24dfd9938b7f9067c0619fef495c91175429819918f.0-usr.mount: Succeeded. Nov 16 11:49:22.419598 localhost systemd[1]: sysroot-ostree-deploy-rhcos-deploy-3362fb42bacf40676ca4d24dfd9938b7f9067c0619fef495c91175429819918f.0-etc.mount: Succeeded. Nov 16 11:49:22.419603 localhost systemd[1]: sysroot-ostree-deploy-rhcos-deploy-3362fb42bacf40676ca4d24dfd9938b7f9067c0619fef495c91175429819918f.0.mount: Succeeded. Nov 16 11:49:22.420011 localhost systemd[1]: Started OSTree Prepare OS/. Nov 16 11:49:22.453355 localhost systemd[1]: Reached target Initrd Root File System. Nov 16 11:49:22.498550 localhost systemd[1]: Starting Mount OSTree /var... Nov 16 11:49:22.517766 localhost ignition-ostree-mount-var[1036]: Mounting /sysroot/sysroot/ostree/deploy/rhcos/var Nov 16 11:49:22.518276 localhost systemd[1]: Starting Afterburn Hostname... Nov 16 11:49:22.538049 localhost systemd[1]: Starting Ignition OSTree: Check Root Filesystem Size... Nov 16 11:49:22.559378 localhost systemd[1]: Started Mount OSTree /var. Nov 16 11:49:22.569014 localhost systemd[1]: Started Ignition OSTree: Check Root Filesystem Size. Nov 16 11:49:22.583464 localhost systemd[1]: Starting Ignition (mount)... Nov 16 11:49:22.616364 localhost ignition[1054]: INFO : Ignition 2.13.0 Nov 16 11:49:22.616364 localhost ignition[1054]: INFO : Stage: mount Nov 16 11:49:22.628058 localhost ignition[1054]: INFO : reading system config file "/usr/lib/ignition/base.d/00-core.ign" Nov 16 11:49:22.628058 localhost ignition[1054]: DEBUG : parsing config with SHA512: ff6a5153be363997e4d5d3ea8cc4048373a457c48c4a5b134a08a30aacd167c1e0f099f0bdf1e24c99ad180628cd02b767b863b5fe3a8fce3fe1886847eb8e2e Nov 16 11:49:22.628058 localhost ignition[1054]: INFO : no config dir at "/usr/lib/ignition/base.platform.d/azure" Nov 16 11:49:22.670158 localhost ignition[1054]: INFO : Adding "root-ca" to list of CAs Nov 16 11:49:22.670158 localhost ignition[1054]: INFO : mount: mount passed Nov 16 11:49:22.670158 localhost ignition[1054]: INFO : Ignition finished successfully Nov 16 11:49:22.636347 localhost systemd[1]: Started Ignition (mount). Nov 16 11:49:22.671022 localhost systemd[1]: Starting Populate OSTree /var... Nov 16 11:49:24.028208 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/home from (null) to system_u:object_r:home_root_t:s0 Nov 16 11:49:24.321582 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/roothome from (null) to system_u:object_r:admin_home_t:s0 Nov 16 11:49:24.321582 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/roothome/.bash_logout from (null) to system_u:object_r:admin_home_t:s0 Nov 16 11:49:24.321582 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/roothome/.bash_profile from (null) to system_u:object_r:admin_home_t:s0 Nov 16 11:49:24.321582 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/roothome/.bashrc from (null) to system_u:object_r:admin_home_t:s0 Nov 16 11:49:24.573528 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/opt from (null) to system_u:object_r:var_t:s0 Nov 16 11:49:24.822649 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/srv from (null) to system_u:object_r:var_t:s0 Nov 16 11:49:25.104630 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/usrlocal from (null) to system_u:object_r:var_t:s0 Nov 16 11:49:25.104630 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/usrlocal/bin from (null) to system_u:object_r:bin_t:s0 Nov 16 11:49:25.104630 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/usrlocal/etc from (null) to system_u:object_r:var_t:s0 Nov 16 11:49:25.104630 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/usrlocal/games from (null) to system_u:object_r:var_t:s0 Nov 16 11:49:25.104630 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/usrlocal/include from (null) to system_u:object_r:var_t:s0 Nov 16 11:49:25.104630 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/usrlocal/lib from (null) to system_u:object_r:var_t:s0 Nov 16 11:49:25.104630 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/usrlocal/man from (null) to system_u:object_r:var_t:s0 Nov 16 11:49:25.104630 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/usrlocal/sbin from (null) to system_u:object_r:var_t:s0 Nov 16 11:49:25.104630 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/usrlocal/share from (null) to system_u:object_r:var_t:s0 Nov 16 11:49:25.104630 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/usrlocal/src from (null) to system_u:object_r:var_t:s0 Nov 16 11:49:25.358034 localhost ignition-ostree-populate-var[1065]: Relabeled /sysroot//var/mnt from (null) to system_u:object_r:var_t:s0 Nov 16 11:49:25.607693 localhost systemd[1]: Started Populate OSTree /var. Nov 16 11:49:44.360028 localhost afterburn[939]: Nov 16 11:49:44.359 WARN Failed to get fabric address from DHCP: maximum number of retries (60) reached Nov 16 11:49:44.360028 localhost afterburn[939]: Nov 16 11:49:44.360 INFO using fallback address Nov 16 11:49:44.384355 localhost afterburn[939]: Nov 16 11:49:44.368 INFO Fetching http://168.63.129.16/?comp=versions: Attempt #1 Nov 16 11:49:44.384355 localhost afterburn[939]: Nov 16 11:49:44.371 INFO Fetch successful Nov 16 11:49:44.384355 localhost afterburn[939]: Nov 16 11:49:44.371 INFO Fetching http://168.63.129.16/machine/?comp=goalstate: Attempt #1 Nov 16 11:49:44.384355 localhost afterburn[939]: Nov 16 11:49:44.376 INFO Fetch successful Nov 16 11:49:44.384355 localhost afterburn[939]: Nov 16 11:49:44.384 INFO Posting http://168.63.129.16/machine/?comp=health: Attempt #1 Nov 16 11:49:44.637996 localhost systemd[1]: Started Afterburn (Check In - from the initramfs). Nov 16 11:49:51.306698 localhost afterburn[1044]: Nov 16 11:49:51.306 WARN Failed to get fabric address from DHCP: maximum number of retries (60) reached Nov 16 11:49:51.306698 localhost afterburn[1044]: Nov 16 11:49:51.306 INFO using fallback address Nov 16 11:49:51.331385 localhost afterburn[1044]: Nov 16 11:49:51.319 INFO Fetching http://168.63.129.16/?comp=versions: Attempt #1 Nov 16 11:49:51.331385 localhost afterburn[1044]: Nov 16 11:49:51.322 INFO Fetch successful Nov 16 11:49:51.331385 localhost afterburn[1044]: Nov 16 11:49:51.329 INFO Fetching http://169.254.169.254/metadata/instance/compute/name?api-version=2017-08-01&format=text: Attempt #1 Nov 16 11:49:51.368065 localhost afterburn[1044]: Nov 16 11:49:51.342 INFO Fetch successful Nov 16 11:49:51.368065 localhost afterburn[1044]: Nov 16 11:49:51.343 INFO wrote hostname ropatil1611az-gs6zj-worker-westus-j4ngg to /sysroot/etc/hostname Nov 16 11:49:51.398670 localhost systemd[1]: afterburn-hostname.service: Succeeded. Nov 16 11:49:51.399087 localhost systemd[1]: Started Afterburn Hostname. Nov 16 11:49:51.416278 localhost systemd[1]: Starting Ignition (files)... Nov 16 11:49:51.446875 localhost ignition[1109]: INFO : Ignition 2.13.0 Nov 16 11:49:51.446875 localhost ignition[1109]: INFO : Stage: files Nov 16 11:49:51.459313 localhost ignition[1109]: INFO : reading system config file "/usr/lib/ignition/base.d/00-core.ign" Nov 16 11:49:51.459313 localhost ignition[1109]: DEBUG : parsing config with SHA512: ff6a5153be363997e4d5d3ea8cc4048373a457c48c4a5b134a08a30aacd167c1e0f099f0bdf1e24c99ad180628cd02b767b863b5fe3a8fce3fe1886847eb8e2e Nov 16 11:49:51.459313 localhost ignition[1109]: INFO : no config dir at "/usr/lib/ignition/base.platform.d/azure" Nov 16 11:49:51.497529 localhost ignition[1109]: INFO : Adding "root-ca" to list of CAs Nov 16 11:49:52.003660 localhost ignition[1109]: INFO : files: ensureUsers: op(1): [started] creating or modifying user "core" Nov 16 11:49:52.003660 localhost ignition[1109]: DEBUG : files: ensureUsers: op(1): executing: "useradd" "--root" "/sysroot" "--create-home" "--password" "*" "--comment" "CoreOS Admin" "--groups" "adm,sudo,systemd-journal,wheel" "core" Nov 16 11:49:55.453956 localhost ignition[1109]: INFO : files: ensureUsers: op(1): [finished] creating or modifying user "core" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: ensureUsers: op(2): [started] adding ssh keys to user "core" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: ensureUsers: op(2): [finished] adding ssh keys to user "core" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(3): [started] writing file "/sysroot/etc/ignition-machine-config-encapsulated.json" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(3): [finished] writing file "/sysroot/etc/ignition-machine-config-encapsulated.json" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(4): [started] writing file "/sysroot/etc/mcs-machine-config-content.json" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(4): [finished] writing file "/sysroot/etc/mcs-machine-config-content.json" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(5): [started] writing file "/sysroot/etc/node-sizing-enabled.env" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(5): [finished] writing file "/sysroot/etc/node-sizing-enabled.env" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(6): [started] writing file "/sysroot/etc/sysctl.d/inotify.conf" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(6): [finished] writing file "/sysroot/etc/sysctl.d/inotify.conf" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(7): [started] writing file "/sysroot/etc/kubernetes/kubeconfig" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(7): [finished] writing file "/sysroot/etc/kubernetes/kubeconfig" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(8): [started] writing file "/sysroot/etc/containers/registries.conf" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(8): [finished] writing file "/sysroot/etc/containers/registries.conf" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(9): [started] writing file "/sysroot/etc/containers/storage.conf" Nov 16 11:49:55.471190 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(9): [finished] writing file "/sysroot/etc/containers/storage.conf" Nov 16 11:49:55.455456 localhost ignition[1109]: wrote ssh authorized keys file for user: core Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(a): [started] writing file "/sysroot/etc/mco/proxy.env" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(a): [finished] writing file "/sysroot/etc/mco/proxy.env" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(b): [started] writing file "/sysroot/etc/tmpfiles.d/cleanup-cni.conf" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(b): [finished] writing file "/sysroot/etc/tmpfiles.d/cleanup-cni.conf" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(c): [started] writing file "/sysroot/etc/modules-load.d/iptables.conf" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(c): [finished] writing file "/sysroot/etc/modules-load.d/iptables.conf" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(d): [started] writing file "/sysroot/etc/containers/policy.json" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(d): [finished] writing file "/sysroot/etc/containers/policy.json" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(e): [started] writing file "/sysroot/etc/sysctl.d/forward.conf" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(e): [finished] writing file "/sysroot/etc/sysctl.d/forward.conf" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(f): [started] writing file "/sysroot/etc/kubernetes/kubelet-ca.crt" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(f): [finished] writing file "/sysroot/etc/kubernetes/kubelet-ca.crt" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(10): [started] writing file "/sysroot/etc/kubernetes/kubelet.conf" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(10): [finished] writing file "/sysroot/etc/kubernetes/kubelet.conf" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(11): [started] writing file "/sysroot/etc/machine-config-daemon/node-annotations.json" Nov 16 11:49:55.704052 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(11): [finished] writing file "/sysroot/etc/machine-config-daemon/node-annotations.json" Nov 16 11:49:55.942560 localhost systemd-journald[374]: Missed 119 kernel messages Nov 16 11:49:55.942560 localhost kernel: EXT4-fs (sdb3): mounted filesystem with ordered data mode. Opts: (null) Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(12): [started] writing file "/sysroot/etc/kubernetes/ca.crt" Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(12): [finished] writing file "/sysroot/etc/kubernetes/ca.crt" Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(13): [started] writing file "/sysroot/etc/kubernetes/cloud.conf" Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(13): [finished] writing file "/sysroot/etc/kubernetes/cloud.conf" Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(14): [started] writing file "/sysroot/etc/NetworkManager/conf.d/sdn.conf" Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(14): [finished] writing file "/sysroot/etc/NetworkManager/conf.d/sdn.conf" Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(15): [started] writing file "/sysroot/var/usrlocal/sbin/dynamic-system-reserved-calc.sh" Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(15): [finished] writing file "/sysroot/var/usrlocal/sbin/dynamic-system-reserved-calc.sh" Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(16): [started] writing file "/sysroot/etc/systemd/system.conf.d/10-default-env-godebug.conf" Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(16): [finished] writing file "/sysroot/etc/systemd/system.conf.d/10-default-env-godebug.conf" Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(17): [started] writing file "/sysroot/var/usrlocal/bin/mco-hostname" Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(17): [finished] writing file "/sysroot/var/usrlocal/bin/mco-hostname" Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(18): [started] writing file "/sysroot/etc/crio/crio.conf.d/01-mc-seccompUseDefault" Nov 16 11:49:55.942631 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(18): [finished] writing file "/sysroot/etc/crio/crio.conf.d/01-mc-seccompUseDefault" Nov 16 11:49:55.726691 localhost systemd[1]: Started Ignition (files). Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(19): [started] writing file "/sysroot/etc/NetworkManager/dispatcher.d/99-vsphere-disable-tx-udp-tnl" Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(19): [finished] writing file "/sysroot/etc/NetworkManager/dispatcher.d/99-vsphere-disable-tx-udp-tnl" Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1a): [started] writing file "/sysroot/var/usrlocal/bin/configure-ovs.sh" Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1a): [finished] writing file "/sysroot/var/usrlocal/bin/configure-ovs.sh" Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1b): [started] writing file "/sysroot/etc/crio/crio.conf.d/00-default" Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1b): [finished] writing file "/sysroot/etc/crio/crio.conf.d/00-default" Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1c): [started] writing file "/sysroot/etc/audit/rules.d/mco-audit-quiet-containers.rules" Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1c): [finished] writing file "/sysroot/etc/audit/rules.d/mco-audit-quiet-containers.rules" Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1d): [started] writing file "/sysroot/etc/systemd/system.conf.d/kubelet-cgroups.conf" Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1d): [finished] writing file "/sysroot/etc/systemd/system.conf.d/kubelet-cgroups.conf" Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1e): [started] writing file "/sysroot/var/lib/kubelet/config.json" Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1e): [finished] writing file "/sysroot/var/lib/kubelet/config.json" Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1f): [started] writing file "/sysroot/etc/NetworkManager/conf.d/20-keyfiles.conf" Nov 16 11:49:56.154801 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1f): [finished] writing file "/sysroot/etc/NetworkManager/conf.d/20-keyfiles.conf" Nov 16 11:49:55.789362 localhost systemd[1]: Starting Reload Configuration from the Real Root... Nov 16 11:49:56.383580 localhost multipathd[736]: exit (signal) Nov 16 11:49:56.383580 localhost multipathd[736]: --------shut down------- Nov 16 11:49:56.397533 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(20): [started] writing file "/sysroot/etc/systemd/system/kubelet.service.d/20-logging.conf" Nov 16 11:49:56.397533 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(20): [finished] writing file "/sysroot/etc/systemd/system/kubelet.service.d/20-logging.conf" Nov 16 11:49:56.397533 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(21): [started] writing file "/sysroot/etc/kubernetes/static-pod-resources/configmaps/cloud-config/ca-bundle.pem" Nov 16 11:49:56.397533 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(21): [finished] writing file "/sysroot/etc/kubernetes/static-pod-resources/configmaps/cloud-config/ca-bundle.pem" Nov 16 11:49:56.397533 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(22): [started] writing file "/sysroot/etc/pki/ca-trust/source/anchors/openshift-config-user-ca-bundle.crt" Nov 16 11:49:56.397533 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(22): [finished] writing file "/sysroot/etc/pki/ca-trust/source/anchors/openshift-config-user-ca-bundle.crt" Nov 16 11:49:56.397533 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(23): [started] writing file "/sysroot/etc/kubernetes/kubelet-plugins/volume/exec/.dummy" Nov 16 11:49:56.397533 localhost ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(23): [finished] writing file "/sysroot/etc/kubernetes/kubelet-plugins/volume/exec/.dummy" Nov 16 11:49:56.397533 localhost ignition[1109]: INFO : files: op(24): [started] processing unit "crio.service" Nov 16 11:49:56.397533 localhost ignition[1109]: INFO : files: op(24): op(25): [started] writing systemd drop-in "10-mco-default-env.conf" at "/sysroot/etc/systemd/system/crio.service.d/10-mco-default-env.conf" Nov 16 11:49:56.397533 localhost ignition[1109]: INFO : files: op(24): op(25): [finished] writing systemd drop-in "10-mco-default-env.conf" at "/sysroot/etc/systemd/system/crio.service.d/10-mco-default-env.conf" Nov 16 11:49:56.397533 localhost ignition[1109]: INFO : files: op(24): op(26): [started] writing systemd drop-in "10-mco-profile-unix-socket.conf" at "/sysroot/etc/systemd/system/crio.service.d/10-mco-profile-unix-socket.conf" Nov 16 11:49:56.397533 localhost ignition[1109]: INFO : files: op(24): op(26): [finished] writing systemd drop-in "10-mco-profile-unix-socket.conf" at "/sysroot/etc/systemd/system/crio.service.d/10-mco-profile-unix-socket.conf" Nov 16 11:49:55.847149 localhost systemd[1]: Starting CoreOS Boot Edit... Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(24): op(27): [started] writing systemd drop-in "10-mco-default-madv.conf" at "/sysroot/etc/systemd/system/crio.service.d/10-mco-default-madv.conf" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(24): op(27): [finished] writing systemd drop-in "10-mco-default-madv.conf" at "/sysroot/etc/systemd/system/crio.service.d/10-mco-default-madv.conf" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(24): [finished] processing unit "crio.service" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(28): [started] processing unit "docker.socket" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(28): op(29): [started] writing systemd drop-in "mco-disabled.conf" at "/sysroot/etc/systemd/system/docker.socket.d/mco-disabled.conf" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(28): op(29): [finished] writing systemd drop-in "mco-disabled.conf" at "/sysroot/etc/systemd/system/docker.socket.d/mco-disabled.conf" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(28): [finished] processing unit "docker.socket" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(2a): [started] processing unit "kubelet-auto-node-size.service" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(2a): op(2b): [started] writing unit "kubelet-auto-node-size.service" at "/sysroot/etc/systemd/system/kubelet-auto-node-size.service" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(2a): op(2b): [finished] writing unit "kubelet-auto-node-size.service" at "/sysroot/etc/systemd/system/kubelet-auto-node-size.service" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(2a): [finished] processing unit "kubelet-auto-node-size.service" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(2c): [started] processing unit "kubelet.service" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(2c): op(2d): [started] writing systemd drop-in "10-mco-default-env.conf" at "/sysroot/etc/systemd/system/kubelet.service.d/10-mco-default-env.conf" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(2c): op(2d): [finished] writing systemd drop-in "10-mco-default-env.conf" at "/sysroot/etc/systemd/system/kubelet.service.d/10-mco-default-env.conf" Nov 16 11:49:56.623715 localhost ignition[1109]: INFO : files: op(2c): op(2e): [started] writing systemd drop-in "10-mco-default-madv.conf" at "/sysroot/etc/systemd/system/kubelet.service.d/10-mco-default-madv.conf" Nov 16 11:49:56.843796 localhost coreos-boot-edit[1137]: Injected kernel arguments into BLS: root=UUID=566864e3-acbe-4961-8488-233c4e33ce96 rw rootflags=prjquota Nov 16 11:49:55.888295 localhost systemd[1]: Reloading. Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(2c): op(2e): [finished] writing systemd drop-in "10-mco-default-madv.conf" at "/sysroot/etc/systemd/system/kubelet.service.d/10-mco-default-madv.conf" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(2c): op(2f): [started] writing unit "kubelet.service" at "/sysroot/etc/systemd/system/kubelet.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(2c): op(2f): [finished] writing unit "kubelet.service" at "/sysroot/etc/systemd/system/kubelet.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(2c): [finished] processing unit "kubelet.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(30): [started] processing unit "machine-config-daemon-firstboot.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(30): op(31): [started] writing unit "machine-config-daemon-firstboot.service" at "/sysroot/etc/systemd/system/machine-config-daemon-firstboot.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(30): op(31): [finished] writing unit "machine-config-daemon-firstboot.service" at "/sysroot/etc/systemd/system/machine-config-daemon-firstboot.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(30): [finished] processing unit "machine-config-daemon-firstboot.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(32): [started] processing unit "machine-config-daemon-pull.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(32): op(33): [started] writing unit "machine-config-daemon-pull.service" at "/sysroot/etc/systemd/system/machine-config-daemon-pull.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(32): op(33): [finished] writing unit "machine-config-daemon-pull.service" at "/sysroot/etc/systemd/system/machine-config-daemon-pull.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(32): [finished] processing unit "machine-config-daemon-pull.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(34): [started] processing unit "node-valid-hostname.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(34): op(35): [started] writing unit "node-valid-hostname.service" at "/sysroot/etc/systemd/system/node-valid-hostname.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(34): op(35): [finished] writing unit "node-valid-hostname.service" at "/sysroot/etc/systemd/system/node-valid-hostname.service" Nov 16 11:49:56.862031 localhost ignition[1109]: INFO : files: op(34): [finished] processing unit "node-valid-hostname.service" Nov 16 11:49:57.088767 localhost coreos-boot-edit[1137]: random.trust_cpu=on console=tty0 console=ttyS0,115200n8 $ignition_firstboot ostree=/ostree/boot.1/rhcos/dfe3636c0ecb6ffa92f58233688e78d3a8f6fec44b763bfd21d40bdea12ba3ff/0 ignition.platform.id=azure root=UUID=566864e3-acbe-4961-8488-233c4e33ce96 rw rootflags=prjquota boot=UUID=4a20a463-024a-4bb3-afd8-31decc06c38d Nov 16 11:49:56.132367 localhost systemd[1]: Started CoreOS Boot Edit. Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(36): [started] processing unit "nodeip-configuration.service" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(36): op(37): [started] writing unit "nodeip-configuration.service" at "/sysroot/etc/systemd/system/nodeip-configuration.service" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(36): op(37): [finished] writing unit "nodeip-configuration.service" at "/sysroot/etc/systemd/system/nodeip-configuration.service" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(36): [finished] processing unit "nodeip-configuration.service" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(38): [started] processing unit "openvswitch.service" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(38): [finished] processing unit "openvswitch.service" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(39): [started] processing unit "ovs-configuration.service" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(39): op(3a): [started] writing unit "ovs-configuration.service" at "/sysroot/etc/systemd/system/ovs-configuration.service" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(39): op(3a): [finished] writing unit "ovs-configuration.service" at "/sysroot/etc/systemd/system/ovs-configuration.service" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(39): [finished] processing unit "ovs-configuration.service" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(3b): [started] processing unit "ovs-vswitchd.service" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(3b): op(3c): [started] writing systemd drop-in "10-ovs-vswitchd-restart.conf" at "/sysroot/etc/systemd/system/ovs-vswitchd.service.d/10-ovs-vswitchd-restart.conf" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(3b): op(3c): [finished] writing systemd drop-in "10-ovs-vswitchd-restart.conf" at "/sysroot/etc/systemd/system/ovs-vswitchd.service.d/10-ovs-vswitchd-restart.conf" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(3b): [finished] processing unit "ovs-vswitchd.service" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(3d): [started] processing unit "ovsdb-server.service" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(3d): op(3e): [started] writing systemd drop-in "10-ovsdb-restart.conf" at "/sysroot/etc/systemd/system/ovsdb-server.service.d/10-ovsdb-restart.conf" Nov 16 11:49:57.128917 localhost ignition[1109]: INFO : files: op(3d): op(3e): [finished] writing systemd drop-in "10-ovsdb-restart.conf" at "/sysroot/etc/systemd/system/ovsdb-server.service.d/10-ovsdb-restart.conf" Nov 16 11:49:56.189419 localhost systemd[1]: Reached target Ignition Boot Disk Setup. Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(3d): [finished] processing unit "ovsdb-server.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(3f): [started] processing unit "pivot.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(3f): op(40): [started] writing systemd drop-in "10-mco-default-env.conf" at "/sysroot/etc/systemd/system/pivot.service.d/10-mco-default-env.conf" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(3f): op(40): [finished] writing systemd drop-in "10-mco-default-env.conf" at "/sysroot/etc/systemd/system/pivot.service.d/10-mco-default-env.conf" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(3f): [finished] processing unit "pivot.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(41): [started] processing unit "zincati.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(41): op(42): [started] writing systemd drop-in "mco-disabled.conf" at "/sysroot/etc/systemd/system/zincati.service.d/mco-disabled.conf" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(41): op(42): [finished] writing systemd drop-in "mco-disabled.conf" at "/sysroot/etc/systemd/system/zincati.service.d/mco-disabled.conf" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(41): [finished] processing unit "zincati.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(43): [started] setting preset to enabled for "kubelet.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(43): [finished] setting preset to enabled for "kubelet.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(44): [started] setting preset to enabled for "machine-config-daemon-pull.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(44): [finished] setting preset to enabled for "machine-config-daemon-pull.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(45): [started] setting preset to enabled for "kubelet-auto-node-size.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(45): [finished] setting preset to enabled for "kubelet-auto-node-size.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(46): [started] setting preset to enabled for "machine-config-daemon-firstboot.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(46): [finished] setting preset to enabled for "machine-config-daemon-firstboot.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(47): [started] setting preset to enabled for "node-valid-hostname.service" Nov 16 11:49:57.383099 localhost ignition[1109]: INFO : files: op(47): [finished] setting preset to enabled for "node-valid-hostname.service" Nov 16 11:49:56.247316 localhost systemd[1]: Reached target Ignition Complete. Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : files: op(48): [started] setting preset to disabled for "nodeip-configuration.service" Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : files: op(48): [finished] setting preset to disabled for "nodeip-configuration.service" Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : files: op(49): [started] setting preset to enabled for "openvswitch.service" Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : files: op(49): [finished] setting preset to enabled for "openvswitch.service" Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : files: op(4a): [started] setting preset to enabled for "ovs-configuration.service" Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : files: op(4a): [finished] setting preset to enabled for "ovs-configuration.service" Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : files: op(4b): [started] setting preset to enabled for "ovsdb-server.service" Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : files: op(4b): [finished] setting preset to enabled for "ovsdb-server.service" Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : files: createResultFile: createFiles: op(4c): [started] writing file "/sysroot/etc/.ignition-result.json" Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : files: createResultFile: createFiles: op(4c): [finished] writing file "/sysroot/etc/.ignition-result.json" Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : files: op(4d): [started] relabeling 64 patterns Nov 16 11:49:57.649829 localhost ignition[1109]: DEBUG : files: op(4d): executing: "setfiles" "-vF0" "-r" "/sysroot" "/sysroot/etc/selinux/targeted/contexts/files/file_contexts" "-f" "-" Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : files: op(4d): [finished] relabeling 64 patterns Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : files: files passed Nov 16 11:49:57.649829 localhost ignition[1109]: INFO : Ignition finished successfully Nov 16 11:49:57.826918 localhost dracut-pre-pivot[1224]: Nov 16 11:49:56 | /etc/multipath.conf does not exist, blacklisting all devices. Nov 16 11:49:57.826918 localhost dracut-pre-pivot[1224]: Nov 16 11:49:56 | You can run "/sbin/mpathconf --enable" to create Nov 16 11:49:57.826918 localhost dracut-pre-pivot[1224]: Nov 16 11:49:56 | /etc/multipath.conf. See man mpathconf(8) for more details Nov 16 11:49:56.297744 localhost systemd[1]: Stopping Device-Mapper Multipath Device Controller... Nov 16 11:49:57.907045 localhost iscsid[739]: iscsid: iscsid shutting down. Nov 16 11:49:56.359107 localhost systemd[1]: multipathd.service: Succeeded. Nov 16 11:49:57.921684 localhost ignition[1254]: INFO : Ignition 2.13.0 Nov 16 11:49:57.921684 localhost ignition[1254]: INFO : Stage: umount Nov 16 11:49:57.921684 localhost ignition[1254]: INFO : reading system config file "/usr/lib/ignition/base.d/00-core.ign" Nov 16 11:49:57.921684 localhost ignition[1254]: DEBUG : parsing config with SHA512: ff6a5153be363997e4d5d3ea8cc4048373a457c48c4a5b134a08a30aacd167c1e0f099f0bdf1e24c99ad180628cd02b767b863b5fe3a8fce3fe1886847eb8e2e Nov 16 11:49:57.921684 localhost ignition[1254]: INFO : no config dir at "/usr/lib/ignition/base.platform.d/azure" Nov 16 11:49:57.921684 localhost ignition[1254]: INFO : Adding "root-ca" to list of CAs Nov 16 11:49:57.921684 localhost ignition[1254]: INFO : umount: umount passed Nov 16 11:49:57.921684 localhost ignition[1254]: INFO : Ignition finished successfully Nov 16 11:49:56.359469 localhost systemd[1]: Stopped Device-Mapper Multipath Device Controller. Nov 16 11:49:56.399186 localhost systemd[1]: initrd-parse-etc.service: Succeeded. Nov 16 11:49:56.399520 localhost systemd[1]: Started Reload Configuration from the Real Root. Nov 16 11:49:56.481404 localhost systemd[1]: Reached target Initrd File Systems. Nov 16 11:49:56.516931 localhost systemd[1]: Reached target Initrd Default Target. Nov 16 11:49:56.560890 localhost systemd[1]: Starting dracut pre-pivot and cleanup hook... Nov 16 11:49:56.751761 localhost systemd[1]: Started dracut pre-pivot and cleanup hook. Nov 16 11:49:58.053678 localhost coreos-teardown-initramfs[1268]: info: taking down network device: eth0 Nov 16 11:49:56.825526 localhost systemd[1]: Starting Cleaning Up and Shutting Down Daemons... Nov 16 11:49:58.072194 localhost coreos-teardown-initramfs[1268]: RTNETLINK answers: Operation not supported Nov 16 11:49:56.894201 localhost systemd[1]: dracut-pre-pivot.service: Succeeded. Nov 16 11:49:58.088994 localhost coreos-teardown-initramfs[1268]: info: flushing all routing Nov 16 11:49:58.088994 localhost coreos-teardown-initramfs[1268]: info: hostname is defined in the real root Nov 16 11:49:58.088994 localhost coreos-teardown-initramfs[1268]: info: will not attempt to propagate initramfs hostname Nov 16 11:49:58.088994 localhost coreos-teardown-initramfs[1268]: info: no networking config is defined in the real root Nov 16 11:49:56.894606 localhost systemd[1]: Stopped dracut pre-pivot and cleanup hook. Nov 16 11:49:56.946775 localhost systemd[1]: clevis-luks-askpass.path: Succeeded. Nov 16 11:49:56.946842 localhost systemd[1]: Stopped Forward Password Requests to Clevis Directory Watch. Nov 16 11:49:57.020672 localhost systemd[1]: Stopped target Initrd Default Target. Nov 16 11:49:57.060998 localhost systemd[1]: Stopped target Remote File Systems. Nov 16 11:49:57.122035 localhost systemd[1]: ignition-ostree-check-rootfs-size.service: Succeeded. Nov 16 11:49:57.122338 localhost systemd[1]: Stopped Ignition OSTree: Check Root Filesystem Size. Nov 16 11:49:57.159549 localhost systemd[1]: ignition-ostree-growfs.service: Succeeded. Nov 16 11:49:57.159848 localhost systemd[1]: Stopped Ignition OSTree: Grow root filesystem. Nov 16 11:49:57.218083 localhost systemd[1]: rhcos-afterburn-checkin.service: Succeeded. Nov 16 11:49:57.218440 localhost systemd[1]: Stopped Afterburn (Check In - from the initramfs). Nov 16 11:49:57.268982 localhost systemd[1]: coreos-touch-run-agetty.service: Succeeded. Nov 16 11:49:57.269277 localhost systemd[1]: Stopped CoreOS: Touch /run/agetty.reload. Nov 16 11:49:58.228719 localhost ignition-ostree-mount-var[1299]: Unmounting /sysroot/var Nov 16 11:49:57.339388 localhost systemd[1]: Stopped target Initrd Root Device. Nov 16 11:49:57.382635 localhost systemd[1]: Stopped target Remote File Systems (Pre). Nov 16 11:49:57.425640 localhost systemd[1]: ignition-ostree-mount-firstboot-sysroot.service: Succeeded. Nov 16 11:49:57.425949 localhost systemd[1]: Stopped Ignition OSTree: Mount (firstboot) /sysroot. Nov 16 11:49:57.479123 localhost systemd[1]: ignition-ostree-uuid-root.service: Succeeded. Nov 16 11:49:57.479445 localhost systemd[1]: Stopped Ignition OSTree: Regenerate Filesystem UUID (root). Nov 16 11:49:57.535342 localhost systemd[1]: Stopped target Timers. Nov 16 11:49:57.589038 localhost systemd[1]: Stopped target Ignition Complete. Nov 16 11:49:57.630950 localhost systemd[1]: Stopped target Ignition Boot Disk Setup. Nov 16 11:49:57.662463 localhost systemd[1]: coreos-boot-edit.service: Succeeded. Nov 16 11:49:57.662757 localhost systemd[1]: Stopped CoreOS Boot Edit. Nov 16 11:49:57.708131 localhost systemd[1]: ignition-files.service: Succeeded. Nov 16 11:49:57.708421 localhost systemd[1]: Stopped Ignition (files). Nov 16 11:49:57.746123 localhost systemd[1]: ignition-ostree-populate-var.service: Succeeded. Nov 16 11:49:57.746409 localhost systemd[1]: Stopped Populate OSTree /var. Nov 16 11:49:57.786088 localhost systemd[1]: Stopping Ignition (mount)... Nov 16 11:49:57.817992 localhost systemd[1]: dracut-pre-mount.service: Succeeded. Nov 16 11:49:57.818330 localhost systemd[1]: Stopped dracut pre-mount hook. Nov 16 11:49:57.843467 localhost systemd[1]: dracut-initqueue.service: Succeeded. Nov 16 11:49:57.843778 localhost systemd[1]: Stopped dracut initqueue hook. Nov 16 11:49:57.846591 localhost systemd[1]: Stopping Open-iSCSI... Nov 16 11:49:57.867163 localhost systemd[1]: iscsid.service: Succeeded. Nov 16 11:49:57.867573 localhost systemd[1]: Stopped Open-iSCSI. Nov 16 11:49:57.885341 localhost systemd[1]: Stopping iSCSI UserSpace I/O driver... Nov 16 11:49:57.890582 localhost systemd[1]: iscsiuio.service: Succeeded. Nov 16 11:49:57.890893 localhost systemd[1]: Stopped iSCSI UserSpace I/O driver. Nov 16 11:49:57.922113 localhost systemd[1]: initrd-cleanup.service: Succeeded. Nov 16 11:49:57.922443 localhost systemd[1]: Started Cleaning Up and Shutting Down Daemons. Nov 16 11:49:57.973490 localhost systemd[1]: ignition-mount.service: Succeeded. Nov 16 11:49:57.973762 localhost systemd[1]: Stopped Ignition (mount). Nov 16 11:49:58.001775 localhost systemd[1]: Stopping CoreOS Tear Down Initramfs... Nov 16 11:49:58.023512 localhost systemd[1]: ignition-disks.service: Succeeded. Nov 16 11:49:58.023871 localhost systemd[1]: Stopped Ignition (disks). Nov 16 11:49:58.053524 localhost systemd[1]: rhcos-fips.service: Succeeded. Nov 16 11:49:58.053803 localhost systemd[1]: Stopped Check for FIPS mode. Nov 16 11:49:58.082119 localhost systemd[1]: ignition-kargs.service: Succeeded. Nov 16 11:49:58.082458 localhost systemd[1]: Stopped Ignition (kargs). Nov 16 11:49:58.114794 localhost systemd[1]: coreos-copy-firstboot-network.service: Succeeded. Nov 16 11:49:58.115069 localhost systemd[1]: Stopped Copy CoreOS Firstboot Networking Config. Nov 16 11:49:58.155225 localhost systemd[1]: Stopping Ignition OSTree: Detect Partition Transposition... Nov 16 11:49:58.180367 localhost systemd[1]: rhcos-fail-boot-for-legacy-luks-config.service: Succeeded. Nov 16 11:49:58.180665 localhost systemd[1]: Stopped RHCOS Check For Legacy LUKS Configuration. Nov 16 11:49:58.210974 localhost systemd[1]: Stopping Mount OSTree /var... Nov 16 11:49:58.229970 localhost systemd[1]: sysroot-var.mount: Succeeded. Nov 16 11:49:58.230882 localhost systemd[1]: ignition-ostree-transposefs-detect.service: Succeeded. Nov 16 11:49:58.231156 localhost systemd[1]: Stopped Ignition OSTree: Detect Partition Transposition. Nov 16 11:49:58.270895 localhost systemd[1]: ignition-fetch.service: Succeeded. Nov 16 11:49:58.271203 localhost systemd[1]: Stopped Ignition (fetch). Nov 16 11:49:58.293462 localhost systemd[1]: coreos-enable-network.service: Succeeded. Nov 16 11:49:58.293879 localhost systemd[1]: Stopped CoreOS Enable Network. Nov 16 11:49:58.320805 localhost systemd[1]: ignition-fetch-offline.service: Succeeded. Nov 16 11:49:58.321095 localhost systemd[1]: Stopped Ignition (fetch-offline). Nov 16 11:49:58.343533 localhost systemd[1]: Stopped target Basic System. Nov 16 11:49:58.366623 localhost systemd[1]: Stopped target System Initialization. Nov 16 11:49:58.392661 localhost systemd[1]: systemd-sysctl.service: Succeeded. Nov 16 11:49:58.392947 localhost systemd[1]: Stopped Apply Kernel Variables. Nov 16 11:49:58.415957 localhost systemd[1]: Stopped target Local Encrypted Volumes. Nov 16 11:49:58.437734 localhost systemd[1]: systemd-tmpfiles-setup.service: Succeeded. Nov 16 11:49:58.438018 localhost systemd[1]: Stopped Create Volatile Files and Directories. Nov 16 11:49:58.472101 localhost systemd[1]: systemd-modules-load.service: Succeeded. Nov 16 11:49:58.472432 localhost systemd[1]: Stopped Load Kernel Modules. Nov 16 11:49:58.689909 localhost coreos-teardown-initramfs[1268]: info: skipping propagation of default networking configs Nov 16 11:49:58.492080 localhost systemd[1]: Stopped target Local File Systems. Nov 16 11:49:58.514314 localhost systemd[1]: Stopped target Local File Systems (Pre). Nov 16 11:49:58.547202 localhost systemd[1]: systemd-udev-settle.service: Succeeded. Nov 16 11:49:58.547621 localhost systemd[1]: Stopped udev Wait for Complete Device Initialization. Nov 16 11:49:58.570648 localhost systemd[1]: systemd-udev-trigger.service: Succeeded. Nov 16 11:49:58.570920 localhost systemd[1]: Stopped udev Coldplug all Devices. Nov 16 11:49:58.598096 localhost systemd[1]: dracut-pre-trigger.service: Succeeded. Nov 16 11:49:58.598677 localhost systemd[1]: Stopped dracut pre-trigger hook. Nov 16 11:49:58.624920 localhost systemd[1]: Stopped target Swap. Nov 16 11:49:58.647171 localhost systemd[1]: Stopped target Slices. Nov 16 11:49:58.671206 localhost systemd[1]: Stopped target Sockets. Nov 16 11:49:58.700436 localhost systemd[1]: iscsiuio.socket: Succeeded. Nov 16 11:49:58.700836 localhost systemd[1]: Closed Open-iSCSI iscsiuio Socket. Nov 16 11:49:58.733346 localhost systemd[1]: iscsid.socket: Succeeded. Nov 16 11:49:58.733657 localhost systemd[1]: Closed Open-iSCSI iscsid Socket. Nov 16 11:49:58.766465 localhost systemd[1]: Stopped target Paths. Nov 16 11:49:58.788136 localhost systemd[1]: systemd-ask-password-console.path: Succeeded. Nov 16 11:49:58.791310 localhost systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. Nov 16 11:49:58.823385 localhost systemd[1]: coreos-ignition-setup-user.service: Succeeded. Nov 16 11:49:58.823673 localhost systemd[1]: Stopped CoreOS Ignition User Config Setup. Nov 16 11:49:58.850310 localhost systemd[1]: ignition-ostree-uuid-boot.service: Succeeded. Nov 16 11:49:58.850623 localhost systemd[1]: Stopped Ignition OSTree: Regenerate Filesystem UUID (boot). Nov 16 11:49:58.872911 localhost systemd[1]: coreos-gpt-setup.service: Succeeded. Nov 16 11:49:58.873199 localhost systemd[1]: Stopped Generate New UUID For Boot Disk GPT. Nov 16 11:49:58.892049 localhost systemd[1]: Stopping udev Kernel Device Manager... Nov 16 11:49:58.903858 localhost systemd[1]: coreos-teardown-initramfs.service: Succeeded. Nov 16 11:49:58.904118 localhost systemd[1]: Stopped CoreOS Tear Down Initramfs. Nov 16 11:49:58.923682 localhost systemd[1]: ignition-ostree-mount-var.service: Succeeded. Nov 16 11:49:58.923951 localhost systemd[1]: Stopped Mount OSTree /var. Nov 16 11:49:58.942325 localhost systemd[1]: systemd-udevd.service: Succeeded. Nov 16 11:49:58.942624 localhost systemd[1]: Stopped udev Kernel Device Manager. Nov 16 11:49:58.965338 localhost systemd[1]: dracut-pre-udev.service: Succeeded. Nov 16 11:49:58.965643 localhost systemd[1]: Stopped dracut pre-udev hook. Nov 16 11:49:58.986124 localhost systemd[1]: dracut-cmdline.service: Succeeded. Nov 16 11:49:58.986397 localhost systemd[1]: Stopped dracut cmdline hook. Nov 16 11:49:59.004500 localhost systemd[1]: dracut-cmdline-ask.service: Succeeded. Nov 16 11:49:59.004895 localhost systemd[1]: Stopped dracut ask for additional cmdline parameters. Nov 16 11:49:59.025702 localhost systemd[1]: systemd-tmpfiles-setup-dev.service: Succeeded. Nov 16 11:49:59.025999 localhost systemd[1]: Stopped Create Static Device Nodes in /dev. Nov 16 11:49:59.045704 localhost systemd[1]: kmod-static-nodes.service: Succeeded. Nov 16 11:49:59.045988 localhost systemd[1]: Stopped Create list of required static device nodes for the current kernel. Nov 16 11:49:59.070603 localhost systemd[1]: systemd-udevd-kernel.socket: Succeeded. Nov 16 11:49:59.070937 localhost systemd[1]: Closed udev Kernel Socket. Nov 16 11:49:59.088099 localhost systemd[1]: systemd-udevd-control.socket: Succeeded. Nov 16 11:49:59.088402 localhost systemd[1]: Closed udev Control Socket. Nov 16 11:49:59.107729 localhost systemd[1]: Starting Cleanup udevd DB... Nov 16 11:49:59.169739 localhost systemd[1]: initrd-udevadm-cleanup-db.service: Succeeded. Nov 16 11:49:59.170055 localhost systemd[1]: Started Cleanup udevd DB. Nov 16 11:49:59.187131 localhost systemd[1]: Reached target Switch Root. Nov 16 11:49:59.197314 localhost systemd[1]: Starting Switch Root... Nov 16 11:49:59.263698 localhost systemd[1]: Switching root. Nov 16 11:49:59.282339 localhost systemd-journald[374]: Journal stopped Nov 16 11:50:04.025674 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(10): [finished] writing file "/sysroot/etc/kubernetes/kubelet.conf" Nov 16 11:50:04.025718 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(11): [started] writing file "/sysroot/etc/machine-config-daemon/node-annotations.json" Nov 16 11:50:04.025730 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(11): [finished] writing file "/sysroot/etc/machine-config-daemon/node-annotations.json" Nov 16 11:50:04.025739 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(12): [started] writing file "/sysroot/etc/kubernetes/ca.crt" Nov 16 11:50:04.025747 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(12): [finished] writing file "/sysroot/etc/kubernetes/ca.crt" Nov 16 11:50:04.025756 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(13): [started] writing file "/sysroot/etc/kubernetes/cloud.conf" Nov 16 11:50:04.025766 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(13): [finished] writing file "/sysroot/etc/kubernetes/cloud.conf" Nov 16 11:50:04.025778 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(14): [started] writing file "/sysroot/etc/NetworkManager/conf.d/sdn.conf" Nov 16 11:50:04.025787 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(14): [finished] writing file "/sysroot/etc/NetworkManager/conf.d/sdn.conf" Nov 16 11:50:04.025796 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(15): [started] writing file "/sysroot/var/usrlocal/sbin/dynamic-system-reserved-calc.sh" Nov 16 11:50:04.025823 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(15): [finished] writing file "/sysroot/var/usrlocal/sbin/dynamic-system-reserved-calc.sh" Nov 16 11:50:04.025832 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(16): [started] writing file "/sysroot/etc/systemd/system.conf.d/10-default-env-godebug.conf" Nov 16 11:50:04.025841 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(16): [finished] writing file "/sysroot/etc/systemd/system.conf.d/10-default-env-godebug.conf" Nov 16 11:50:04.025851 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(17): [started] writing file "/sysroot/var/usrlocal/bin/mco-hostname" Nov 16 11:50:04.025869 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(17): [finished] writing file "/sysroot/var/usrlocal/bin/mco-hostname" Nov 16 11:50:04.025879 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(18): [started] writing file "/sysroot/etc/crio/crio.conf.d/01-mc-seccompUseDefault" Nov 16 11:50:04.025888 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(18): [finished] writing file "/sysroot/etc/crio/crio.conf.d/01-mc-seccompUseDefault" Nov 16 11:50:04.025898 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Ignition (files). Nov 16 11:50:04.025910 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(19): [started] writing file "/sysroot/etc/NetworkManager/dispatcher.d/99-vsphere-disable-tx-udp-tnl" Nov 16 11:50:04.025920 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(19): [finished] writing file "/sysroot/etc/NetworkManager/dispatcher.d/99-vsphere-disable-tx-udp-tnl" Nov 16 11:50:04.025929 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1a): [started] writing file "/sysroot/var/usrlocal/bin/configure-ovs.sh" Nov 16 11:50:04.025938 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1a): [finished] writing file "/sysroot/var/usrlocal/bin/configure-ovs.sh" Nov 16 11:50:04.025947 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1b): [started] writing file "/sysroot/etc/crio/crio.conf.d/00-default" Nov 16 11:50:04.025957 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1b): [finished] writing file "/sysroot/etc/crio/crio.conf.d/00-default" Nov 16 11:50:04.025966 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1c): [started] writing file "/sysroot/etc/audit/rules.d/mco-audit-quiet-containers.rules" Nov 16 11:50:04.025977 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1c): [finished] writing file "/sysroot/etc/audit/rules.d/mco-audit-quiet-containers.rules" Nov 16 11:50:04.025987 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1d): [started] writing file "/sysroot/etc/systemd/system.conf.d/kubelet-cgroups.conf" Nov 16 11:50:04.025996 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1d): [finished] writing file "/sysroot/etc/systemd/system.conf.d/kubelet-cgroups.conf" Nov 16 11:50:04.026005 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1e): [started] writing file "/sysroot/var/lib/kubelet/config.json" Nov 16 11:50:04.026015 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1e): [finished] writing file "/sysroot/var/lib/kubelet/config.json" Nov 16 11:50:04.026024 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1f): [started] writing file "/sysroot/etc/NetworkManager/conf.d/20-keyfiles.conf" Nov 16 11:50:04.026033 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(1f): [finished] writing file "/sysroot/etc/NetworkManager/conf.d/20-keyfiles.conf" Nov 16 11:50:04.026042 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Reload Configuration from the Real Root... Nov 16 11:50:04.026055 ropatil1611az-gs6zj-worker-westus-j4ngg multipathd[736]: exit (signal) Nov 16 11:50:04.026065 ropatil1611az-gs6zj-worker-westus-j4ngg multipathd[736]: --------shut down------- Nov 16 11:50:04.026074 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(20): [started] writing file "/sysroot/etc/systemd/system/kubelet.service.d/20-logging.conf" Nov 16 11:50:04.026084 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(20): [finished] writing file "/sysroot/etc/systemd/system/kubelet.service.d/20-logging.conf" Nov 16 11:50:04.026093 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(21): [started] writing file "/sysroot/etc/kubernetes/static-pod-resources/configmaps/cloud-config/ca-bundle.pem" Nov 16 11:50:04.026103 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(21): [finished] writing file "/sysroot/etc/kubernetes/static-pod-resources/configmaps/cloud-config/ca-bundle.pem" Nov 16 11:50:04.026112 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(22): [started] writing file "/sysroot/etc/pki/ca-trust/source/anchors/openshift-config-user-ca-bundle.crt" Nov 16 11:50:04.026127 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(22): [finished] writing file "/sysroot/etc/pki/ca-trust/source/anchors/openshift-config-user-ca-bundle.crt" Nov 16 11:50:04.026136 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(23): [started] writing file "/sysroot/etc/kubernetes/kubelet-plugins/volume/exec/.dummy" Nov 16 11:50:04.026146 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createFilesystemsFiles: createFiles: op(23): [finished] writing file "/sysroot/etc/kubernetes/kubelet-plugins/volume/exec/.dummy" Nov 16 11:50:04.026155 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(24): [started] processing unit "crio.service" Nov 16 11:50:04.026165 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(24): op(25): [started] writing systemd drop-in "10-mco-default-env.conf" at "/sysroot/etc/systemd/system/crio.service.d/10-mco-default-env.conf" Nov 16 11:50:04.026178 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(24): op(25): [finished] writing systemd drop-in "10-mco-default-env.conf" at "/sysroot/etc/systemd/system/crio.service.d/10-mco-default-env.conf" Nov 16 11:50:04.026187 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(24): op(26): [started] writing systemd drop-in "10-mco-profile-unix-socket.conf" at "/sysroot/etc/systemd/system/crio.service.d/10-mco-profile-unix-socket.conf" Nov 16 11:50:04.026197 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(24): op(26): [finished] writing systemd drop-in "10-mco-profile-unix-socket.conf" at "/sysroot/etc/systemd/system/crio.service.d/10-mco-profile-unix-socket.conf" Nov 16 11:50:04.026206 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting CoreOS Boot Edit... Nov 16 11:50:04.026243 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(24): op(27): [started] writing systemd drop-in "10-mco-default-madv.conf" at "/sysroot/etc/systemd/system/crio.service.d/10-mco-default-madv.conf" Nov 16 11:50:04.026259 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(24): op(27): [finished] writing systemd drop-in "10-mco-default-madv.conf" at "/sysroot/etc/systemd/system/crio.service.d/10-mco-default-madv.conf" Nov 16 11:50:04.026269 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(24): [finished] processing unit "crio.service" Nov 16 11:50:04.026286 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(28): [started] processing unit "docker.socket" Nov 16 11:50:04.026296 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(28): op(29): [started] writing systemd drop-in "mco-disabled.conf" at "/sysroot/etc/systemd/system/docker.socket.d/mco-disabled.conf" Nov 16 11:50:04.026307 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(28): op(29): [finished] writing systemd drop-in "mco-disabled.conf" at "/sysroot/etc/systemd/system/docker.socket.d/mco-disabled.conf" Nov 16 11:50:04.026317 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(28): [finished] processing unit "docker.socket" Nov 16 11:50:04.026327 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(2a): [started] processing unit "kubelet-auto-node-size.service" Nov 16 11:50:04.026337 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(2a): op(2b): [started] writing unit "kubelet-auto-node-size.service" at "/sysroot/etc/systemd/system/kubelet-auto-node-size.service" Nov 16 11:50:04.026347 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(2a): op(2b): [finished] writing unit "kubelet-auto-node-size.service" at "/sysroot/etc/systemd/system/kubelet-auto-node-size.service" Nov 16 11:50:04.026357 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(2a): [finished] processing unit "kubelet-auto-node-size.service" Nov 16 11:50:04.026370 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(2c): [started] processing unit "kubelet.service" Nov 16 11:50:04.026380 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(2c): op(2d): [started] writing systemd drop-in "10-mco-default-env.conf" at "/sysroot/etc/systemd/system/kubelet.service.d/10-mco-default-env.conf" Nov 16 11:50:04.026390 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(2c): op(2d): [finished] writing systemd drop-in "10-mco-default-env.conf" at "/sysroot/etc/systemd/system/kubelet.service.d/10-mco-default-env.conf" Nov 16 11:50:04.026400 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(2c): op(2e): [started] writing systemd drop-in "10-mco-default-madv.conf" at "/sysroot/etc/systemd/system/kubelet.service.d/10-mco-default-madv.conf" Nov 16 11:50:04.026411 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-boot-edit[1137]: Injected kernel arguments into BLS: root=UUID=566864e3-acbe-4961-8488-233c4e33ce96 rw rootflags=prjquota Nov 16 11:50:04.026421 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reloading. Nov 16 11:50:04.026431 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(2c): op(2e): [finished] writing systemd drop-in "10-mco-default-madv.conf" at "/sysroot/etc/systemd/system/kubelet.service.d/10-mco-default-madv.conf" Nov 16 11:50:04.026448 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(2c): op(2f): [started] writing unit "kubelet.service" at "/sysroot/etc/systemd/system/kubelet.service" Nov 16 11:50:04.026459 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(2c): op(2f): [finished] writing unit "kubelet.service" at "/sysroot/etc/systemd/system/kubelet.service" Nov 16 11:50:04.026469 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(2c): [finished] processing unit "kubelet.service" Nov 16 11:50:04.026479 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(30): [started] processing unit "machine-config-daemon-firstboot.service" Nov 16 11:50:04.026489 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(30): op(31): [started] writing unit "machine-config-daemon-firstboot.service" at "/sysroot/etc/systemd/system/machine-config-daemon-firstboot.service" Nov 16 11:50:04.026504 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(30): op(31): [finished] writing unit "machine-config-daemon-firstboot.service" at "/sysroot/etc/systemd/system/machine-config-daemon-firstboot.service" Nov 16 11:50:04.026514 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(30): [finished] processing unit "machine-config-daemon-firstboot.service" Nov 16 11:50:04.026524 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(32): [started] processing unit "machine-config-daemon-pull.service" Nov 16 11:50:04.026556 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(32): op(33): [started] writing unit "machine-config-daemon-pull.service" at "/sysroot/etc/systemd/system/machine-config-daemon-pull.service" Nov 16 11:50:04.026583 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(32): op(33): [finished] writing unit "machine-config-daemon-pull.service" at "/sysroot/etc/systemd/system/machine-config-daemon-pull.service" Nov 16 11:50:04.026613 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(32): [finished] processing unit "machine-config-daemon-pull.service" Nov 16 11:50:04.026623 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(34): [started] processing unit "node-valid-hostname.service" Nov 16 11:50:04.026632 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(34): op(35): [started] writing unit "node-valid-hostname.service" at "/sysroot/etc/systemd/system/node-valid-hostname.service" Nov 16 11:50:04.026642 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(34): op(35): [finished] writing unit "node-valid-hostname.service" at "/sysroot/etc/systemd/system/node-valid-hostname.service" Nov 16 11:50:04.026651 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(34): [finished] processing unit "node-valid-hostname.service" Nov 16 11:50:04.026661 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-boot-edit[1137]: random.trust_cpu=on console=tty0 console=ttyS0,115200n8 $ignition_firstboot ostree=/ostree/boot.1/rhcos/dfe3636c0ecb6ffa92f58233688e78d3a8f6fec44b763bfd21d40bdea12ba3ff/0 ignition.platform.id=azure root=UUID=566864e3-acbe-4961-8488-233c4e33ce96 rw rootflags=prjquota boot=UUID=4a20a463-024a-4bb3-afd8-31decc06c38d Nov 16 11:50:04.026674 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started CoreOS Boot Edit. Nov 16 11:50:04.026683 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(36): [started] processing unit "nodeip-configuration.service" Nov 16 11:50:04.026692 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(36): op(37): [started] writing unit "nodeip-configuration.service" at "/sysroot/etc/systemd/system/nodeip-configuration.service" Nov 16 11:50:04.026702 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(36): op(37): [finished] writing unit "nodeip-configuration.service" at "/sysroot/etc/systemd/system/nodeip-configuration.service" Nov 16 11:50:04.026711 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(36): [finished] processing unit "nodeip-configuration.service" Nov 16 11:50:04.026720 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(38): [started] processing unit "openvswitch.service" Nov 16 11:50:04.026730 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(38): [finished] processing unit "openvswitch.service" Nov 16 11:50:04.026740 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(39): [started] processing unit "ovs-configuration.service" Nov 16 11:50:04.026751 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(39): op(3a): [started] writing unit "ovs-configuration.service" at "/sysroot/etc/systemd/system/ovs-configuration.service" Nov 16 11:50:04.026761 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(39): op(3a): [finished] writing unit "ovs-configuration.service" at "/sysroot/etc/systemd/system/ovs-configuration.service" Nov 16 11:50:04.026770 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(39): [finished] processing unit "ovs-configuration.service" Nov 16 11:50:04.026780 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(3b): [started] processing unit "ovs-vswitchd.service" Nov 16 11:50:04.026789 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(3b): op(3c): [started] writing systemd drop-in "10-ovs-vswitchd-restart.conf" at "/sysroot/etc/systemd/system/ovs-vswitchd.service.d/10-ovs-vswitchd-restart.conf" Nov 16 11:50:04.026799 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(3b): op(3c): [finished] writing systemd drop-in "10-ovs-vswitchd-restart.conf" at "/sysroot/etc/systemd/system/ovs-vswitchd.service.d/10-ovs-vswitchd-restart.conf" Nov 16 11:50:04.026808 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(3b): [finished] processing unit "ovs-vswitchd.service" Nov 16 11:50:04.026821 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(3d): [started] processing unit "ovsdb-server.service" Nov 16 11:50:04.026831 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(3d): op(3e): [started] writing systemd drop-in "10-ovsdb-restart.conf" at "/sysroot/etc/systemd/system/ovsdb-server.service.d/10-ovsdb-restart.conf" Nov 16 11:50:04.026841 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(3d): op(3e): [finished] writing systemd drop-in "10-ovsdb-restart.conf" at "/sysroot/etc/systemd/system/ovsdb-server.service.d/10-ovsdb-restart.conf" Nov 16 11:50:04.026850 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Ignition Boot Disk Setup. Nov 16 11:50:04.026860 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(3d): [finished] processing unit "ovsdb-server.service" Nov 16 11:50:04.026869 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(3f): [started] processing unit "pivot.service" Nov 16 11:50:04.026878 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(3f): op(40): [started] writing systemd drop-in "10-mco-default-env.conf" at "/sysroot/etc/systemd/system/pivot.service.d/10-mco-default-env.conf" Nov 16 11:50:04.026888 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(3f): op(40): [finished] writing systemd drop-in "10-mco-default-env.conf" at "/sysroot/etc/systemd/system/pivot.service.d/10-mco-default-env.conf" Nov 16 11:50:04.026900 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(3f): [finished] processing unit "pivot.service" Nov 16 11:50:04.026909 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(41): [started] processing unit "zincati.service" Nov 16 11:50:04.026918 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(41): op(42): [started] writing systemd drop-in "mco-disabled.conf" at "/sysroot/etc/systemd/system/zincati.service.d/mco-disabled.conf" Nov 16 11:50:04.026928 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(41): op(42): [finished] writing systemd drop-in "mco-disabled.conf" at "/sysroot/etc/systemd/system/zincati.service.d/mco-disabled.conf" Nov 16 11:50:04.026937 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(41): [finished] processing unit "zincati.service" Nov 16 11:50:04.026947 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(43): [started] setting preset to enabled for "kubelet.service" Nov 16 11:50:04.026956 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(43): [finished] setting preset to enabled for "kubelet.service" Nov 16 11:50:04.026965 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(44): [started] setting preset to enabled for "machine-config-daemon-pull.service" Nov 16 11:50:04.026977 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(44): [finished] setting preset to enabled for "machine-config-daemon-pull.service" Nov 16 11:50:04.026987 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(45): [started] setting preset to enabled for "kubelet-auto-node-size.service" Nov 16 11:50:04.026997 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(45): [finished] setting preset to enabled for "kubelet-auto-node-size.service" Nov 16 11:50:04.027024 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(46): [started] setting preset to enabled for "machine-config-daemon-firstboot.service" Nov 16 11:50:04.027041 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(46): [finished] setting preset to enabled for "machine-config-daemon-firstboot.service" Nov 16 11:50:04.027052 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(47): [started] setting preset to enabled for "node-valid-hostname.service" Nov 16 11:50:04.027064 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(47): [finished] setting preset to enabled for "node-valid-hostname.service" Nov 16 11:50:04.027074 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Ignition Complete. Nov 16 11:50:04.027091 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(48): [started] setting preset to disabled for "nodeip-configuration.service" Nov 16 11:50:04.027101 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(48): [finished] setting preset to disabled for "nodeip-configuration.service" Nov 16 11:50:04.027110 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(49): [started] setting preset to enabled for "openvswitch.service" Nov 16 11:50:04.027120 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(49): [finished] setting preset to enabled for "openvswitch.service" Nov 16 11:50:04.027130 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(4a): [started] setting preset to enabled for "ovs-configuration.service" Nov 16 11:50:04.027140 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(4a): [finished] setting preset to enabled for "ovs-configuration.service" Nov 16 11:50:04.027149 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(4b): [started] setting preset to enabled for "ovsdb-server.service" Nov 16 11:50:04.027162 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(4b): [finished] setting preset to enabled for "ovsdb-server.service" Nov 16 11:50:04.027176 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createResultFile: createFiles: op(4c): [started] writing file "/sysroot/etc/.ignition-result.json" Nov 16 11:50:04.027186 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: createResultFile: createFiles: op(4c): [finished] writing file "/sysroot/etc/.ignition-result.json" Nov 16 11:50:04.027196 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(4d): [started] relabeling 64 patterns Nov 16 11:50:04.027206 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: DEBUG : files: op(4d): executing: "setfiles" "-vF0" "-r" "/sysroot" "/sysroot/etc/selinux/targeted/contexts/files/file_contexts" "-f" "-" Nov 16 11:50:04.027252 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: op(4d): [finished] relabeling 64 patterns Nov 16 11:50:04.027263 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : files: files passed Nov 16 11:50:04.027273 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1109]: INFO : Ignition finished successfully Nov 16 11:50:04.027284 ropatil1611az-gs6zj-worker-westus-j4ngg dracut-pre-pivot[1224]: Nov 16 11:49:56 | /etc/multipath.conf does not exist, blacklisting all devices. Nov 16 11:50:04.027308 ropatil1611az-gs6zj-worker-westus-j4ngg dracut-pre-pivot[1224]: Nov 16 11:49:56 | You can run "/sbin/mpathconf --enable" to create Nov 16 11:50:04.027342 ropatil1611az-gs6zj-worker-westus-j4ngg dracut-pre-pivot[1224]: Nov 16 11:49:56 | /etc/multipath.conf. See man mpathconf(8) for more details Nov 16 11:50:04.027368 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Device-Mapper Multipath Device Controller... Nov 16 11:50:04.027378 ropatil1611az-gs6zj-worker-westus-j4ngg iscsid[739]: iscsid: iscsid shutting down. Nov 16 11:50:04.027388 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: multipathd.service: Succeeded. Nov 16 11:50:04.027397 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1254]: INFO : Ignition 2.13.0 Nov 16 11:50:04.027407 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1254]: INFO : Stage: umount Nov 16 11:50:04.027419 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1254]: INFO : reading system config file "/usr/lib/ignition/base.d/00-core.ign" Nov 16 11:50:04.027429 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1254]: DEBUG : parsing config with SHA512: ff6a5153be363997e4d5d3ea8cc4048373a457c48c4a5b134a08a30aacd167c1e0f099f0bdf1e24c99ad180628cd02b767b863b5fe3a8fce3fe1886847eb8e2e Nov 16 11:50:04.027438 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1254]: INFO : no config dir at "/usr/lib/ignition/base.platform.d/azure" Nov 16 11:50:04.027448 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1254]: INFO : Adding "root-ca" to list of CAs Nov 16 11:50:04.027458 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1254]: INFO : umount: umount passed Nov 16 11:50:04.027468 ropatil1611az-gs6zj-worker-westus-j4ngg ignition[1254]: INFO : Ignition finished successfully Nov 16 11:50:04.027478 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Device-Mapper Multipath Device Controller. Nov 16 11:50:04.027487 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: initrd-parse-etc.service: Succeeded. Nov 16 11:50:04.027497 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Reload Configuration from the Real Root. Nov 16 11:50:04.027509 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Initrd File Systems. Nov 16 11:50:04.027519 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Initrd Default Target. Nov 16 11:50:04.027529 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting dracut pre-pivot and cleanup hook... Nov 16 11:50:04.027538 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started dracut pre-pivot and cleanup hook. Nov 16 11:50:04.027564 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-teardown-initramfs[1268]: info: taking down network device: eth0 Nov 16 11:50:04.027575 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Cleaning Up and Shutting Down Daemons... Nov 16 11:50:04.027584 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-teardown-initramfs[1268]: RTNETLINK answers: Operation not supported Nov 16 11:50:04.027594 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-pre-pivot.service: Succeeded. Nov 16 11:50:04.027611 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-teardown-initramfs[1268]: info: flushing all routing Nov 16 11:50:04.027621 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-teardown-initramfs[1268]: info: hostname is defined in the real root Nov 16 11:50:04.027631 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-teardown-initramfs[1268]: info: will not attempt to propagate initramfs hostname Nov 16 11:50:04.027640 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-teardown-initramfs[1268]: info: no networking config is defined in the real root Nov 16 11:50:04.027650 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut pre-pivot and cleanup hook. Nov 16 11:50:04.027660 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: clevis-luks-askpass.path: Succeeded. Nov 16 11:50:04.027669 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Forward Password Requests to Clevis Directory Watch. Nov 16 11:50:04.027679 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Initrd Default Target. Nov 16 11:50:04.027690 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Remote File Systems. Nov 16 11:50:04.027700 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-ostree-check-rootfs-size.service: Succeeded. Nov 16 11:50:04.027710 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Ignition OSTree: Check Root Filesystem Size. Nov 16 11:50:04.027719 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-ostree-growfs.service: Succeeded. Nov 16 11:50:04.027728 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Ignition OSTree: Grow root filesystem. Nov 16 11:50:04.027738 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: rhcos-afterburn-checkin.service: Succeeded. Nov 16 11:50:04.027748 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Afterburn (Check In - from the initramfs). Nov 16 11:50:04.027757 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-touch-run-agetty.service: Succeeded. Nov 16 11:50:04.027766 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped CoreOS: Touch /run/agetty.reload. Nov 16 11:50:04.027779 ropatil1611az-gs6zj-worker-westus-j4ngg ignition-ostree-mount-var[1299]: Unmounting /sysroot/var Nov 16 11:50:04.027789 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Initrd Root Device. Nov 16 11:50:04.027799 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Remote File Systems (Pre). Nov 16 11:50:04.027808 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-ostree-mount-firstboot-sysroot.service: Succeeded. Nov 16 11:50:04.027818 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Ignition OSTree: Mount (firstboot) /sysroot. Nov 16 11:50:04.027827 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-ostree-uuid-root.service: Succeeded. Nov 16 11:50:04.027836 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Ignition OSTree: Regenerate Filesystem UUID (root). Nov 16 11:50:04.027846 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Timers. Nov 16 11:50:04.027855 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Ignition Complete. Nov 16 11:50:04.027867 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Ignition Boot Disk Setup. Nov 16 11:50:04.027877 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-boot-edit.service: Succeeded. Nov 16 11:50:04.027887 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped CoreOS Boot Edit. Nov 16 11:50:04.027896 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-files.service: Succeeded. Nov 16 11:50:04.027905 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Ignition (files). Nov 16 11:50:04.027914 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-ostree-populate-var.service: Succeeded. Nov 16 11:50:04.027924 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Populate OSTree /var. Nov 16 11:50:04.027936 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Ignition (mount)... Nov 16 11:50:04.027945 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-pre-mount.service: Succeeded. Nov 16 11:50:04.027954 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut pre-mount hook. Nov 16 11:50:04.027964 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-initqueue.service: Succeeded. Nov 16 11:50:04.027973 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut initqueue hook. Nov 16 11:50:04.027982 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Open-iSCSI... Nov 16 11:50:04.027992 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: iscsid.service: Succeeded. Nov 16 11:50:04.028002 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Open-iSCSI. Nov 16 11:50:04.028011 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping iSCSI UserSpace I/O driver... Nov 16 11:50:04.028023 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: iscsiuio.service: Succeeded. Nov 16 11:50:04.028033 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped iSCSI UserSpace I/O driver. Nov 16 11:50:04.028043 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: initrd-cleanup.service: Succeeded. Nov 16 11:50:04.028052 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Cleaning Up and Shutting Down Daemons. Nov 16 11:50:04.028061 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-mount.service: Succeeded. Nov 16 11:50:04.028071 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Ignition (mount). Nov 16 11:50:04.028080 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping CoreOS Tear Down Initramfs... Nov 16 11:50:04.028090 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-disks.service: Succeeded. Nov 16 11:50:04.028099 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Ignition (disks). Nov 16 11:50:04.028111 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: rhcos-fips.service: Succeeded. Nov 16 11:50:04.028121 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Check for FIPS mode. Nov 16 11:50:04.028134 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-kargs.service: Succeeded. Nov 16 11:50:04.028164 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Ignition (kargs). Nov 16 11:50:04.028174 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-copy-firstboot-network.service: Succeeded. Nov 16 11:50:04.028184 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Copy CoreOS Firstboot Networking Config. Nov 16 11:50:04.028194 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Ignition OSTree: Detect Partition Transposition... Nov 16 11:50:04.028204 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: rhcos-fail-boot-for-legacy-luks-config.service: Succeeded. Nov 16 11:50:04.028222 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped RHCOS Check For Legacy LUKS Configuration. Nov 16 11:50:04.028254 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Mount OSTree /var... Nov 16 11:50:04.028268 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-var.mount: Succeeded. Nov 16 11:50:04.028279 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-ostree-transposefs-detect.service: Succeeded. Nov 16 11:50:04.028290 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Ignition OSTree: Detect Partition Transposition. Nov 16 11:50:04.028300 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-fetch.service: Succeeded. Nov 16 11:50:04.028311 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Ignition (fetch). Nov 16 11:50:04.028321 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-enable-network.service: Succeeded. Nov 16 11:50:04.028331 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped CoreOS Enable Network. Nov 16 11:50:04.028342 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-fetch-offline.service: Succeeded. Nov 16 11:50:04.028353 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Ignition (fetch-offline). Nov 16 11:50:04.028366 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Basic System. Nov 16 11:50:04.028377 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target System Initialization. Nov 16 11:50:04.028387 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-sysctl.service: Succeeded. Nov 16 11:50:04.028397 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Apply Kernel Variables. Nov 16 11:50:04.028407 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Local Encrypted Volumes. Nov 16 11:50:04.028418 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-tmpfiles-setup.service: Succeeded. Nov 16 11:50:04.028428 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Create Volatile Files and Directories. Nov 16 11:50:04.028438 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-modules-load.service: Succeeded. Nov 16 11:50:04.028448 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Load Kernel Modules. Nov 16 11:50:04.028477 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-teardown-initramfs[1268]: info: skipping propagation of default networking configs Nov 16 11:50:04.028487 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Local File Systems. Nov 16 11:50:04.028500 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Local File Systems (Pre). Nov 16 11:50:04.028510 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-udev-settle.service: Succeeded. Nov 16 11:50:04.028535 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped udev Wait for Complete Device Initialization. Nov 16 11:50:04.028545 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-udev-trigger.service: Succeeded. Nov 16 11:50:04.028555 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped udev Coldplug all Devices. Nov 16 11:50:04.028564 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-pre-trigger.service: Succeeded. Nov 16 11:50:04.028573 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut pre-trigger hook. Nov 16 11:50:04.028586 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Swap. Nov 16 11:50:04.028596 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Slices. Nov 16 11:50:04.028605 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Sockets. Nov 16 11:50:04.028615 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: iscsiuio.socket: Succeeded. Nov 16 11:50:04.028624 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Closed Open-iSCSI iscsiuio Socket. Nov 16 11:50:04.028640 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: iscsid.socket: Succeeded. Nov 16 11:50:04.028649 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Closed Open-iSCSI iscsid Socket. Nov 16 11:50:04.028658 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Paths. Nov 16 11:50:04.028668 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-ask-password-console.path: Succeeded. Nov 16 11:50:04.028686 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. Nov 16 11:50:04.028698 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-ignition-setup-user.service: Succeeded. Nov 16 11:50:04.028707 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped CoreOS Ignition User Config Setup. Nov 16 11:50:04.028716 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-ostree-uuid-boot.service: Succeeded. Nov 16 11:50:04.028726 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Ignition OSTree: Regenerate Filesystem UUID (boot). Nov 16 11:50:04.028737 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-gpt-setup.service: Succeeded. Nov 16 11:50:04.028747 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Generate New UUID For Boot Disk GPT. Nov 16 11:50:04.028756 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping udev Kernel Device Manager... Nov 16 11:50:04.028765 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-teardown-initramfs.service: Succeeded. Nov 16 11:50:04.028777 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped CoreOS Tear Down Initramfs. Nov 16 11:50:04.028787 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ignition-ostree-mount-var.service: Succeeded. Nov 16 11:50:04.028799 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Mount OSTree /var. Nov 16 11:50:04.028808 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-udevd.service: Succeeded. Nov 16 11:50:04.028817 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped udev Kernel Device Manager. Nov 16 11:50:04.028827 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-pre-udev.service: Succeeded. Nov 16 11:50:04.028837 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut pre-udev hook. Nov 16 11:50:04.028846 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-cmdline.service: Succeeded. Nov 16 11:50:04.028855 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut cmdline hook. Nov 16 11:50:04.028864 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-cmdline-ask.service: Succeeded. Nov 16 11:50:04.028873 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut ask for additional cmdline parameters. Nov 16 11:50:04.028886 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-tmpfiles-setup-dev.service: Succeeded. Nov 16 11:50:04.028895 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Create Static Device Nodes in /dev. Nov 16 11:50:04.028904 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kmod-static-nodes.service: Succeeded. Nov 16 11:50:04.028914 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Create list of required static device nodes for the current kernel. Nov 16 11:50:04.028923 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-udevd-kernel.socket: Succeeded. Nov 16 11:50:04.028933 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Closed udev Kernel Socket. Nov 16 11:50:04.028944 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-udevd-control.socket: Succeeded. Nov 16 11:50:04.028953 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Closed udev Control Socket. Nov 16 11:50:04.028963 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Cleanup udevd DB... Nov 16 11:50:04.028975 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: initrd-udevadm-cleanup-db.service: Succeeded. Nov 16 11:50:04.028984 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Cleanup udevd DB. Nov 16 11:50:04.028993 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Switch Root. Nov 16 11:50:04.029003 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Switch Root... Nov 16 11:50:04.029012 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Switching root. Nov 16 11:50:04.029022 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: printk: systemd-journal: 1 output lines suppressed due to ratelimiting Nov 16 11:50:04.029031 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: printk: systemd: 29 output lines suppressed due to ratelimiting Nov 16 11:50:04.029041 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: audit: type=1404 audit(1668599399.961:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295 enabled=1 old-enabled=1 lsm=selinux res=1 Nov 16 11:50:04.029052 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: SELinux: policy capability network_peer_controls=1 Nov 16 11:50:04.029061 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: SELinux: policy capability open_perms=1 Nov 16 11:50:04.029070 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: SELinux: policy capability extended_socket_class=1 Nov 16 11:50:04.029078 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: SELinux: policy capability always_check_network=0 Nov 16 11:50:04.029087 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: SELinux: policy capability cgroup_seclabel=1 Nov 16 11:50:04.029095 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: SELinux: policy capability nnp_nosuid_transition=1 Nov 16 11:50:04.029104 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: audit: type=1403 audit(1668599400.847:3): auid=4294967295 ses=4294967295 lsm=selinux res=1 Nov 16 11:50:04.029113 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Successfully loaded SELinux policy in 888.596ms. Nov 16 11:50:04.029122 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Relabelled /dev, /run and /sys/fs/cgroup in 32.061ms. Nov 16 11:50:04.029135 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd 239 (239-45.el8_4.12) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) Nov 16 11:50:04.029147 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Detected virtualization microsoft. Nov 16 11:50:04.029156 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Detected architecture x86-64. Nov 16 11:50:04.029172 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Running with unpopulated /etc. Nov 16 11:50:04.029181 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Set hostname to . Nov 16 11:50:04.029191 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Initializing machine ID from random generator. Nov 16 11:50:04.029200 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-platform-chrony: Updated chrony to use azure configuration /run/coreos-platform-chrony.conf Nov 16 11:50:04.029217 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Populated /etc with preset unit settings. Nov 16 11:50:04.029269 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: /usr/lib/systemd/system/bootupd.service:22: Unknown lvalue 'ProtectHostname' in section 'Service' Nov 16 11:50:04.029280 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journald.service: Succeeded. Nov 16 11:50:04.029291 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journald.service: Consumed 0 CPU time Nov 16 11:50:04.029301 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: initrd-switch-root.service: Succeeded. Nov 16 11:50:04.029311 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Switch Root. Nov 16 11:50:04.029322 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: initrd-switch-root.service: Consumed 0 CPU time Nov 16 11:50:04.029332 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. Nov 16 11:50:04.029342 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. Nov 16 11:50:04.029370 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Journal Service. Nov 16 11:50:04.029383 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journald.service: Consumed 0 CPU time Nov 16 11:50:04.029393 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Journal Service... Nov 16 11:50:04.029404 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on Device-mapper event daemon FIFOs. Nov 16 11:50:04.029414 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on udev Control Socket. Nov 16 11:50:04.029424 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on initctl Compatibility Named Pipe. Nov 16 11:50:04.029435 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on Process Core Dump Socket. Nov 16 11:50:04.029445 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Load Kernel Modules... Nov 16 11:50:04.029455 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice User and Session Slice. Nov 16 11:50:04.029465 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Create System Users... Nov 16 11:50:04.029475 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Swap. Nov 16 11:50:04.029487 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting Temporary Directory (/tmp)... Nov 16 11:50:04.029498 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice system-serial\x2dgetty.slice. Nov 16 11:50:04.029508 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Dispatch Password Requests to Console Directory Watch. Nov 16 11:50:04.029518 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting Huge Pages File System... Nov 16 11:50:04.029528 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Create list of required static device nodes for the current kernel... Nov 16 11:50:04.029539 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point. Nov 16 11:50:04.029571 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-journald[1367]: Journal started Nov 16 11:50:04.029642 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-journald[1367]: Runtime journal (/run/log/journal/17c3dc7b18784633a4bd61ccb76c02a9) is 8.0M, max 801.1M, 793.1M free. Nov 16 11:50:04.052000 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting Kernel Debug File System... Nov 16 11:50:04.061409 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Switch Root. Nov 16 11:50:04.077901 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Forward Password Requests to Wall Directory Watch. Nov 16 11:50:04.095275 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting POSIX Message Queue File System... Nov 16 11:50:04.113417 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling... Nov 16 11:50:04.114571 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-modules-load[1368]: Module 'msr' is builtin Nov 16 11:50:04.129191 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Forward Password Requests to Clevis Directory Watch. Nov 16 11:50:04.143596 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Local Encrypted Volumes (Pre). Nov 16 11:50:04.143612 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-modules-load[1368]: Inserted module 'ip_tables' Nov 16 11:50:04.157270 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Remote Encrypted Volumes. Nov 16 11:50:04.171881 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Local Encrypted Volumes. Nov 16 11:50:04.186014 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on LVM2 poll daemon socket. Nov 16 11:50:04.198656 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Remote File Systems. Nov 16 11:50:04.211130 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Initrd File Systems. Nov 16 11:50:04.224049 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on udev Kernel Socket. Nov 16 11:50:04.237416 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting udev Coldplug all Devices... Nov 16 11:50:04.250271 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Host and Network Name Lookups. Nov 16 11:50:04.263299 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice system-getty.slice. Nov 16 11:50:04.279137 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice system-sshd\x2dkeygen.slice. Nov 16 11:50:04.293941 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on RPCbind Server Activation Socket. Nov 16 11:50:04.307189 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target RPC Port Mapper. Nov 16 11:50:04.319343 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice system-systemd\x2dfsck.slice. Nov 16 11:50:04.338311 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Synchronize afterburn-sshkeys@.service template instances. Nov 16 11:50:04.356114 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Initrd Root File System. Nov 16 11:50:04.370942 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Rebuild Hardware Database... Nov 16 11:50:04.390012 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ostree-prepare-root.service: Succeeded. Nov 16 11:50:04.390051 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped OSTree Prepare OS/. Nov 16 11:50:04.409868 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ostree-prepare-root.service: Consumed 0 CPU time Nov 16 11:50:04.409910 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Slices. Nov 16 11:50:04.427276 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-usr.mount: Succeeded. Nov 16 11:50:04.445116 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-usr.mount: Consumed 0 CPU time Nov 16 11:50:04.445163 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-sysroot-ostree-deploy-rhcos-var.mount: Succeeded. Nov 16 11:50:04.462190 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-sysroot-ostree-deploy-rhcos-var.mount: Consumed 0 CPU time Nov 16 11:50:04.462278 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-etc.mount: Succeeded. Nov 16 11:50:04.477022 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-etc.mount: Consumed 0 CPU time Nov 16 11:50:04.477065 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-sysroot.mount: Succeeded. Nov 16 11:50:04.493031 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-sysroot.mount: Consumed 0 CPU time Nov 16 11:50:04.493076 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Journal Service. Nov 16 11:50:04.498943 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Load Kernel Modules. Nov 16 11:50:04.504908 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Create System Users. Nov 16 11:50:04.512840 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted Temporary Directory (/tmp). Nov 16 11:50:04.518483 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted Huge Pages File System. Nov 16 11:50:04.524674 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Create list of required static device nodes for the current kernel. Nov 16 11:50:04.532588 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted Kernel Debug File System. Nov 16 11:50:04.538088 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted POSIX Message Queue File System. Nov 16 11:50:04.548397 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Create Static Device Nodes in /dev... Nov 16 11:50:04.554774 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Apply Kernel Variables... Nov 16 11:50:04.561841 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting FUSE Control File System... Nov 16 11:50:04.567785 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted FUSE Control File System. Nov 16 11:50:04.582746 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Create Static Device Nodes in /dev. Nov 16 11:50:04.592209 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Apply Kernel Variables. Nov 16 11:50:04.607014 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started udev Coldplug all Devices. Nov 16 11:50:04.613392 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting udev Wait for Complete Device Initialization... Nov 16 11:50:05.096880 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Rebuild Hardware Database. Nov 16 11:50:05.105476 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting udev Kernel Device Manager... Nov 16 11:50:05.360597 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started udev Kernel Device Manager. Nov 16 11:50:05.398729 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[1419]: Using default interface naming scheme 'rhel-8.0'. Nov 16 11:50:05.399227 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[1419]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:50:05.428725 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_utils: Registering HyperV Utility Driver Nov 16 11:50:05.428778 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_vmbus: registering driver hv_utils Nov 16 11:50:05.437306 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_vmbus: registering driver hyperv_fb Nov 16 11:50:05.437353 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_utils: Heartbeat IC version 3.0 Nov 16 11:50:05.448284 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_utils: Shutdown IC version 3.2 Nov 16 11:50:05.448338 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_utils: TimeSync IC version 4.0 Nov 16 11:50:05.454039 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_vmbus: registering driver hv_balloon Nov 16 11:50:06.689319 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hyperv_fb: Synthvid Version major 3, minor 5 Nov 16 11:50:06.689364 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_balloon: Using Dynamic Memory protocol version 2.0 Nov 16 11:50:06.696616 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hyperv_fb: Screen resolution: 1024x768, Color depth: 32 Nov 16 11:50:06.715276 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hyperv_fb: Unable to allocate enough contiguous physical memory on Gen 1 VM. Using MMIO instead. Nov 16 11:50:06.718725 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: Console: switching to colour frame buffer device 128x48 Nov 16 11:50:06.745670 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: piix4_smbus 0000:00:07.3: SMBus base address uninitialized - upgrade BIOS or use force_addr=0xaddr Nov 16 11:50:06.756666 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: input: PC Speaker as /devices/platform/pcspkr/input/input6 Nov 16 11:50:06.791566 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[1409]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:50:06.891461 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling. Nov 16 11:50:06.897762 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer Nov 16 11:50:06.945761 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: EDAC sbridge: Seeking for: PCI ID 8086:6fa0 Nov 16 11:50:06.945852 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: EDAC sbridge: Ver: 1.1.2 Nov 16 11:50:07.105370 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started udev Wait for Complete Device Initialization. Nov 16 11:50:07.112169 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Local File Systems (Pre). Nov 16 11:50:07.117539 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var.mount: Directory /var to mount over is not empty, mounting anyway. Nov 16 11:50:07.118769 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting /var... Nov 16 11:50:07.123817 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting File System Check on /dev/disk/by-uuid/4a20a463-024a-4bb3-afd8-31decc06c38d... Nov 16 11:50:07.132822 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted /var. Nov 16 11:50:07.138339 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting OSTree Remount OS/ Bind Mounts... Nov 16 11:50:07.159099 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started OSTree Remount OS/ Bind Mounts. Nov 16 11:50:07.164526 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Load/Save Random Seed... Nov 16 11:50:07.168548 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Flush Journal to Persistent Storage... Nov 16 11:50:07.185178 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Load/Save Random Seed. Nov 16 11:50:07.210521 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-journald[1367]: Time spent on flushing to /var is 13.355ms for 1593 entries. Nov 16 11:50:07.210521 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-journald[1367]: System journal (/var/log/journal/17c3dc7b18784633a4bd61ccb76c02a9) is 8.0M, max 4.0G, 3.9G free. Nov 16 11:50:07.245167 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Flush Journal to Persistent Storage. Nov 16 11:50:07.383611 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-fsck[1466]: boot: clean, 321/98304 files, 126961/393216 blocks Nov 16 11:50:07.394756 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started File System Check on /dev/disk/by-uuid/4a20a463-024a-4bb3-afd8-31decc06c38d. Nov 16 11:50:07.406943 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting CoreOS Dynamic Mount for /boot... Nov 16 11:50:07.422661 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: EXT4-fs (sdb3): mounted filesystem with ordered data mode. Opts: (null) Nov 16 11:50:07.424843 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted CoreOS Dynamic Mount for /boot. Nov 16 11:50:07.430169 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Local File Systems. Nov 16 11:50:07.435807 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Create Volatile Files and Directories... Nov 16 11:50:07.441054 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Rebuild Dynamic Linker Cache... Nov 16 11:50:07.445084 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1484]: [/usr/lib/tmpfiles.d/pkg-dbus-daemon.conf:1] Duplicate line for path "/var/lib/dbus", ignoring. Nov 16 11:50:07.445792 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Rebuild Journal Catalog... Nov 16 11:50:07.448132 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1484]: [/usr/lib/tmpfiles.d/tmp.conf:12] Duplicate line for path "/var/tmp", ignoring. Nov 16 11:50:07.448242 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1484]: [/usr/lib/tmpfiles.d/var.conf:14] Duplicate line for path "/var/log", ignoring. Nov 16 11:50:07.448405 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1484]: [/usr/lib/tmpfiles.d/var.conf:19] Duplicate line for path "/var/cache", ignoring. Nov 16 11:50:07.448417 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1484]: [/usr/lib/tmpfiles.d/var.conf:21] Duplicate line for path "/var/lib", ignoring. Nov 16 11:50:07.448422 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1484]: [/usr/lib/tmpfiles.d/var.conf:23] Duplicate line for path "/var/spool", ignoring. Nov 16 11:50:07.450915 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Run update-ca-trust... Nov 16 11:50:07.456314 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Restore /run/initramfs on shutdown... Nov 16 11:50:07.466627 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1484]: "/home" already exists and is not a directory. Nov 16 11:50:07.466779 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1484]: "/srv" already exists and is not a directory. Nov 16 11:50:07.494337 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Restore /run/initramfs on shutdown. Nov 16 11:50:07.531589 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Rebuild Journal Catalog. Nov 16 11:50:07.581808 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Create Volatile Files and Directories. Nov 16 11:50:07.588815 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Security Auditing Service... Nov 16 11:50:07.593943 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting RHCOS Fix SELinux Labeling For /usr/local/sbin... Nov 16 11:50:07.596704 ropatil1611az-gs6zj-worker-westus-j4ngg chcon[1494]: changing security context of '/usr/local/sbin' Nov 16 11:50:07.656114 ropatil1611az-gs6zj-worker-westus-j4ngg sh[1496]: changing security context of '/var/usrlocal/sbin' Nov 16 11:50:07.657487 ropatil1611az-gs6zj-worker-westus-j4ngg sh[1496]: changing security context of '/var/usrlocal/sbin/dynamic-system-reserved-calc.sh' Nov 16 11:50:07.687433 ropatil1611az-gs6zj-worker-westus-j4ngg auditd[1501]: No plugins found, not dispatching events Nov 16 11:50:07.687947 ropatil1611az-gs6zj-worker-westus-j4ngg auditd[1501]: Init complete, auditd 3.0 listening for events (startup state enable) Nov 16 11:50:07.710712 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started RHCOS Fix SELinux Labeling For /usr/local/sbin. Nov 16 11:50:08.030361 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Run update-ca-trust. Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: No rules Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: enabled 1 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: failure 1 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: pid 1501 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: rate_limit 0 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: backlog_limit 8192 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: lost 0 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: backlog 4 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: backlog_wait_time 60000 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: enabled 1 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: failure 1 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: pid 1501 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: rate_limit 0 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: backlog_limit 8192 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: lost 0 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: backlog 0 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: backlog_wait_time 60000 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: enabled 1 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: failure 1 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: pid 1501 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: rate_limit 0 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: backlog_limit 8192 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: lost 0 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: backlog 0 Nov 16 11:50:08.118989 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1504]: backlog_wait_time 60000 Nov 16 11:50:08.126006 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Security Auditing Service. Nov 16 11:50:08.134925 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Update UTMP about System Boot/Shutdown... Nov 16 11:50:08.148254 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Update UTMP about System Boot/Shutdown. Nov 16 11:50:12.778230 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Rebuild Dynamic Linker Cache. Nov 16 11:50:12.785289 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Update is Completed... Nov 16 11:50:12.795819 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Update is Completed. Nov 16 11:50:12.801376 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target System Initialization. Nov 16 11:50:12.806941 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Daily rotation of log files. Nov 16 11:50:12.813104 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on D-Bus System Message Bus Socket. Nov 16 11:50:12.818253 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started OSTree Monitor Staged Deployment. Nov 16 11:50:12.823760 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started daily update of the root trust anchor for DNSSEC. Nov 16 11:50:12.829354 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Daily Cleanup of Temporary Directories. Nov 16 11:50:12.834541 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Timers. Nov 16 11:50:12.839073 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Monitor console-login-helper-messages runtime issue snippets directory for changes. Nov 16 11:50:12.846343 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Paths. Nov 16 11:50:12.851978 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on bootupd.socket. Nov 16 11:50:12.857037 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Sockets. Nov 16 11:50:12.861553 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Basic System. Nov 16 11:50:12.867535 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Generation of shadow ID ranges for CRI-O... Nov 16 11:50:12.872893 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting CoreOS Generate iSCSI Initiator Name... Nov 16 11:50:12.878466 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting OpenSSH ecdsa Server Key Generation... Nov 16 11:50:12.885446 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Open vSwitch Database Unit... Nov 16 11:50:12.889864 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Check if Ignition config is provided... Nov 16 11:50:12.894682 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Generate console-login-helper-messages issue snippet... Nov 16 11:50:12.901058 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting CRI-O Auto Update Script... Nov 16 11:50:12.905348 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started D-Bus System Message Bus. Nov 16 11:50:12.912812 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting CoreOS Mark Ignition Boot Complete... Nov 16 11:50:12.919172 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting NTP client/server... Nov 16 11:50:12.923707 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting System Security Services Daemon... Nov 16 11:50:12.928710 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: EXT4-fs (sdb3): re-mounted. Opts: Nov 16 11:50:12.933551 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started irqbalance daemon. Nov 16 11:50:12.940216 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting OpenSSH ed25519 Server Key Generation... Nov 16 11:50:12.946008 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting OpenSSH rsa Server Key Generation... Nov 16 11:50:12.969279 ropatil1611az-gs6zj-worker-westus-j4ngg chown[1571]: /usr/bin/chown: cannot access '/var/run/openvswitch': No such file or directory Nov 16 11:50:12.993842 ropatil1611az-gs6zj-worker-westus-j4ngg chronyd[1584]: chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG) Nov 16 11:50:13.024742 ropatil1611az-gs6zj-worker-westus-j4ngg chronyd[1584]: Using right/UTC timezone to obtain leap second data Nov 16 11:50:13.258794 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started CoreOS Generate iSCSI Initiator Name. Nov 16 11:50:13.264597 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sshd-keygen@ecdsa.service: Succeeded. Nov 16 11:50:13.265299 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started OpenSSH ecdsa Server Key Generation. Nov 16 11:50:13.270962 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sshd-keygen@ecdsa.service: Consumed 19ms CPU time Nov 16 11:50:13.271882 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Check if Ignition config is provided. Nov 16 11:50:13.277623 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started CoreOS Mark Ignition Boot Complete. Nov 16 11:50:13.286386 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sshd-keygen@ed25519.service: Succeeded. Nov 16 11:50:13.287019 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started OpenSSH ed25519 Server Key Generation. Nov 16 11:50:13.292622 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sshd-keygen@ed25519.service: Consumed 23ms CPU time Nov 16 11:50:13.304965 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started NTP client/server. Nov 16 11:50:13.366379 ropatil1611az-gs6zj-worker-westus-j4ngg groupadd[1600]: group added to /etc/group: name=containers, GID=995 Nov 16 11:50:13.388000 ropatil1611az-gs6zj-worker-westus-j4ngg groupadd[1600]: group added to /etc/gshadow: name=containers Nov 16 11:50:13.439370 ropatil1611az-gs6zj-worker-westus-j4ngg sssd[sssd][1561]: Starting up Nov 16 11:50:13.481547 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sshd-keygen@rsa.service: Succeeded. Nov 16 11:50:13.482259 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started OpenSSH rsa Server Key Generation. Nov 16 11:50:13.487546 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sshd-keygen@rsa.service: Consumed 292ms CPU time Nov 16 11:50:13.487881 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target sshd-keygen.target. Nov 16 11:50:13.493627 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Generate SSH keys snippet for display via console-login-helper-messages... Nov 16 11:50:13.545289 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sssd.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED Nov 16 11:50:13.545489 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sssd.service: Failed with result 'exit-code'. Nov 16 11:50:13.546148 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Failed to start System Security Services Daemon. Nov 16 11:50:13.555681 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sssd.service: Consumed 30ms CPU time Nov 16 11:50:13.555982 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target User and Group Name Lookups. Nov 16 11:50:13.562114 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Login Service... Nov 16 11:50:13.616172 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Generate SSH keys snippet for display via console-login-helper-messages. Nov 16 11:50:13.661701 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-logind[1660]: New seat seat0. Nov 16 11:50:13.663624 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-logind[1660]: Watching system buttons on /dev/input/event0 (Power Button) Nov 16 11:50:13.663896 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-logind[1660]: Watching system buttons on /dev/input/event4 (AT Translated Set 2 keyboard) Nov 16 11:50:13.664045 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-logind[1660]: Watching system buttons on /dev/input/event1 (AT Translated Set 2 keyboard) Nov 16 11:50:13.664226 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sssd.service: Service RestartSec=100ms expired, scheduling restart. Nov 16 11:50:13.664413 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sssd.service: Scheduled restart job, restart counter is at 1. Nov 16 11:50:13.665155 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped System Security Services Daemon. Nov 16 11:50:13.673223 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sssd.service: Consumed 0 CPU time Nov 16 11:50:13.674421 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting System Security Services Daemon... Nov 16 11:50:13.680486 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Login Service. Nov 16 11:50:13.780198 ropatil1611az-gs6zj-worker-westus-j4ngg sssd[sssd][1675]: Starting up Nov 16 11:50:13.781051 ropatil1611az-gs6zj-worker-westus-j4ngg groupadd[1600]: new group: name=containers, GID=995 Nov 16 11:50:13.849255 ropatil1611az-gs6zj-worker-westus-j4ngg useradd[1680]: new user: name=containers, UID=993, GID=995, home=/var/home/containers, shell=/sbin/nologin Nov 16 11:50:13.857358 ropatil1611az-gs6zj-worker-westus-j4ngg sssd[be[implicit_files]][1678]: Starting up Nov 16 11:50:13.962382 ropatil1611az-gs6zj-worker-westus-j4ngg sssd[nss][1682]: Starting up Nov 16 11:50:13.989934 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: console-login-helper-messages-issuegen.service: Succeeded. Nov 16 11:50:13.990680 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Generate console-login-helper-messages issue snippet. Nov 16 11:50:13.997909 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: console-login-helper-messages-issuegen.service: Consumed 19ms CPU time Nov 16 11:50:14.112722 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started System Security Services Daemon. Nov 16 11:50:14.414316 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1606]: /etc/openvswitch/conf.db does not exist ... (warning). Nov 16 11:50:14.474148 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1606]: Creating empty database /etc/openvswitch/conf.db. Nov 16 11:50:14.495064 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-subid.service: Succeeded. Nov 16 11:50:14.495832 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Generation of shadow ID ranges for CRI-O. Nov 16 11:50:14.502341 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-subid.service: Consumed 157ms CPU time Nov 16 11:50:15.067650 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1606]: 2022-11-16T11:50:15Z|00001|dns_resolve|WARN|Failed to read /etc/resolv.conf: No such file or directory Nov 16 11:50:15.067620 ropatil1611az-gs6zj-worker-westus-j4ngg ovsdb-server[1715]: ovs|00001|dns_resolve|WARN|Failed to read /etc/resolv.conf: No such file or directory Nov 16 11:50:15.118900 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1606]: Starting ovsdb-server. Nov 16 11:50:15.651775 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vsctl[1722]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait -- init -- set Open_vSwitch . db-version=8.3.0 Nov 16 11:50:16.691580 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1606]: 2022-11-16T11:50:16Z|00001|dns_resolve|WARN|Failed to read /etc/resolv.conf: No such file or directory Nov 16 11:50:16.691576 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1724]: ovs|00001|dns_resolve|WARN|Failed to read /etc/resolv.conf: No such file or directory Nov 16 11:50:16.784555 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vsctl[1733]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait set Open_vSwitch . ovs-version=2.16.5 "external-ids:system-id=\"45649f20-b8cb-4065-81f0-c8f1463b1647\"" "external-ids:rundir=\"/var/run/openvswitch\"" "system-type=\"rhcos\"" "system-version=\"4.10\"" Nov 16 11:50:16.936043 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1606]: Configuring Open vSwitch system IDs. Nov 16 11:50:17.003768 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vsctl[1739]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait add Open_vSwitch . external-ids hostname=ropatil1611az-gs6zj-worker-westus-j4ngg Nov 16 11:50:17.410450 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1606]: Enabling remote OVSDB managers. Nov 16 11:50:17.411903 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Open vSwitch Database Unit. Nov 16 11:50:17.418873 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Open vSwitch Delete Transient Ports... Nov 16 11:50:17.443521 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Open vSwitch Delete Transient Ports. Nov 16 11:50:17.450503 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Open vSwitch Forwarding Unit... Nov 16 11:50:17.672697 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: openvswitch: Open vSwitch switching datapath Nov 16 11:50:17.673359 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1758]: Inserting openvswitch module. Nov 16 11:50:17.685017 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1758]: 2022-11-16T11:50:17Z|00001|dns_resolve|WARN|Failed to read /etc/resolv.conf: No such file or directory Nov 16 11:50:17.684972 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1802]: ovs|00001|dns_resolve|WARN|Failed to read /etc/resolv.conf: No such file or directory Nov 16 11:50:20.065248 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1758]: Starting ovs-vswitchd. Nov 16 11:50:20.077046 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vsctl[1812]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait add Open_vSwitch . external-ids hostname=ropatil1611az-gs6zj-worker-westus-j4ngg Nov 16 11:50:20.077606 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1758]: Enabling remote OVSDB managers. Nov 16 11:50:20.085604 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Open vSwitch Forwarding Unit. Nov 16 11:50:20.092105 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Open vSwitch... Nov 16 11:50:20.098131 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Open vSwitch. Nov 16 11:50:20.103852 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Network Manager... Nov 16 11:50:20.565903 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.5655] NetworkManager (version 1.30.0-16.el8_4) is starting... (for the first time) Nov 16 11:50:20.565946 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.5659] Read config: /etc/NetworkManager/NetworkManager.conf (lib: 10-disable-default-plugins.conf, 20-client-id-from-mac.conf) (etc: 20-keyfiles.conf, sdn.conf) Nov 16 11:50:20.580779 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.5807] bus-manager: acquired D-Bus service "org.freedesktop.NetworkManager" Nov 16 11:50:20.580780 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Network Manager. Nov 16 11:50:20.586368 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Network. Nov 16 11:50:20.593558 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting OpenSSH server daemon... Nov 16 11:50:20.598283 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Afterburn (Check In)... Nov 16 11:50:20.603849 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Permit User Sessions... Nov 16 11:50:20.609170 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Network Manager Wait Online... Nov 16 11:50:20.614746 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Permit User Sessions. Nov 16 11:50:20.621523 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Serial Getty on ttyS0. Nov 16 11:50:20.628073 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Getty on tty1. Nov 16 11:50:20.633466 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Login Prompts. Nov 16 11:50:20.696654 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.6966] manager[0x55bcdbb71070]: monitoring kernel firmware directory '/lib/firmware'. Nov 16 11:50:20.713369 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1555]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.13' (uid=0 pid=1816 comm="/usr/sbin/NetworkManager --no-daemon " label="system_u:system_r:NetworkManager_t:s0") Nov 16 11:50:20.731852 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Hostname Service... Nov 16 11:50:20.734496 ropatil1611az-gs6zj-worker-westus-j4ngg sshd[1819]: Server listening on 0.0.0.0 port 22. Nov 16 11:50:20.736109 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started OpenSSH server daemon. Nov 16 11:50:20.734543 ropatil1611az-gs6zj-worker-westus-j4ngg sshd[1819]: Server listening on :: port 22. Nov 16 11:50:20.838923 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1555]: [system] Successfully activated service 'org.freedesktop.hostname1' Nov 16 11:50:20.839192 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Hostname Service. Nov 16 11:50:20.841582 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.8415] hostname: hostname: using hostnamed Nov 16 11:50:20.841599 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.8415] hostname: hostname changed from (none) to "ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:50:20.842093 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.8420] dns-mgr[0x55bcdbb63170]: init: dns=default,systemd-resolved rc-manager=symlink Nov 16 11:50:20.932241 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.9322] Loaded device plugin: NMOvsFactory (/usr/lib64/NetworkManager/1.30.0-16.el8_4/libnm-device-plugin-ovs.so) Nov 16 11:50:20.950905 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.9508] Loaded device plugin: NMTeamFactory (/usr/lib64/NetworkManager/1.30.0-16.el8_4/libnm-device-plugin-team.so) Nov 16 11:50:20.951007 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.9509] manager: rfkill: Wi-Fi enabled by radio killswitch; enabled by state file Nov 16 11:50:20.951111 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.9511] manager: rfkill: WWAN enabled by radio killswitch; enabled by state file Nov 16 11:50:20.951180 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.9511] manager: Networking is enabled by state file Nov 16 11:50:20.951310 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.9513] dhcp-init: Using DHCP client 'internal' Nov 16 11:50:20.951380 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.9513] settings: Loaded settings plugin: keyfile (internal) Nov 16 11:50:20.951396 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1555]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested by ':1.13' (uid=0 pid=1816 comm="/usr/sbin/NetworkManager --no-daemon " label="system_u:system_r:NetworkManager_t:s0") Nov 16 11:50:20.955181 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Network Manager Script Dispatcher Service... Nov 16 11:50:20.977397 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1555]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher' Nov 16 11:50:20.977746 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Network Manager Script Dispatcher Service. Nov 16 11:50:20.993923 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599420.9939] settings: Loaded settings plugin: ifcfg-rh ("/usr/lib64/NetworkManager/1.30.0-16.el8_4/libnm-settings-plugin-ifcfg-rh.so") Nov 16 11:50:21.018858 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.0188] device (lo): carrier: link connected Nov 16 11:50:21.019165 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.0191] manager: (lo): new Generic device (/org/freedesktop/NetworkManager/Devices/1) Nov 16 11:50:21.020283 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.0202] manager: (eth0): new Ethernet device (/org/freedesktop/NetworkManager/Devices/2) Nov 16 11:50:21.078159 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.0781] settings: (eth0): created default wired connection 'Wired connection 1' Nov 16 11:50:21.078715 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.0786] device (eth0): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external') Nov 16 11:50:21.084658 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 11:50:21.085148 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.0851] device (eth0): carrier: link connected Nov 16 11:50:21.104269 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1042] device (eth0): state change: unavailable -> disconnected (reason 'none', sys-iface-state: 'managed') Nov 16 11:50:21.106128 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1061] policy: auto-activating connection 'Wired connection 1' (28cb2a02-f1b7-3f80-a052-46e4e7c562e5) Nov 16 11:50:21.106832 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1068] device (eth0): Activation: starting connection 'Wired connection 1' (28cb2a02-f1b7-3f80-a052-46e4e7c562e5) Nov 16 11:50:21.106954 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1069] device (eth0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed') Nov 16 11:50:21.107326 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1073] manager: NetworkManager state is now CONNECTING Nov 16 11:50:21.107667 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1076] device (eth0): state change: prepare -> config (reason 'none', sys-iface-state: 'managed') Nov 16 11:50:21.108110 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1080] device (eth0): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed') Nov 16 11:50:21.120781 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1207] dhcp4 (eth0): activation: beginning transaction (timeout in 45 seconds) Nov 16 11:50:21.146125 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1461] dhcp4 (eth0): state changed unknown -> bound, address=10.0.1.4 Nov 16 11:50:21.147359 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1473] device (eth0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'managed') Nov 16 11:50:21.154554 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1545] device (eth0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'managed') Nov 16 11:50:21.154750 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1554]: time="2022-11-16 11:50:21.154393500Z" level=info msg="Starting CRI-O, version: 1.23.3-17.rhaos4.10.git016b1ca.el8, git: ()" Nov 16 11:50:21.154827 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1548] device (eth0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'managed') Nov 16 11:50:21.155270 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1552] manager: NetworkManager state is now CONNECTED_LOCAL Nov 16 11:50:21.156240 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1562] manager: NetworkManager state is now CONNECTED_SITE Nov 16 11:50:21.156349 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1563] policy: set 'Wired connection 1' (eth0) as default for IPv4 routing and DNS Nov 16 11:50:21.156965 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1569] device (eth0): Activation: successful, device activated. Nov 16 11:50:21.157627 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1576] manager: NetworkManager state is now CONNECTED_GLOBAL Nov 16 11:50:21.158025 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1555]: [system] Activating via systemd: service name='org.freedesktop.resolve1' unit='dbus-org.freedesktop.resolve1.service' requested by ':1.13' (uid=0 pid=1816 comm="/usr/sbin/NetworkManager --no-daemon " label="system_u:system_r:NetworkManager_t:s0") Nov 16 11:50:21.158259 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599421.1582] manager: startup complete Nov 16 11:50:21.160724 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1555]: [system] Activation via systemd failed for unit 'dbus-org.freedesktop.resolve1.service': Unit dbus-org.freedesktop.resolve1.service not found. Nov 16 11:50:21.195086 ropatil1611az-gs6zj-worker-westus-j4ngg nm-dispatcher[1832]: Error: Device '' not found. Nov 16 11:50:21.198196 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Network Manager Wait Online. Nov 16 11:50:21.204333 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Wait for a non-localhost hostname... Nov 16 11:50:21.207427 ropatil1611az-gs6zj-worker-westus-j4ngg mco-hostname[1855]: waiting for non-localhost hostname to be assigned Nov 16 11:50:21.208735 ropatil1611az-gs6zj-worker-westus-j4ngg mco-hostname[1855]: node identified as ropatil1611az-gs6zj-worker-westus-j4ngg Nov 16 11:50:21.210692 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Wait for a non-localhost hostname. Nov 16 11:50:21.217291 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Network is Online. Nov 16 11:50:21.224096 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting NFS status monitor for NFSv2/3 locking.... Nov 16 11:50:21.230830 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Dynamically sets the system reserved for the kubelet... Nov 16 11:50:21.231092 ropatil1611az-gs6zj-worker-westus-j4ngg nm-dispatcher[1832]: Error: Device '' not found. Nov 16 11:50:21.241799 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Dynamically sets the system reserved for the kubelet. Nov 16 11:50:21.251106 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: iscsi.service: Unit cannot be reloaded because it is inactive. Nov 16 11:50:21.768968 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting RPC Bind... Nov 16 11:50:21.833068 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Generate console-login-helper-messages issue snippet... Nov 16 11:50:21.880862 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started RPC Bind. Nov 16 11:50:21.881623 ropatil1611az-gs6zj-worker-westus-j4ngg rpc.statd[1902]: Version 2.3.3 starting Nov 16 11:50:21.881656 ropatil1611az-gs6zj-worker-westus-j4ngg rpc.statd[1902]: Flags: TI-RPC Nov 16 11:50:21.884964 ropatil1611az-gs6zj-worker-westus-j4ngg rpc.statd[1902]: Initializing NSM state Nov 16 11:50:21.888357 ropatil1611az-gs6zj-worker-westus-j4ngg nm-dispatcher[1832]: Error: Device '' not found. Nov 16 11:50:21.988684 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started NFS status monitor for NFSv2/3 locking.. Nov 16 11:50:22.162093 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-metacopy\x2dcheck1660058635-merged.mount: Succeeded. Nov 16 11:50:22.239430 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-opaque\x2dbug\x2dcheck1726331514-merged.mount: Succeeded. Nov 16 11:50:22.272887 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1554]: time="2022-11-16 11:50:22.272550500Z" level=info msg="Checking whether cri-o should wipe containers: version file /var/run/crio/version not found: open /var/run/crio/version: no such file or directory" Nov 16 11:50:22.272887 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1554]: time="2022-11-16 11:50:22.272882700Z" level=info msg="version file /var/lib/crio/version not found: open /var/lib/crio/version: no such file or directory: triggering wipe of images" Nov 16 11:50:22.277096 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-wipe.service: Succeeded. Nov 16 11:50:22.277760 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started CRI-O Auto Update Script. Nov 16 11:50:22.283278 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-wipe.service: Consumed 352ms CPU time Nov 16 11:50:22.285159 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Machine Config Daemon Pull... Nov 16 11:50:22.858820 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: console-login-helper-messages-issuegen.service: Succeeded. Nov 16 11:50:22.859474 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Generate console-login-helper-messages issue snippet. Nov 16 11:50:22.865975 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: console-login-helper-messages-issuegen.service: Consumed 22ms CPU time Nov 16 11:50:32.567298 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: NetworkManager-dispatcher.service: Succeeded. Nov 16 11:50:32.568310 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: NetworkManager-dispatcher.service: Consumed 185ms CPU time Nov 16 11:50:36.808196 ropatil1611az-gs6zj-worker-westus-j4ngg chronyd[1584]: Selected source PHC0 Nov 16 11:50:36.808398 ropatil1611az-gs6zj-worker-westus-j4ngg chronyd[1584]: System clock TAI offset set to 37 seconds Nov 16 11:50:41.601091 ropatil1611az-gs6zj-worker-westus-j4ngg sh[1908]: 733c1343f71786d137b938b4071bba9cc7fb64b301e1464e63f775ec8d1531c7 Nov 16 11:50:41.699143 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay.mount: Succeeded. Nov 16 11:50:41.699882 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay.mount: Consumed 0 CPU time Nov 16 11:50:41.961719 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: overlayfs: unrecognized mount option "volatile" or missing value Nov 16 11:50:43.411442 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice machine.slice. Nov 16 11:50:43.412156 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container ec0aff5ffc7a40081c5c08b9ea08d38129dd141541de4b8c75d05ebed2424fc8. Nov 16 11:50:43.431717 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: cgroup: cgroup: disabling cgroup2 socket matching due to net_prio or net_cls activation Nov 16 11:50:43.447987 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:50:43.595391 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: libpod-ec0aff5ffc7a40081c5c08b9ea08d38129dd141541de4b8c75d05ebed2424fc8.scope: Succeeded. Nov 16 11:50:43.596253 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: libpod-ec0aff5ffc7a40081c5c08b9ea08d38129dd141541de4b8c75d05ebed2424fc8.scope: Consumed 68ms CPU time Nov 16 11:50:43.687434 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay\x2dcontainers-ec0aff5ffc7a40081c5c08b9ea08d38129dd141541de4b8c75d05ebed2424fc8-userdata-shm.mount: Succeeded. Nov 16 11:50:43.697337 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-9996d301501eed04b002fefb42c8a20f22b1270a658055d490eaa24b252da473-merged.mount: Succeeded. Nov 16 11:50:50.579524 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1820]: Nov 16 11:50:50.519 WARN Failed to get fabric address from DHCP: maximum number of retries (60) reached Nov 16 11:50:50.579524 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1820]: Nov 16 11:50:50.579 INFO using fallback address Nov 16 11:50:50.750038 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1820]: Nov 16 11:50:50.749 INFO Fetching http://168.63.129.16/?comp=versions: Attempt #1 Nov 16 11:50:50.854747 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1820]: Nov 16 11:50:50.854 INFO Fetch successful Nov 16 11:50:50.862176 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-hostnamed.service: Succeeded. Nov 16 11:50:50.862965 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-hostnamed.service: Consumed 50ms CPU time Nov 16 11:50:50.876196 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay.mount: Succeeded. Nov 16 11:50:50.890582 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Machine Config Daemon Pull. Nov 16 11:50:50.893031 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Machine Config Daemon Firstboot... Nov 16 11:50:50.893164 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1820]: Nov 16 11:50:50.891 INFO Fetching http://168.63.129.16/machine/?comp=goalstate: Attempt #1 Nov 16 11:50:50.893516 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1820]: Nov 16 11:50:50.893 INFO Fetch successful Nov 16 11:50:50.897847 ropatil1611az-gs6zj-worker-westus-j4ngg sh[2210]: sed: can't read /etc/yum.repos.d/*.repo: No such file or directory Nov 16 11:50:50.913780 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay.mount: Succeeded. Nov 16 11:50:50.928455 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:50:50.928394 2213 update.go:1950] Running: systemctl start rpm-ostreed Nov 16 11:50:50.988329 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting rpm-ostree System Management Daemon... Nov 16 11:50:50.994792 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1820]: Nov 16 11:50:50.994 INFO Posting http://168.63.129.16/machine/?comp=health: Attempt #1 Nov 16 11:50:51.002185 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Afterburn (Check In). Nov 16 11:50:52.808529 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: Reading config file '/etc/rpm-ostreed.conf' Nov 16 11:50:52.809462 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1555]: [system] Activating via systemd: service name='org.freedesktop.PolicyKit1' unit='polkit.service' requested by ':1.42' (uid=0 pid=2226 comm="/usr/bin/rpm-ostree start-daemon " label="system_u:system_r:install_t:s0") Nov 16 11:50:52.812238 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Authorization Manager... Nov 16 11:50:53.304743 ropatil1611az-gs6zj-worker-westus-j4ngg polkitd[2230]: Started polkitd version 0.115 Nov 16 11:50:54.567246 ropatil1611az-gs6zj-worker-westus-j4ngg polkitd[2230]: Loading rules from directory /etc/polkit-1/rules.d Nov 16 11:50:54.567377 ropatil1611az-gs6zj-worker-westus-j4ngg polkitd[2230]: Loading rules from directory /usr/share/polkit-1/rules.d Nov 16 11:50:54.659758 ropatil1611az-gs6zj-worker-westus-j4ngg polkitd[2230]: Finished loading, compiling and executing 3 rules Nov 16 11:50:54.660796 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1555]: [system] Successfully activated service 'org.freedesktop.PolicyKit1' Nov 16 11:50:54.661175 ropatil1611az-gs6zj-worker-westus-j4ngg polkitd[2230]: Acquired the name org.freedesktop.PolicyKit1 on the system bus Nov 16 11:50:54.661291 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Authorization Manager. Nov 16 11:50:54.842984 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_balloon: Max. dynamic memory size: 16384 MB Nov 16 11:50:54.898003 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: In idle state; will auto-exit in 61 seconds Nov 16 11:50:54.898888 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started rpm-ostree System Management Daemon. Nov 16 11:50:54.900142 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:50:54.899897 2213 rpm-ostree.go:325] Running captured: rpm-ostree status --json Nov 16 11:50:55.004260 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: client(id:cli dbus:1.45 unit:machine-config-daemon-firstboot.service uid:0) added; new total=1 Nov 16 11:50:55.015587 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: client(id:cli dbus:1.45 unit:machine-config-daemon-firstboot.service uid:0) vanished; remaining=0 Nov 16 11:50:55.015608 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: In idle state; will auto-exit in 61 seconds Nov 16 11:50:55.310200 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:50:55.310165 2213 rpm-ostree.go:325] Running captured: rpm-ostree status --json Nov 16 11:50:55.339782 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: client(id:cli dbus:1.46 unit:machine-config-daemon-firstboot.service uid:0) added; new total=1 Nov 16 11:50:55.350236 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: client(id:cli dbus:1.46 unit:machine-config-daemon-firstboot.service uid:0) vanished; remaining=0 Nov 16 11:50:55.350254 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: In idle state; will auto-exit in 64 seconds Nov 16 11:50:55.352493 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:50:55.352457 2213 daemon.go:223] Booted osImageURL: (410.84.202210040010-0) Nov 16 11:50:55.354400 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:50:55.354335 2213 update.go:1987] Adding SIGTERM protection Nov 16 11:50:55.354780 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:50:55.354753 2213 update.go:512] Checking Reconcilable for config mco-empty-mc to rendered-worker-8b35090bacf1ad998c4e01e0fd1046d1 Nov 16 11:50:55.355717 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:50:55.355608 2213 update.go:1965] Starting update from mco-empty-mc to rendered-worker-8b35090bacf1ad998c4e01e0fd1046d1: &{osUpdate:true kargs:false fips:false passwd:false files:false units:false kernelType:false extensions:false} Nov 16 11:50:55.444747 ropatil1611az-gs6zj-worker-westus-j4ngg root[2251]: machine-config-daemon[2213]: Starting update from mco-empty-mc to rendered-worker-8b35090bacf1ad998c4e01e0fd1046d1: &{osUpdate:true kargs:false fips:false passwd:false files:false units:false kernelType:false extensions:false} Nov 16 11:50:55.445100 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:50:55.445074 2213 update.go:1131] Updating files Nov 16 11:50:55.445100 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:50:55.445103 2213 update.go:1196] Deleting stale data Nov 16 11:50:55.445316 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:50:55.445297 2213 run.go:18] Running: nice -- ionice -c 3 oc image extract --path /:/run/mco-machine-os-content/os-content-3749548381 --registry-config /var/lib/kubelet/config.json quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4db499051ae7bf9fe9f1ee1963add35d18316956b4f6b040b5ffb85c2e7ff526 Nov 16 11:51:20.525710 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:51:20.525568 2213 update.go:1827] Updating OS to quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4db499051ae7bf9fe9f1ee1963add35d18316956b4f6b040b5ffb85c2e7ff526 Nov 16 11:51:20.525710 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:51:20.525617 2213 rpm-ostree.go:325] Running captured: rpm-ostree status --json Nov 16 11:51:20.556480 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: client(id:cli dbus:1.47 unit:machine-config-daemon-firstboot.service uid:0) added; new total=1 Nov 16 11:51:20.567752 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: client(id:cli dbus:1.47 unit:machine-config-daemon-firstboot.service uid:0) vanished; remaining=0 Nov 16 11:51:20.567773 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: In idle state; will auto-exit in 61 seconds Nov 16 11:51:20.569757 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:51:20.569700 2213 rpm-ostree.go:238] Current origin is not custom Nov 16 11:51:22.194335 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:51:22.194258 2213 rpm-ostree.go:265] Pivoting to: 410.84.202211032138-0 (8d68051abeecf4991283052b4e6d38a5bad50e3c8e7b678973d5c39d30145ab0) Nov 16 11:51:22.194335 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:51:22.194285 2213 rpm-ostree.go:297] Executing rebase from repo path /run/mco-machine-os-content/os-content-3749548381/srv/repo with customImageURL pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4db499051ae7bf9fe9f1ee1963add35d18316956b4f6b040b5ffb85c2e7ff526 and checksum 8d68051abeecf4991283052b4e6d38a5bad50e3c8e7b678973d5c39d30145ab0 Nov 16 11:51:22.194335 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:51:22.194312 2213 update.go:1950] Running: rpm-ostree rebase --experimental /run/mco-machine-os-content/os-content-3749548381/srv/repo:8d68051abeecf4991283052b4e6d38a5bad50e3c8e7b678973d5c39d30145ab0 --custom-origin-url pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4db499051ae7bf9fe9f1ee1963add35d18316956b4f6b040b5ffb85c2e7ff526 --custom-origin-description Managed by machine-config-operator Nov 16 11:51:22.225516 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: client(id:cli dbus:1.48 unit:machine-config-daemon-firstboot.service uid:0) added; new total=1 Nov 16 11:51:22.494404 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: Locked sysroot Nov 16 11:51:22.494761 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: EXT4-fs (sdb3): re-mounted. Opts: Nov 16 11:51:22.494526 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: Initiated txn UpdateDeployment for client(id:cli dbus:1.48 unit:machine-config-daemon-firstboot.service uid:0): /org/projectatomic/rpmostree1/rhcos Nov 16 11:52:08.780317 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: 886 metadata, 2928 content objects imported; 992.6 MB content written Nov 16 11:52:08.780317 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: Writing objects: 59...done Nov 16 11:52:08.792476 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started OSTree Finalize Staged Deployment. Nov 16 11:52:14.556846 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: Staging deployment...done Nov 16 11:52:14.557683 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: Created new deployment /ostree/deploy/rhcos/deploy/8d68051abeecf4991283052b4e6d38a5bad50e3c8e7b678973d5c39d30145ab0.0 Nov 16 11:52:14.611382 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: sanitycheck(/usr/bin/true) successful Nov 16 11:52:23.336732 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: Txn UpdateDeployment on /org/projectatomic/rpmostree1/rhcos successful Nov 16 11:52:23.338949 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: Unlocked sysroot Nov 16 11:52:23.746792 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: Upgraded: Nov 16 11:52:23.746792 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: bind-libs 32:9.11.26-4.el8_4 -> 32:9.11.26-4.el8_4.1 Nov 16 11:52:23.746792 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: bind-libs-lite 32:9.11.26-4.el8_4 -> 32:9.11.26-4.el8_4.1 Nov 16 11:52:23.746792 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: bind-license 32:9.11.26-4.el8_4 -> 32:9.11.26-4.el8_4.1 Nov 16 11:52:23.746792 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: bind-utils 32:9.11.26-4.el8_4 -> 32:9.11.26-4.el8_4.1 Nov 16 11:52:23.746792 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: ca-certificates 2021.2.50-80.0.el8_4 -> 2022.2.54-80.2.el8_4 Nov 16 11:52:23.746792 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: cri-o 1.23.3-17.rhaos4.10.git016b1ca.el8 -> 1.23.3-20.rhaos4.10.git89344de.el8 Nov 16 11:52:23.746792 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: device-mapper-multipath 0.8.4-10.el8_4.3 -> 0.8.4-10.el8_4.4 Nov 16 11:52:23.746792 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: device-mapper-multipath-libs 0.8.4-10.el8_4.3 -> 0.8.4-10.el8_4.4 Nov 16 11:52:23.746792 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: expat 2.2.5-4.el8_4.3 -> 2.2.5-4.el8_4.4 Nov 16 11:52:23.746792 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: kernel 4.18.0-305.62.1.el8_4 -> 4.18.0-305.65.1.el8_4 Nov 16 11:52:23.746792 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: kernel-core 4.18.0-305.62.1.el8_4 -> 4.18.0-305.65.1.el8_4 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: kernel-modules 4.18.0-305.62.1.el8_4 -> 4.18.0-305.65.1.el8_4 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: kernel-modules-extra 4.18.0-305.62.1.el8_4 -> 4.18.0-305.65.1.el8_4 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: kpartx 0.8.4-10.el8_4.3 -> 0.8.4-10.el8_4.4 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: openshift-clients 4.10.0-202209291217.p0.g442535c.assembly.stream.el8 -> 4.10.0-202210242129.p0.g8df677d.assembly.stream.el8 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: openshift-hyperkube 4.10.0-202209161727.p0.g8471591.assembly.stream.el8 -> 4.10.0-202210242129.p0.g7566c4d.assembly.stream.el8 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: openvswitch2.16 2.16.0-89.3.el8fdp -> 2.16.0-103.el8fdp Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: python3-bind 32:9.11.26-4.el8_4 -> 32:9.11.26-4.el8_4.1 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: rpm 4.14.3-14.el8_4.2 -> 4.14.3-14.el8_4.3 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: rpm-libs 4.14.3-14.el8_4.2 -> 4.14.3-14.el8_4.3 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: rpm-plugin-selinux 4.14.3-14.el8_4.2 -> 4.14.3-14.el8_4.3 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: stalld 1.15-2.el8_4 -> 1.15-3.el8_4 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: systemd 239-45.el8_4.12 -> 239-45.el8_4.14 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: systemd-journal-remote 239-45.el8_4.12 -> 239-45.el8_4.14 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: systemd-libs 239-45.el8_4.12 -> 239-45.el8_4.14 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: systemd-pam 239-45.el8_4.12 -> 239-45.el8_4.14 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: systemd-udev 239-45.el8_4.12 -> 239-45.el8_4.14 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: tzdata 2022c-1.el8 -> 2022f-1.el8 Nov 16 11:52:23.747581 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: Run "systemctl reboot" to start a reboot Nov 16 11:52:23.748401 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: client(id:cli dbus:1.48 unit:machine-config-daemon-firstboot.service uid:0) vanished; remaining=0 Nov 16 11:52:23.748443 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[2226]: In idle state; will auto-exit in 62 seconds Nov 16 11:52:24.188782 ropatil1611az-gs6zj-worker-westus-j4ngg logger[2399]: rendered-worker-8b35090bacf1ad998c4e01e0fd1046d1 Nov 16 11:52:24.189407 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:52:24.189170 2213 update.go:1965] Rebooting node Nov 16 11:52:24.193268 ropatil1611az-gs6zj-worker-westus-j4ngg root[2400]: machine-config-daemon[2213]: Rebooting node Nov 16 11:52:24.193626 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:52:24.193594 2213 update.go:1995] Removing SIGTERM protection Nov 16 11:52:24.193741 ropatil1611az-gs6zj-worker-westus-j4ngg machine-config-daemon[2213]: I1116 11:52:24.193705 2213 update.go:1965] initiating reboot: Completing firstboot provisioning to rendered-worker-8b35090bacf1ad998c4e01e0fd1046d1 Nov 16 11:52:24.196176 ropatil1611az-gs6zj-worker-westus-j4ngg root[2401]: machine-config-daemon[2213]: initiating reboot: Completing firstboot provisioning to rendered-worker-8b35090bacf1ad998c4e01e0fd1046d1 Nov 16 11:52:24.236561 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started machine-config-daemon: Completing firstboot provisioning to rendered-worker-8b35090bacf1ad998c4e01e0fd1046d1. Nov 16 11:52:24.274305 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-logind[1660]: System is rebooting. Nov 16 11:52:24.279312 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice machine.slice. Nov 16 11:52:24.284506 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: machine.slice: Consumed 68ms CPU time Nov 16 11:52:24.284968 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping rpm-ostree System Management Daemon... Nov 16 11:52:24.288605 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice system-sshd\x2dkeygen.slice. Nov 16 11:52:24.288905 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: system-sshd\x2dkeygen.slice: Consumed 335ms CPU time Nov 16 11:52:24.288947 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Timers. Nov 16 11:52:24.290046 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-tmpfiles-clean.timer: Succeeded. Nov 16 11:52:24.290066 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Daily Cleanup of Temporary Directories. Nov 16 11:52:24.291032 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: unbound-anchor.timer: Succeeded. Nov 16 11:52:24.291059 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped daily update of the root trust anchor for DNSSEC. Nov 16 11:52:24.292249 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Authorization Manager... Nov 16 11:52:24.293409 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Restore /run/initramfs on shutdown... Nov 16 11:52:24.294124 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping machine-config-daemon: Completing firstboot provisioning to rendered-worker-8b35090bacf1ad998c4e01e0fd1046d1... Nov 16 11:52:24.294906 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: logrotate.timer: Succeeded. Nov 16 11:52:24.294942 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Daily rotation of log files. Nov 16 11:52:24.295980 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: lvm2-lvmpolld.socket: Succeeded. Nov 16 11:52:24.296710 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Closed LVM2 poll daemon socket. Nov 16 11:52:24.296819 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: lvm2-lvmpolld.socket: Consumed 0 CPU time Nov 16 11:52:24.296867 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Remote Encrypted Volumes. Nov 16 11:52:24.297977 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-ignition-firstboot-complete.service: Succeeded. Nov 16 11:52:24.298567 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped CoreOS Mark Ignition Boot Complete. Nov 16 11:52:24.298918 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-ignition-firstboot-complete.service: Consumed 0 CPU time Nov 16 11:52:24.299147 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-update-ca-trust.service: Succeeded. Nov 16 11:52:24.299792 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Run update-ca-trust. Nov 16 11:52:24.300800 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-update-ca-trust.service: Consumed 0 CPU time Nov 16 11:52:24.300828 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target RPC Port Mapper. Nov 16 11:52:24.302061 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: afterburn-checkin.service: Succeeded. Nov 16 11:52:24.302726 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Afterburn (Check In). Nov 16 11:52:24.302832 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: afterburn-checkin.service: Consumed 0 CPU time Nov 16 11:52:24.302855 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Synchronize afterburn-sshkeys@.service template instances. Nov 16 11:52:24.303798 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Login Prompts. Nov 16 11:52:24.305020 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Serial Getty on ttyS0... Nov 16 11:52:24.305910 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Getty on tty1... Nov 16 11:52:24.312830 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping NTP client/server... Nov 16 11:52:24.312873 ropatil1611az-gs6zj-worker-westus-j4ngg chronyd[1584]: chronyd exiting Nov 16 11:52:24.315949 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping irqbalance daemon... Nov 16 11:52:24.318681 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Login Service... Nov 16 11:52:24.322908 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping OpenSSH server daemon... Nov 16 11:52:24.323216 ropatil1611az-gs6zj-worker-westus-j4ngg sshd[1819]: Received signal 15; terminating. Nov 16 11:52:24.327250 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-generate-iscsi-initiatorname.service: Succeeded. Nov 16 11:52:24.327854 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped CoreOS Generate iSCSI Initiator Name. Nov 16 11:52:24.334536 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-generate-iscsi-initiatorname.service: Consumed 0 CPU time Nov 16 11:52:24.334749 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: console-login-helper-messages-gensnippet-ssh-keys.service: Succeeded. Nov 16 11:52:24.335348 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Generate SSH keys snippet for display via console-login-helper-messages. Nov 16 11:52:24.342912 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: console-login-helper-messages-gensnippet-ssh-keys.service: Consumed 0 CPU time Nov 16 11:52:24.343132 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping NFS status monitor for NFSv2/3 locking.... Nov 16 11:52:24.347361 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubelet-auto-node-size.service: Succeeded. Nov 16 11:52:24.348038 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Dynamically sets the system reserved for the kubelet. Nov 16 11:52:24.352342 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubelet-auto-node-size.service: Consumed 0 CPU time Nov 16 11:52:24.355174 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: irqbalance.service: Succeeded. Nov 16 11:52:24.355794 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped irqbalance daemon. Nov 16 11:52:24.358852 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: irqbalance.service: Consumed 10ms CPU time Nov 16 11:52:24.359260 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: chronyd.service: Succeeded. Nov 16 11:52:24.359848 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped NTP client/server. Nov 16 11:52:24.363124 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: chronyd.service: Consumed 42ms CPU time Nov 16 11:52:24.363675 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sshd.service: Succeeded. Nov 16 11:52:24.364458 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped OpenSSH server daemon. Nov 16 11:52:24.369605 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sshd.service: Consumed 14ms CPU time Nov 16 11:52:24.373074 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: serial-getty@ttyS0.service: Succeeded. Nov 16 11:52:24.373667 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Serial Getty on ttyS0. Nov 16 11:52:24.377163 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: serial-getty@ttyS0.service: Consumed 30ms CPU time Nov 16 11:52:24.379464 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: getty@tty1.service: Succeeded. Nov 16 11:52:24.380207 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Getty on tty1. Nov 16 11:52:24.383034 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: getty@tty1.service: Consumed 6ms CPU time Nov 16 11:52:24.383597 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: rpc-statd.service: Succeeded. Nov 16 11:52:24.384234 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped NFS status monitor for NFSv2/3 locking.. Nov 16 11:52:24.388736 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: rpc-statd.service: Consumed 32ms CPU time Nov 16 11:52:24.389106 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: machine-config-daemon-firstboot.service: Main process exited, code=killed, status=15/TERM Nov 16 11:52:24.389206 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: machine-config-daemon-firstboot.service: Failed with result 'signal'. Nov 16 11:52:24.389819 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Machine Config Daemon Firstboot. Nov 16 11:52:24.393317 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: machine-config-daemon-firstboot.service: Consumed 17.986s CPU time Nov 16 11:52:24.393699 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-logind.service: Succeeded. Nov 16 11:52:24.394336 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Login Service. Nov 16 11:52:24.397584 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-logind.service: Consumed 60ms CPU time Nov 16 11:52:24.398051 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: rpm-ostreed.service: Succeeded. Nov 16 11:52:24.398665 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped rpm-ostree System Management Daemon. Nov 16 11:52:24.402532 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: rpm-ostreed.service: Consumed 9.083s CPU time Nov 16 11:52:24.403213 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: polkit.service: Succeeded. Nov 16 11:52:24.403857 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Authorization Manager. Nov 16 11:52:24.407456 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: polkit.service: Consumed 66ms CPU time Nov 16 11:52:24.408107 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: machine-config-daemon-reboot.service: Succeeded. Nov 16 11:52:24.408658 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped machine-config-daemon: Completing firstboot provisioning to rendered-worker-8b35090bacf1ad998c4e01e0fd1046d1. Nov 16 11:52:24.417572 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: machine-config-daemon-reboot.service: Consumed 17ms CPU time Nov 16 11:52:24.418248 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-shutdown.service: Succeeded. Nov 16 11:52:24.418915 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Restore /run/initramfs on shutdown. Nov 16 11:52:24.423328 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-shutdown.service: Consumed 2ms CPU time Nov 16 11:52:24.426503 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: machine-config-daemon-pull.service: Succeeded. Nov 16 11:52:24.427044 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Machine Config Daemon Pull. Nov 16 11:52:24.430653 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: machine-config-daemon-pull.service: Consumed 22.085s CPU time Nov 16 11:52:24.430717 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Host and Network Name Lookups. Nov 16 11:52:24.435296 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Network is Online. Nov 16 11:52:24.438987 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: NetworkManager-wait-online.service: Succeeded. Nov 16 11:52:24.439528 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Network Manager Wait Online. Nov 16 11:52:24.443249 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: NetworkManager-wait-online.service: Consumed 0 CPU time Nov 16 11:52:24.443429 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: node-valid-hostname.service: Succeeded. Nov 16 11:52:24.443967 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Wait for a non-localhost hostname. Nov 16 11:52:24.448278 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: node-valid-hostname.service: Consumed 0 CPU time Nov 16 11:52:24.448949 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice system-getty.slice. Nov 16 11:52:24.452296 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: system-getty.slice: Consumed 6ms CPU time Nov 16 11:52:24.452867 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Permit User Sessions... Nov 16 11:52:24.456785 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice system-serial\x2dgetty.slice. Nov 16 11:52:24.460750 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: system-serial\x2dgetty.slice: Consumed 30ms CPU time Nov 16 11:52:24.460780 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target sshd-keygen.target. Nov 16 11:52:24.465793 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-user-sessions.service: Succeeded. Nov 16 11:52:24.466421 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Permit User Sessions. Nov 16 11:52:24.469702 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-user-sessions.service: Consumed 12ms CPU time Nov 16 11:52:24.470270 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-check-ignition-config.service: Succeeded. Nov 16 11:52:24.470875 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Check if Ignition config is provided. Nov 16 11:52:24.474442 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-check-ignition-config.service: Consumed 0 CPU time Nov 16 11:52:24.474468 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Remote File Systems. Nov 16 11:52:24.478393 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target User and Group Name Lookups. Nov 16 11:52:24.483227 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Network. Nov 16 11:52:24.486020 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Network Manager... Nov 16 11:52:24.486339 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599544.4860] caught SIGTERM, shutting down normally. Nov 16 11:52:24.487386 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1803]: ovs|00009|memory|INFO|143708 kB peak resident set size after 126.8 seconds Nov 16 11:52:24.488945 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping System Security Services Daemon... Nov 16 11:52:24.490048 ropatil1611az-gs6zj-worker-westus-j4ngg sssd[nss][1682]: Shutting down (status = 0) Nov 16 11:52:24.501183 ropatil1611az-gs6zj-worker-westus-j4ngg sssd[be[implicit_files]][1678]: Shutting down (status = 0) Nov 16 11:52:24.501840 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599544.5017] dhcp4 (eth0): canceled DHCP transaction Nov 16 11:52:24.501880 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599544.5018] dhcp4 (eth0): state changed bound -> done Nov 16 11:52:24.501924 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599544.5019] device (eth0): DHCPv4: trying to acquire a new lease within 90 seconds Nov 16 11:52:24.502322 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599544.5023] manager: NetworkManager state is now CONNECTED_SITE Nov 16 11:52:24.502999 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1555]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested by ':1.13' (uid=0 pid=1816 comm="/usr/sbin/NetworkManager --no-daemon " label="system_u:system_r:NetworkManager_t:s0") Nov 16 11:52:24.503222 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1555]: [system] Activation via systemd failed for unit 'dbus-org.freedesktop.nm-dispatcher.service': Refusing activation, D-Bus is shutting down. Nov 16 11:52:24.508788 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sssd.service: Succeeded. Nov 16 11:52:24.509387 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped System Security Services Daemon. Nov 16 11:52:24.511869 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1816]: [1668599544.5118] exiting (success) Nov 16 11:52:24.516761 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sssd.service: Consumed 286ms CPU time Nov 16 11:52:24.533243 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: NetworkManager.service: Succeeded. Nov 16 11:52:24.533801 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Network Manager. Nov 16 11:52:24.540013 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: NetworkManager.service: Consumed 108ms CPU time Nov 16 11:52:24.541116 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping D-Bus System Message Bus... Nov 16 11:52:24.547328 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Open vSwitch... Nov 16 11:52:24.551632 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dbus.service: Succeeded. Nov 16 11:52:24.552238 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped D-Bus System Message Bus. Nov 16 11:52:24.558670 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dbus.service: Consumed 172ms CPU time Nov 16 11:52:24.559039 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: openvswitch.service: Succeeded. Nov 16 11:52:24.559550 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Open vSwitch. Nov 16 11:52:24.565229 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: openvswitch.service: Consumed 1ms CPU time Nov 16 11:52:24.566539 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Open vSwitch Forwarding Unit... Nov 16 11:52:24.599904 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[2432]: Exiting ovs-vswitchd (1803). Nov 16 11:52:24.701972 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ovs-vswitchd.service: Succeeded. Nov 16 11:52:24.702802 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Open vSwitch Forwarding Unit. Nov 16 11:52:24.710311 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ovs-vswitchd.service: Consumed 249ms CPU time Nov 16 11:52:24.711030 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ovs-delete-transient-ports.service: Succeeded. Nov 16 11:52:24.711606 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Open vSwitch Delete Transient Ports. Nov 16 11:52:24.718825 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ovs-delete-transient-ports.service: Consumed 0 CPU time Nov 16 11:52:24.719488 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Open vSwitch Database Unit... Nov 16 11:52:24.752456 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[2452]: Exiting ovsdb-server (1721). Nov 16 11:52:24.854593 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ovsdb-server.service: Succeeded. Nov 16 11:52:24.855480 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Open vSwitch Database Unit. Nov 16 11:52:24.863107 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ovsdb-server.service: Consumed 267ms CPU time Nov 16 11:52:24.863628 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Basic System. Nov 16 11:52:24.869544 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Paths. Nov 16 11:52:24.875087 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: console-login-helper-messages-issuegen.path: Succeeded. Nov 16 11:52:24.878781 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Monitor console-login-helper-messages runtime issue snippets directory for changes. Nov 16 11:52:24.886402 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Slices. Nov 16 11:52:24.889849 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice User and Session Slice. Nov 16 11:52:24.894032 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: user.slice: Consumed 0 CPU time Nov 16 11:52:24.894724 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping OSTree Finalize Staged Deployment... Nov 16 11:52:24.898454 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Sockets. Nov 16 11:52:24.900806 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: bootupd.socket: Succeeded. Nov 16 11:52:24.901443 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Closed bootupd.socket. Nov 16 11:52:24.904380 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: bootupd.socket: Consumed 0 CPU time Nov 16 11:52:24.904472 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dbus.socket: Succeeded. Nov 16 11:52:24.905094 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Closed D-Bus System Message Bus Socket. Nov 16 11:52:24.909020 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dbus.socket: Consumed 0 CPU time Nov 16 11:52:25.555702 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: EXT4-fs (sdb3): re-mounted. Opts: Nov 16 11:52:25.556627 ropatil1611az-gs6zj-worker-westus-j4ngg ostree[2472]: Finalizing staged deployment Nov 16 11:52:35.667858 ropatil1611az-gs6zj-worker-westus-j4ngg ostree[2472]: Copying /etc changes: 13 modified, 0 removed, 103 added Nov 16 11:52:35.668350 ropatil1611az-gs6zj-worker-westus-j4ngg ostree[2472]: Copying /etc changes: 13 modified, 0 removed, 103 added Nov 16 11:52:45.819920 ropatil1611az-gs6zj-worker-westus-j4ngg ostree[2472]: Bootloader updated; bootconfig swap: yes; deployment count change: 1 Nov 16 11:52:45.820580 ropatil1611az-gs6zj-worker-westus-j4ngg ostree[2472]: Bootloader updated; bootconfig swap: yes; deployment count change: 1 Nov 16 11:52:45.910477 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ostree-finalize-staged.service: Succeeded. Nov 16 11:52:45.911395 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped OSTree Finalize Staged Deployment. Nov 16 11:52:45.916521 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ostree-finalize-staged.service: Consumed 2.066s CPU time Nov 16 11:52:45.917090 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ostree-finalize-staged.path: Succeeded. Nov 16 11:52:45.920746 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped OSTree Monitor Staged Deployment. Nov 16 11:52:45.924538 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target System Initialization. Nov 16 11:52:45.931326 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Local Encrypted Volumes. Nov 16 11:52:45.938338 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Local Encrypted Volumes (Pre). Nov 16 11:52:45.945135 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-ask-password-console.path: Succeeded. Nov 16 11:52:45.945241 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. Nov 16 11:52:45.952712 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-ask-password-wall.path: Succeeded. Nov 16 11:52:45.952781 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Forward Password Requests to Wall Directory Watch. Nov 16 11:52:45.962100 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Load/Save Random Seed... Nov 16 11:52:45.967037 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-sysctl.service: Succeeded. Nov 16 11:52:45.967847 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Apply Kernel Variables. Nov 16 11:52:45.974186 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-sysctl.service: Consumed 0 CPU time Nov 16 11:52:45.974798 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Update UTMP about System Boot/Shutdown... Nov 16 11:52:45.980659 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-modules-load.service: Succeeded. Nov 16 11:52:45.981247 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Load Kernel Modules. Nov 16 11:52:45.987464 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-modules-load.service: Consumed 0 CPU time Nov 16 11:52:45.987665 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-update-done.service: Succeeded. Nov 16 11:52:45.988307 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Update is Completed. Nov 16 11:52:45.994994 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-update-done.service: Consumed 0 CPU time Nov 16 11:52:45.995196 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ldconfig.service: Succeeded. Nov 16 11:52:45.995795 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Rebuild Dynamic Linker Cache. Nov 16 11:52:46.002130 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ldconfig.service: Consumed 0 CPU time Nov 16 11:52:46.002353 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-hwdb-update.service: Succeeded. Nov 16 11:52:46.002900 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Rebuild Hardware Database. Nov 16 11:52:46.009746 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-hwdb-update.service: Consumed 0 CPU time Nov 16 11:52:46.009945 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journal-catalog-update.service: Succeeded. Nov 16 11:52:46.010611 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Rebuild Journal Catalog. Nov 16 11:52:46.017024 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journal-catalog-update.service: Consumed 0 CPU time Nov 16 11:52:46.017474 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-random-seed.service: Succeeded. Nov 16 11:52:46.017989 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Load/Save Random Seed. Nov 16 11:52:46.024028 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-random-seed.service: Consumed 5ms CPU time Nov 16 11:52:46.026867 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-update-utmp.service: Succeeded. Nov 16 11:52:46.027555 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Update UTMP about System Boot/Shutdown. Nov 16 11:52:46.035221 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-update-utmp.service: Consumed 6ms CPU time Nov 16 11:52:46.035700 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Security Auditing Service... Nov 16 11:52:46.066701 ropatil1611az-gs6zj-worker-westus-j4ngg auditd[1501]: The audit daemon is exiting. Nov 16 11:52:46.079667 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: audit: type=1305 audit(1668599566.066:113): op=set audit_pid=0 old=1501 auid=4294967295 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=1 Nov 16 11:52:46.089329 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: auditd.service: Succeeded. Nov 16 11:52:46.090167 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Security Auditing Service. Nov 16 11:52:46.096845 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: auditd.service: Consumed 67ms CPU time Nov 16 11:52:46.097287 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-tmpfiles-setup.service: Succeeded. Nov 16 11:52:46.097906 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Create Volatile Files and Directories. Nov 16 11:52:46.115706 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: audit: type=1131 audit(1668599566.096:114): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=auditd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 16 11:52:46.123443 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-tmpfiles-setup.service: Consumed 0 CPU time Nov 16 11:52:46.123472 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Local File Systems. Nov 16 11:52:46.142658 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: audit: type=1131 audit(1668599566.122:115): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-tmpfiles-setup comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 16 11:52:46.150200 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Unmounting /var/lib/containers/storage/overlay... Nov 16 11:52:46.157108 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Unmounting /etc... Nov 16 11:52:46.159613 ropatil1611az-gs6zj-worker-westus-j4ngg umount[2483]: umount: /etc: target is busy. Nov 16 11:52:46.161374 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Unmounting CoreOS Dynamic Mount for /boot... Nov 16 11:52:46.167196 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Unmounting Temporary Directory (/tmp)... Nov 16 11:52:46.172811 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journal-flush.service: Succeeded. Nov 16 11:52:46.173534 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Flush Journal to Persistent Storage. Nov 16 11:52:46.181347 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journal-flush.service: Consumed 0 CPU time Nov 16 11:52:46.181589 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ostree-remount.service: Succeeded. Nov 16 11:52:46.182232 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped OSTree Remount OS/ Bind Mounts. Nov 16 11:52:46.201717 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: audit: type=1131 audit(1668599566.180:116): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-journal-flush comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 16 11:52:46.208559 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ostree-remount.service: Consumed 0 CPU time Nov 16 11:52:46.211883 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay.mount: Succeeded. Nov 16 11:52:46.212751 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Unmounted /var/lib/containers/storage/overlay. Nov 16 11:52:46.231669 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: audit: type=1131 audit(1668599566.207:117): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=ostree-remount comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 16 11:52:46.238179 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay.mount: Consumed 5ms CPU time Nov 16 11:52:46.238438 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: etc.mount: Mount process exited, code=exited status=32 Nov 16 11:52:46.238459 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Failed unmounting /etc. Nov 16 11:52:46.244787 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: tmp.mount: Succeeded. Nov 16 11:52:46.245543 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Unmounted Temporary Directory (/tmp). Nov 16 11:52:46.252593 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: tmp.mount: Consumed 6ms CPU time Nov 16 11:52:46.253179 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Swap. Nov 16 11:52:46.259740 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Unmounting /var... Nov 16 11:52:46.263121 ropatil1611az-gs6zj-worker-westus-j4ngg umount[2489]: umount: /var: target is busy. Nov 16 11:52:46.264207 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var.mount: Mount process exited, code=exited status=32 Nov 16 11:52:46.264249 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Failed unmounting /var. Nov 16 11:52:46.271747 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Unmounting sysroot-ostree-deploy-rhcos-var.mount... Nov 16 11:52:46.288077 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-ostree-deploy-rhcos-var.mount: Succeeded. Nov 16 11:52:46.288669 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Unmounted sysroot-ostree-deploy-rhcos-var.mount. Nov 16 11:52:46.295371 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-ostree-deploy-rhcos-var.mount: Consumed 2ms CPU time Nov 16 11:52:46.296176 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Unmounting /sysroot... Nov 16 11:52:46.302055 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: boot.mount: Succeeded. Nov 16 11:52:46.302667 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Unmounted CoreOS Dynamic Mount for /boot. Nov 16 11:52:46.310077 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: boot.mount: Consumed 46ms CPU time Nov 16 11:52:46.310283 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot.mount: Succeeded. Nov 16 11:52:46.310925 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Unmounted /sysroot. Nov 16 11:52:46.317280 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot.mount: Consumed 2ms CPU time Nov 16 11:52:46.317738 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Unmount All Filesystems. Nov 16 11:52:46.324985 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-fsck@dev-disk-by\x2duuid-4a20a463\x2d024a\x2d4bb3\x2dafd8\x2d31decc06c38d.service: Succeeded. Nov 16 11:52:46.325016 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped File System Check on /dev/disk/by-uuid/4a20a463-024a-4bb3-afd8-31decc06c38d. Nov 16 11:52:46.334201 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-fsck@dev-disk-by\x2duuid-4a20a463\x2d024a\x2d4bb3\x2dafd8\x2d31decc06c38d.service: Consumed 17ms CPU time Nov 16 11:52:46.334228 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Local File Systems (Pre). Nov 16 11:52:46.358704 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: audit: type=1131 audit(1668599566.333:118): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-fsck@dev-disk-by\x2duuid-4a20a463\x2d024a\x2d4bb3\x2dafd8\x2d31decc06c38d comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 16 11:52:46.366215 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling... Nov 16 11:52:46.373125 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-tmpfiles-setup-dev.service: Succeeded. Nov 16 11:52:46.373848 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Create Static Device Nodes in /dev. Nov 16 11:52:46.381142 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-tmpfiles-setup-dev.service: Consumed 0 CPU time Nov 16 11:52:46.381397 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-sysusers.service: Succeeded. Nov 16 11:52:46.382111 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Create System Users. Nov 16 11:52:46.402818 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: audit: type=1131 audit(1668599566.380:119): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-tmpfiles-setup-dev comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 16 11:52:46.408231 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-sysusers.service: Consumed 0 CPU time Nov 16 11:52:46.408899 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice system-systemd\x2dfsck.slice. Nov 16 11:52:46.428166 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: audit: type=1131 audit(1668599566.407:120): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-sysusers comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 16 11:52:46.434817 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: system-systemd\x2dfsck.slice: Consumed 17ms CPU time Nov 16 11:52:46.443498 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: lvm2-monitor.service: Succeeded. Nov 16 11:52:46.444243 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling. Nov 16 11:52:46.452692 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: lvm2-monitor.service: Consumed 26ms CPU time Nov 16 11:52:46.452963 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Shutdown. Nov 16 11:52:46.472750 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: audit: type=1131 audit(1668599566.452:121): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=lvm2-monitor comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 16 11:52:46.478359 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Final Step. Nov 16 11:52:46.485589 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Reboot... Nov 16 11:52:46.493698 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Shutting down. Nov 16 11:52:46.783087 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-shutdown[1]: Syncing filesystems and block devices. Nov 16 11:52:47.203055 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-shutdown[1]: Sending SIGTERM to remaining processes... Nov 16 11:52:47.203195 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-journald[1367]: Journal stopped -- Logs begin at Wed 2022-11-16 11:49:03 UTC, end at Wed 2022-11-16 17:07:21 UTC. -- Nov 16 11:53:01.129468 localhost kernel: Linux version 4.18.0-305.65.1.el8_4.x86_64 (mockbuild@x86-vm-09.build.eng.bos.redhat.com) (gcc version 8.4.1 20200928 (Red Hat 8.4.1-1) (GCC)) #1 SMP Thu Sep 22 08:28:21 EDT 2022 Nov 16 11:53:01.129489 localhost kernel: Command line: BOOT_IMAGE=(hd0,gpt3)/ostree/rhcos-7e6edd7ff4609c0b1e388de010a2ee1ae8883be1667b7adf26f17cf24f0b3760/vmlinuz-4.18.0-305.65.1.el8_4.x86_64 random.trust_cpu=on console=tty0 console=ttyS0,115200n8 ostree=/ostree/boot.0/rhcos/7e6edd7ff4609c0b1e388de010a2ee1ae8883be1667b7adf26f17cf24f0b3760/0 ignition.platform.id=azure root=UUID=566864e3-acbe-4961-8488-233c4e33ce96 rw rootflags=prjquota boot=UUID=4a20a463-024a-4bb3-afd8-31decc06c38d Nov 16 11:53:01.129501 localhost kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' Nov 16 11:53:01.129508 localhost kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' Nov 16 11:53:01.129513 localhost kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' Nov 16 11:53:01.129519 localhost kernel: x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 Nov 16 11:53:01.129526 localhost kernel: x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. Nov 16 11:53:01.129532 localhost kernel: BIOS-provided physical RAM map: Nov 16 11:53:01.129537 localhost kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable Nov 16 11:53:01.129546 localhost kernel: BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved Nov 16 11:53:01.129551 localhost kernel: BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved Nov 16 11:53:01.129557 localhost kernel: BIOS-e820: [mem 0x0000000000100000-0x000000003ffeffff] usable Nov 16 11:53:01.129563 localhost kernel: BIOS-e820: [mem 0x000000003fff0000-0x000000003fffefff] ACPI data Nov 16 11:53:01.129568 localhost kernel: BIOS-e820: [mem 0x000000003ffff000-0x000000003fffffff] ACPI NVS Nov 16 11:53:01.129575 localhost kernel: BIOS-e820: [mem 0x0000000100000000-0x00000004bfffffff] usable Nov 16 11:53:01.129584 localhost kernel: NX (Execute Disable) protection: active Nov 16 11:53:01.129590 localhost kernel: SMBIOS 2.3 present. Nov 16 11:53:01.129596 localhost kernel: DMI: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008 12/07/2018 Nov 16 11:53:01.129602 localhost kernel: Hypervisor detected: Microsoft Hyper-V Nov 16 11:53:01.129608 localhost kernel: Hyper-V: features 0x2e7f, hints 0x60c2c, misc 0xed7b2 Nov 16 11:53:01.129614 localhost kernel: Hyper-V Host Build:20348-10.0-1-0.1272 Nov 16 11:53:01.129620 localhost kernel: Hyper-V: LAPIC Timer Frequency: 0x30d40 Nov 16 11:53:01.129626 localhost kernel: tsc: Marking TSC unstable due to running on Hyper-V Nov 16 11:53:01.129631 localhost kernel: Hyper-V: Using hypercall for remote TLB flush Nov 16 11:53:01.129638 localhost kernel: clocksource: hyperv_clocksource_tsc_page: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns Nov 16 11:53:01.129644 localhost kernel: tsc: Detected 2294.686 MHz processor Nov 16 11:53:01.129667 localhost kernel: e820: update [mem 0x00000000-0x00000fff] usable ==> reserved Nov 16 11:53:01.129675 localhost kernel: e820: remove [mem 0x000a0000-0x000fffff] usable Nov 16 11:53:01.129686 localhost kernel: last_pfn = 0x4c0000 max_arch_pfn = 0x400000000 Nov 16 11:53:01.129691 localhost kernel: MTRR default type: uncachable Nov 16 11:53:01.129697 localhost kernel: MTRR fixed ranges enabled: Nov 16 11:53:01.129702 localhost kernel: 00000-9FFFF write-back Nov 16 11:53:01.129707 localhost kernel: A0000-DFFFF uncachable Nov 16 11:53:01.129713 localhost kernel: E0000-FFFFF write-back Nov 16 11:53:01.129718 localhost kernel: MTRR variable ranges enabled: Nov 16 11:53:01.129724 localhost kernel: 0 base 000000000000 mask 3FFFC0000000 write-back Nov 16 11:53:01.129729 localhost kernel: 1 base 000100000000 mask 3FF000000000 write-back Nov 16 11:53:01.129735 localhost kernel: 2 disabled Nov 16 11:53:01.129740 localhost kernel: 3 disabled Nov 16 11:53:01.129749 localhost kernel: 4 disabled Nov 16 11:53:01.129758 localhost kernel: 5 disabled Nov 16 11:53:01.129764 localhost kernel: 6 disabled Nov 16 11:53:01.129769 localhost kernel: 7 disabled Nov 16 11:53:01.129775 localhost kernel: x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT Nov 16 11:53:01.129780 localhost kernel: e820: update [mem 0x40000000-0xffffffff] usable ==> reserved Nov 16 11:53:01.129786 localhost kernel: last_pfn = 0x3fff0 max_arch_pfn = 0x400000000 Nov 16 11:53:01.129792 localhost kernel: found SMP MP-table at [mem 0x000ff780-0x000ff78f] Nov 16 11:53:01.129797 localhost kernel: Using GB pages for direct mapping Nov 16 11:53:01.129805 localhost kernel: BRK [0x2be802000, 0x2be802fff] PGTABLE Nov 16 11:53:01.129811 localhost kernel: BRK [0x2be803000, 0x2be803fff] PGTABLE Nov 16 11:53:01.129816 localhost kernel: BRK [0x2be804000, 0x2be804fff] PGTABLE Nov 16 11:53:01.129822 localhost kernel: BRK [0x2be805000, 0x2be805fff] PGTABLE Nov 16 11:53:01.129828 localhost kernel: BRK [0x2be806000, 0x2be806fff] PGTABLE Nov 16 11:53:01.129833 localhost kernel: RAMDISK: [mem 0x2d1c0000-0x328d7fff] Nov 16 11:53:01.129839 localhost kernel: ACPI: Early table checksum verification disabled Nov 16 11:53:01.129844 localhost kernel: ACPI: RSDP 0x00000000000F5C00 000014 (v00 ACPIAM) Nov 16 11:53:01.129850 localhost kernel: ACPI: RSDT 0x000000003FFF0000 000040 (v01 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:53:01.129856 localhost kernel: ACPI: FACP 0x000000003FFF0200 000081 (v02 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:53:01.129861 localhost kernel: ACPI: DSDT 0x000000003FFF1D24 003CD5 (v01 MSFTVM MSFTVM02 00000002 INTL 02002026) Nov 16 11:53:01.129867 localhost kernel: ACPI: FACS 0x000000003FFFF000 000040 Nov 16 11:53:01.129875 localhost kernel: ACPI: WAET 0x000000003FFF1A80 000028 (v01 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:53:01.129881 localhost kernel: ACPI: SLIC 0x000000003FFF1AC0 000176 (v01 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:53:01.129887 localhost kernel: ACPI: OEM0 0x000000003FFF1CC0 000064 (v01 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:53:01.129892 localhost kernel: ACPI: SRAT 0x000000003FFF0800 0001A0 (v02 VRTUAL MICROSFT 00000001 MSFT 00000001) Nov 16 11:53:01.129898 localhost kernel: ACPI: APIC 0x000000003FFF0300 000092 (v01 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:53:01.129904 localhost kernel: ACPI: OEMB 0x000000003FFFF040 000064 (v01 VRTUAL MICROSFT 12001807 MSFT 00000097) Nov 16 11:53:01.129909 localhost kernel: ACPI: Local APIC address 0xfee00000 Nov 16 11:53:01.129915 localhost kernel: SRAT: PXM 0 -> APIC 0x00 -> Node 0 Nov 16 11:53:01.129921 localhost kernel: SRAT: PXM 0 -> APIC 0x01 -> Node 0 Nov 16 11:53:01.129926 localhost kernel: SRAT: PXM 0 -> APIC 0x02 -> Node 0 Nov 16 11:53:01.129935 localhost kernel: SRAT: PXM 0 -> APIC 0x03 -> Node 0 Nov 16 11:53:01.129940 localhost kernel: SRAT: PXM 0 -> APIC 0x04 -> Node 0 Nov 16 11:53:01.129946 localhost kernel: SRAT: PXM 0 -> APIC 0x05 -> Node 0 Nov 16 11:53:01.129951 localhost kernel: SRAT: PXM 0 -> APIC 0x06 -> Node 0 Nov 16 11:53:01.129957 localhost kernel: SRAT: PXM 0 -> APIC 0x07 -> Node 0 Nov 16 11:53:01.129963 localhost kernel: ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x3fffffff] hotplug Nov 16 11:53:01.129968 localhost kernel: ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0x4bfffffff] hotplug Nov 16 11:53:01.129974 localhost kernel: ACPI: SRAT: Node 0 PXM 0 [mem 0x4c0200000-0xfdfffffff] hotplug Nov 16 11:53:01.129980 localhost kernel: ACPI: SRAT: Node 0 PXM 0 [mem 0x1000000000-0xffffffffff] hotplug Nov 16 11:53:01.129985 localhost kernel: ACPI: SRAT: Node 0 PXM 0 [mem 0x10000200000-0x1ffffffffff] hotplug Nov 16 11:53:01.130007 localhost kernel: ACPI: SRAT: Node 0 PXM 0 [mem 0x20000200000-0x3ffffffffff] hotplug Nov 16 11:53:01.130012 localhost kernel: NUMA: Node 0 [mem 0x00000000-0x3fffffff] + [mem 0x100000000-0x4bfffffff] -> [mem 0x00000000-0x4bfffffff] Nov 16 11:53:01.130020 localhost kernel: NODE_DATA(0) allocated [mem 0x4bffd6000-0x4bfffffff] Nov 16 11:53:01.130026 localhost kernel: Zone ranges: Nov 16 11:53:01.130031 localhost kernel: DMA [mem 0x0000000000001000-0x0000000000ffffff] Nov 16 11:53:01.130037 localhost kernel: DMA32 [mem 0x0000000001000000-0x00000000ffffffff] Nov 16 11:53:01.130042 localhost kernel: Normal [mem 0x0000000100000000-0x00000004bfffffff] Nov 16 11:53:01.130048 localhost kernel: Device empty Nov 16 11:53:01.130053 localhost kernel: Movable zone start for each node Nov 16 11:53:01.130058 localhost kernel: Early memory node ranges Nov 16 11:53:01.130064 localhost kernel: node 0: [mem 0x0000000000001000-0x000000000009efff] Nov 16 11:53:01.130069 localhost kernel: node 0: [mem 0x0000000000100000-0x000000003ffeffff] Nov 16 11:53:01.130075 localhost kernel: node 0: [mem 0x0000000100000000-0x00000004bfffffff] Nov 16 11:53:01.130080 localhost kernel: Zeroed struct page in unavailable ranges: 114 pages Nov 16 11:53:01.130086 localhost kernel: Initmem setup node 0 [mem 0x0000000000001000-0x00000004bfffffff] Nov 16 11:53:01.130094 localhost kernel: On node 0 totalpages: 4194190 Nov 16 11:53:01.130100 localhost kernel: DMA zone: 64 pages used for memmap Nov 16 11:53:01.130105 localhost kernel: DMA zone: 21 pages reserved Nov 16 11:53:01.130110 localhost kernel: DMA zone: 3998 pages, LIFO batch:0 Nov 16 11:53:01.130116 localhost kernel: DMA32 zone: 4032 pages used for memmap Nov 16 11:53:01.130121 localhost kernel: DMA32 zone: 258032 pages, LIFO batch:63 Nov 16 11:53:01.130127 localhost kernel: Normal zone: 61440 pages used for memmap Nov 16 11:53:01.130132 localhost kernel: Normal zone: 3932160 pages, LIFO batch:63 Nov 16 11:53:01.130138 localhost kernel: ACPI: PM-Timer IO Port: 0x408 Nov 16 11:53:01.130159 localhost kernel: ACPI: Local APIC address 0xfee00000 Nov 16 11:53:01.130164 localhost kernel: ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) Nov 16 11:53:01.130169 localhost kernel: IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 Nov 16 11:53:01.130177 localhost kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) Nov 16 11:53:01.130183 localhost kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) Nov 16 11:53:01.130188 localhost kernel: ACPI: IRQ0 used by override. Nov 16 11:53:01.130193 localhost kernel: ACPI: IRQ9 used by override. Nov 16 11:53:01.130198 localhost kernel: Using ACPI (MADT) for SMP configuration information Nov 16 11:53:01.130204 localhost kernel: smpboot: Allowing 8 CPUs, 0 hotplug CPUs Nov 16 11:53:01.130209 localhost kernel: PM: Registered nosave memory: [mem 0x00000000-0x00000fff] Nov 16 11:53:01.130214 localhost kernel: PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] Nov 16 11:53:01.130219 localhost kernel: PM: Registered nosave memory: [mem 0x000a0000-0x000dffff] Nov 16 11:53:01.130224 localhost kernel: PM: Registered nosave memory: [mem 0x000e0000-0x000fffff] Nov 16 11:53:01.130230 localhost kernel: PM: Registered nosave memory: [mem 0x3fff0000-0x3fffefff] Nov 16 11:53:01.130262 localhost kernel: PM: Registered nosave memory: [mem 0x3ffff000-0x3fffffff] Nov 16 11:53:01.130268 localhost kernel: PM: Registered nosave memory: [mem 0x40000000-0xffffffff] Nov 16 11:53:01.130274 localhost kernel: [mem 0x40000000-0xffffffff] available for PCI devices Nov 16 11:53:01.130279 localhost kernel: Booting paravirtualized kernel on Hyper-V Nov 16 11:53:01.130285 localhost kernel: clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns Nov 16 11:53:01.130409 localhost kernel: setup_percpu: NR_CPUS:8192 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1 Nov 16 11:53:01.130419 localhost kernel: percpu: Embedded 54 pages/cpu s184320 r8192 d28672 u262144 Nov 16 11:53:01.130426 localhost kernel: pcpu-alloc: s184320 r8192 d28672 u262144 alloc=1*2097152 Nov 16 11:53:01.130433 localhost kernel: pcpu-alloc: [0] 0 1 2 3 4 5 6 7 Nov 16 11:53:01.130439 localhost kernel: Hyper-V: PV spinlocks enabled Nov 16 11:53:01.130445 localhost kernel: PV qspinlock hash table entries: 256 (order: 0, 4096 bytes) Nov 16 11:53:01.130451 localhost kernel: Built 1 zonelists, mobility grouping on. Total pages: 4128633 Nov 16 11:53:01.130458 localhost kernel: Policy zone: Normal Nov 16 11:53:01.130465 localhost kernel: Kernel command line: BOOT_IMAGE=(hd0,gpt3)/ostree/rhcos-7e6edd7ff4609c0b1e388de010a2ee1ae8883be1667b7adf26f17cf24f0b3760/vmlinuz-4.18.0-305.65.1.el8_4.x86_64 random.trust_cpu=on console=tty0 console=ttyS0,115200n8 ostree=/ostree/boot.0/rhcos/7e6edd7ff4609c0b1e388de010a2ee1ae8883be1667b7adf26f17cf24f0b3760/0 ignition.platform.id=azure root=UUID=566864e3-acbe-4961-8488-233c4e33ce96 rw rootflags=prjquota boot=UUID=4a20a463-024a-4bb3-afd8-31decc06c38d Nov 16 11:53:01.130473 localhost kernel: Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. Nov 16 11:53:01.130479 localhost kernel: Memory: 1024388K/16776760K available (12293K kernel code, 2225K rwdata, 7716K rodata, 2480K init, 14040K bss, 464420K reserved, 0K cma-reserved) Nov 16 11:53:01.130491 localhost kernel: SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1 Nov 16 11:53:01.130497 localhost kernel: Kernel/User page tables isolation: enabled Nov 16 11:53:01.130504 localhost kernel: ftrace: allocating 38619 entries in 151 pages Nov 16 11:53:01.130510 localhost kernel: ftrace: allocated 151 pages with 5 groups Nov 16 11:53:01.130516 localhost kernel: rcu: Hierarchical RCU implementation. Nov 16 11:53:01.130524 localhost kernel: rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=8. Nov 16 11:53:01.130531 localhost kernel: rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. Nov 16 11:53:01.130537 localhost kernel: rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8 Nov 16 11:53:01.130544 localhost kernel: NR_IRQS: 524544, nr_irqs: 488, preallocated irqs: 16 Nov 16 11:53:01.130550 localhost kernel: random: crng done (trusting CPU's manufacturer) Nov 16 11:53:01.130556 localhost kernel: Console: colour VGA+ 80x25 Nov 16 11:53:01.130566 localhost kernel: printk: console [tty0] enabled Nov 16 11:53:01.130572 localhost kernel: printk: console [ttyS0] enabled Nov 16 11:53:01.130578 localhost kernel: ACPI: Core revision 20200925 Nov 16 11:53:01.130584 localhost kernel: APIC: Switch to symmetric I/O mode setup Nov 16 11:53:01.130591 localhost kernel: Hyper-V: Using IPI hypercalls Nov 16 11:53:01.130597 localhost kernel: Hyper-V: Using enlightened APIC (xapic mode) Nov 16 11:53:01.130604 localhost kernel: ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 Nov 16 11:53:01.130611 localhost kernel: Calibrating delay loop (skipped), value calculated using timer frequency.. 4589.37 BogoMIPS (lpj=2294686) Nov 16 11:53:01.130618 localhost kernel: pid_max: default: 32768 minimum: 301 Nov 16 11:53:01.130624 localhost kernel: Security Framework initialized Nov 16 11:53:01.130631 localhost kernel: Yama: becoming mindful. Nov 16 11:53:01.130640 localhost kernel: SELinux: Initializing. Nov 16 11:53:01.130646 localhost kernel: Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes) Nov 16 11:53:01.130653 localhost kernel: Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes) Nov 16 11:53:01.130660 localhost kernel: Mount-cache hash table entries: 32768 (order: 6, 262144 bytes) Nov 16 11:53:01.130667 localhost kernel: Mountpoint-cache hash table entries: 32768 (order: 6, 262144 bytes) Nov 16 11:53:01.130676 localhost kernel: Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8 Nov 16 11:53:01.130682 localhost kernel: Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4 Nov 16 11:53:01.130689 localhost kernel: FEATURE SPEC_CTRL Not Present Nov 16 11:53:01.130696 localhost kernel: FEATURE IBPB_SUPPORT Not Present Nov 16 11:53:01.130702 localhost kernel: Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization Nov 16 11:53:01.130709 localhost kernel: Spectre V2 : Mitigation: Full generic retpoline Nov 16 11:53:01.130715 localhost kernel: Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch Nov 16 11:53:01.130722 localhost kernel: Speculative Store Bypass: Vulnerable Nov 16 11:53:01.130728 localhost kernel: MDS: Mitigation: Clear CPU buffers Nov 16 11:53:01.130738 localhost kernel: TAA: Mitigation: Clear CPU buffers Nov 16 11:53:01.130744 localhost kernel: MMIO Stale Data: Vulnerable: Clear CPU buffers attempted, no microcode Nov 16 11:53:01.130751 localhost kernel: Freeing SMP alternatives memory: 32K Nov 16 11:53:01.130758 localhost kernel: smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz (family: 0x6, model: 0x4f, stepping: 0x1) Nov 16 11:53:01.130764 localhost kernel: Performance Events: unsupported p6 CPU model 79 no PMU driver, software events only. Nov 16 11:53:01.130771 localhost kernel: rcu: Hierarchical SRCU implementation. Nov 16 11:53:01.130777 localhost kernel: NMI watchdog: Perf NMI watchdog permanently disabled Nov 16 11:53:01.130784 localhost kernel: smp: Bringing up secondary CPUs ... Nov 16 11:53:01.130790 localhost kernel: x86: Booting SMP configuration: Nov 16 11:53:01.130799 localhost kernel: .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 Nov 16 11:53:01.130806 localhost kernel: smp: Brought up 1 node, 8 CPUs Nov 16 11:53:01.130813 localhost kernel: smpboot: Max logical packages: 1 Nov 16 11:53:01.130819 localhost kernel: smpboot: Total of 8 processors activated (36714.97 BogoMIPS) Nov 16 11:53:01.130826 localhost kernel: node 0 deferred pages initialised in 14ms Nov 16 11:53:01.130832 localhost kernel: devtmpfs: initialized Nov 16 11:53:01.130839 localhost kernel: x86/mm: Memory block size: 128MB Nov 16 11:53:01.130845 localhost kernel: PM: Registering ACPI NVS region [mem 0x3ffff000-0x3fffffff] (4096 bytes) Nov 16 11:53:01.130852 localhost kernel: clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns Nov 16 11:53:01.130859 localhost kernel: futex hash table entries: 2048 (order: 5, 131072 bytes) Nov 16 11:53:01.130868 localhost kernel: pinctrl core: initialized pinctrl subsystem Nov 16 11:53:01.130874 localhost kernel: NET: Registered protocol family 16 Nov 16 11:53:01.130881 localhost kernel: DMA: preallocated 2048 KiB GFP_KERNEL pool for atomic allocations Nov 16 11:53:01.130888 localhost kernel: DMA: preallocated 2048 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations Nov 16 11:53:01.130894 localhost kernel: DMA: preallocated 2048 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations Nov 16 11:53:01.130901 localhost kernel: audit: initializing netlink subsys (disabled) Nov 16 11:53:01.130908 localhost kernel: audit: type=2000 audit(1668599573.088:1): state=initialized audit_enabled=0 res=1 Nov 16 11:53:01.130931 localhost kernel: cpuidle: using governor menu Nov 16 11:53:01.130937 localhost kernel: ACPI: bus type PCI registered Nov 16 11:53:01.130964 localhost kernel: acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 Nov 16 11:53:01.130970 localhost kernel: PCI: Using configuration type 1 for base access Nov 16 11:53:01.130977 localhost kernel: mtrr: your CPUs had inconsistent variable MTRR settings Nov 16 11:53:01.130983 localhost kernel: mtrr: probably your BIOS does not setup all CPUs. Nov 16 11:53:01.130989 localhost kernel: mtrr: corrected configuration. Nov 16 11:53:01.130995 localhost kernel: HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages Nov 16 11:53:01.131001 localhost kernel: HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages Nov 16 11:53:01.131007 localhost kernel: cryptd: max_cpu_qlen set to 1000 Nov 16 11:53:01.131013 localhost kernel: ACPI: Added _OSI(Module Device) Nov 16 11:53:01.131022 localhost kernel: ACPI: Added _OSI(Processor Device) Nov 16 11:53:01.131028 localhost kernel: ACPI: Added _OSI(3.0 _SCP Extensions) Nov 16 11:53:01.131034 localhost kernel: ACPI: Added _OSI(Processor Aggregator Device) Nov 16 11:53:01.131040 localhost kernel: ACPI: Added _OSI(Linux-Dell-Video) Nov 16 11:53:01.131046 localhost kernel: ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio) Nov 16 11:53:01.131052 localhost kernel: ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics) Nov 16 11:53:01.131058 localhost kernel: ACPI: 1 ACPI AML tables successfully acquired and loaded Nov 16 11:53:01.131065 localhost kernel: ACPI: Interpreter enabled Nov 16 11:53:01.131071 localhost kernel: ACPI: (supports S0 S5) Nov 16 11:53:01.131093 localhost kernel: ACPI: Using IOAPIC for interrupt routing Nov 16 11:53:01.131101 localhost kernel: PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug Nov 16 11:53:01.131107 localhost kernel: ACPI: Enabled 1 GPEs in block 00 to 0F Nov 16 11:53:01.131131 localhost kernel: ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) Nov 16 11:53:01.131367 localhost kernel: acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI EDR HPX-Type3] Nov 16 11:53:01.131503 localhost kernel: acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. Nov 16 11:53:01.131514 localhost kernel: PCI host bridge to bus 0000:00 Nov 16 11:53:01.131675 localhost kernel: pci_bus 0000:00: root bus resource [mem 0xfe0000000-0xfffffffff window] Nov 16 11:53:01.131802 localhost kernel: pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] Nov 16 11:53:01.131923 localhost kernel: pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] Nov 16 11:53:01.132028 localhost kernel: pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] Nov 16 11:53:01.132167 localhost kernel: pci_bus 0000:00: root bus resource [mem 0x40000000-0xfffbffff window] Nov 16 11:53:01.132286 localhost kernel: pci_bus 0000:00: root bus resource [bus 00-ff] Nov 16 11:53:01.132466 localhost kernel: pci 0000:00:00.0: [8086:7192] type 00 class 0x060000 Nov 16 11:53:01.132607 localhost kernel: pci 0000:00:07.0: [8086:7110] type 00 class 0x060100 Nov 16 11:53:01.132787 localhost kernel: pci 0000:00:07.1: [8086:7111] type 00 class 0x010180 Nov 16 11:53:01.132922 localhost kernel: pci 0000:00:07.1: reg 0x20: [io 0xffa0-0xffaf] Nov 16 11:53:01.133094 localhost kernel: pci 0000:00:07.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] Nov 16 11:53:01.133244 localhost kernel: pci 0000:00:07.1: legacy IDE quirk: reg 0x14: [io 0x03f6] Nov 16 11:53:01.133401 localhost kernel: pci 0000:00:07.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] Nov 16 11:53:01.133526 localhost kernel: pci 0000:00:07.1: legacy IDE quirk: reg 0x1c: [io 0x0376] Nov 16 11:53:01.133679 localhost kernel: pci 0000:00:07.3: [8086:7113] type 00 class 0x068000 Nov 16 11:53:01.133691 localhost kernel: * Found PM-Timer Bug on the chipset. Due to workarounds for a bug, * this clock source is slow. Consider trying other clock sources Nov 16 11:53:01.133796 localhost kernel: pci 0000:00:07.3: quirk: [io 0x0400-0x043f] claimed by PIIX4 ACPI Nov 16 11:53:01.133911 localhost kernel: pci 0000:00:08.0: [1414:5353] type 00 class 0x030000 Nov 16 11:53:01.134040 localhost kernel: pci 0000:00:08.0: reg 0x10: [mem 0xf8000000-0xfbffffff] Nov 16 11:53:01.134050 localhost kernel: ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 7 9 10 *11 12 14 15) Nov 16 11:53:01.134056 localhost kernel: ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 7 9 10 11 12 14 15) *0, disabled. Nov 16 11:53:01.134066 localhost kernel: ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 7 9 10 11 12 14 15) *0, disabled. Nov 16 11:53:01.134072 localhost kernel: ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 7 9 10 11 12 14 15) *0, disabled. Nov 16 11:53:01.134078 localhost kernel: iommu: Default domain type: Passthrough Nov 16 11:53:01.134213 localhost kernel: pci 0000:00:08.0: vgaarb: setting as boot VGA device Nov 16 11:53:01.134373 localhost kernel: pci 0000:00:08.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none Nov 16 11:53:01.134500 localhost kernel: pci 0000:00:08.0: vgaarb: bridge control possible Nov 16 11:53:01.134510 localhost kernel: vgaarb: loaded Nov 16 11:53:01.134517 localhost kernel: SCSI subsystem initialized Nov 16 11:53:01.134524 localhost kernel: ACPI: bus type USB registered Nov 16 11:53:01.134535 localhost kernel: usbcore: registered new interface driver usbfs Nov 16 11:53:01.134542 localhost kernel: usbcore: registered new interface driver hub Nov 16 11:53:01.134549 localhost kernel: usbcore: registered new device driver usb Nov 16 11:53:01.134556 localhost kernel: pps_core: LinuxPPS API ver. 1 registered Nov 16 11:53:01.134562 localhost kernel: pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti Nov 16 11:53:01.134569 localhost kernel: PTP clock support registered Nov 16 11:53:01.134575 localhost kernel: EDAC MC: Ver: 3.0.0 Nov 16 11:53:01.134582 localhost kernel: PCI: Using ACPI for IRQ routing Nov 16 11:53:01.134588 localhost kernel: PCI: pci_cache_line_size set to 64 bytes Nov 16 11:53:01.134598 localhost kernel: e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff] Nov 16 11:53:01.134604 localhost kernel: e820: reserve RAM buffer [mem 0x3fff0000-0x3fffffff] Nov 16 11:53:01.134611 localhost kernel: NetLabel: Initializing Nov 16 11:53:01.134617 localhost kernel: NetLabel: domain hash size = 128 Nov 16 11:53:01.134624 localhost kernel: NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO Nov 16 11:53:01.134630 localhost kernel: NetLabel: unlabeled traffic allowed by default Nov 16 11:53:01.134637 localhost kernel: clocksource: Switched to clocksource hyperv_clocksource_tsc_page Nov 16 11:53:01.134648 localhost kernel: VFS: Disk quotas dquot_6.6.0 Nov 16 11:53:01.134673 localhost kernel: VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) Nov 16 11:53:01.134680 localhost kernel: pnp: PnP ACPI init Nov 16 11:53:01.134842 localhost kernel: pnp 00:00: Plug and Play ACPI device, IDs PNP0b00 (active) Nov 16 11:53:01.135075 localhost kernel: pnp 00:01: Plug and Play ACPI device, IDs PNP0303 PNP030b (active) Nov 16 11:53:01.135205 localhost kernel: pnp 00:02: Plug and Play ACPI device, IDs PNP0f03 PNP0f13 (active) Nov 16 11:53:01.135421 localhost kernel: pnp 00:03: [dma 0 disabled] Nov 16 11:53:01.135547 localhost kernel: pnp 00:03: Plug and Play ACPI device, IDs PNP0501 (active) Nov 16 11:53:01.135715 localhost kernel: pnp 00:04: [dma 0 disabled] Nov 16 11:53:01.135839 localhost kernel: pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active) Nov 16 11:53:01.135960 localhost kernel: pnp 00:05: [dma 2] Nov 16 11:53:01.136103 localhost kernel: pnp 00:05: Plug and Play ACPI device, IDs PNP0700 (active) Nov 16 11:53:01.136233 localhost kernel: system 00:06: [io 0x01e0-0x01ef] has been reserved Nov 16 11:53:01.136373 localhost kernel: system 00:06: [io 0x0160-0x016f] has been reserved Nov 16 11:53:01.136500 localhost kernel: system 00:06: [io 0x0278-0x027f] has been reserved Nov 16 11:53:01.136620 localhost kernel: system 00:06: [io 0x0378-0x037f] has been reserved Nov 16 11:53:01.136734 localhost kernel: system 00:06: [io 0x0678-0x067f] has been reserved Nov 16 11:53:01.136865 localhost kernel: system 00:06: [io 0x0778-0x077f] has been reserved Nov 16 11:53:01.137002 localhost kernel: system 00:06: [io 0x04d0-0x04d1] has been reserved Nov 16 11:53:01.137127 localhost kernel: system 00:06: Plug and Play ACPI device, IDs PNP0c02 (active) Nov 16 11:53:01.137238 localhost kernel: system 00:07: [io 0x0400-0x043f] has been reserved Nov 16 11:53:01.137413 localhost kernel: system 00:07: [io 0x0370-0x0371] has been reserved Nov 16 11:53:01.137538 localhost kernel: system 00:07: [io 0x0440-0x044f] has been reserved Nov 16 11:53:01.137652 localhost kernel: system 00:07: [mem 0xfec00000-0xfec00fff] could not be reserved Nov 16 11:53:01.137814 localhost kernel: system 00:07: [mem 0xfee00000-0xfee00fff] has been reserved Nov 16 11:53:01.137921 localhost kernel: system 00:07: Plug and Play ACPI device, IDs PNP0c02 (active) Nov 16 11:53:01.138045 localhost kernel: system 00:08: [mem 0x00000000-0x0009ffff] could not be reserved Nov 16 11:53:01.138153 localhost kernel: system 00:08: [mem 0x000c0000-0x000dffff] could not be reserved Nov 16 11:53:01.138267 localhost kernel: system 00:08: [mem 0x000e0000-0x000fffff] could not be reserved Nov 16 11:53:01.138421 localhost kernel: system 00:08: [mem 0x00100000-0x3fffffff] could not be reserved Nov 16 11:53:01.138540 localhost kernel: system 00:08: [mem 0xfffc0000-0xffffffff] has been reserved Nov 16 11:53:01.138666 localhost kernel: system 00:08: Plug and Play ACPI device, IDs PNP0c01 (active) Nov 16 11:53:01.138677 localhost kernel: pnp: PnP ACPI: found 9 devices Nov 16 11:53:01.138684 localhost kernel: clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns Nov 16 11:53:01.138795 localhost kernel: pci_bus 0000:00: resource 4 [mem 0xfe0000000-0xfffffffff window] Nov 16 11:53:01.138926 localhost kernel: pci_bus 0000:00: resource 5 [io 0x0000-0x0cf7 window] Nov 16 11:53:01.139065 localhost kernel: pci_bus 0000:00: resource 6 [io 0x0d00-0xffff window] Nov 16 11:53:01.139171 localhost kernel: pci_bus 0000:00: resource 7 [mem 0x000a0000-0x000bffff window] Nov 16 11:53:01.139276 localhost kernel: pci_bus 0000:00: resource 8 [mem 0x40000000-0xfffbffff window] Nov 16 11:53:01.139286 localhost kernel: NET: Registered protocol family 2 Nov 16 11:53:01.139321 localhost kernel: IP idents hash table entries: 262144 (order: 9, 2097152 bytes) Nov 16 11:53:01.139329 localhost kernel: tcp_listen_portaddr_hash hash table entries: 8192 (order: 5, 131072 bytes) Nov 16 11:53:01.139336 localhost kernel: TCP established hash table entries: 131072 (order: 8, 1048576 bytes) Nov 16 11:53:01.139343 localhost kernel: TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) Nov 16 11:53:01.139349 localhost kernel: TCP: Hash tables configured (established 131072 bind 65536) Nov 16 11:53:01.139361 localhost kernel: MPTCP token hash table entries: 16384 (order: 6, 393216 bytes) Nov 16 11:53:01.139367 localhost kernel: UDP hash table entries: 8192 (order: 6, 262144 bytes) Nov 16 11:53:01.139374 localhost kernel: UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes) Nov 16 11:53:01.139380 localhost kernel: NET: Registered protocol family 1 Nov 16 11:53:01.139387 localhost kernel: NET: Registered protocol family 44 Nov 16 11:53:01.139518 localhost kernel: pci 0000:00:00.0: Limiting direct PCI/PCI transfers Nov 16 11:53:01.139672 localhost kernel: pci 0000:00:08.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] Nov 16 11:53:01.139804 localhost kernel: pci 0000:00:08.0: pci_fixup_video+0x0/0x110 took 10276 usecs Nov 16 11:53:01.139813 localhost kernel: PCI: CLS 0 bytes, default 64 Nov 16 11:53:01.139824 localhost kernel: Unpacking initramfs... Nov 16 11:53:01.139830 localhost kernel: Freeing initrd memory: 89184K Nov 16 11:53:01.139837 localhost kernel: PCI-DMA: Using software bounce buffering for IO (SWIOTLB) Nov 16 11:53:01.139843 localhost kernel: software IO TLB: mapped [mem 0x3bff0000-0x3fff0000] (64MB) Nov 16 11:53:01.139849 localhost kernel: Initialise system trusted keyrings Nov 16 11:53:01.139855 localhost kernel: Key type blacklist registered Nov 16 11:53:01.139861 localhost kernel: workingset: timestamp_bits=36 max_order=22 bucket_order=0 Nov 16 11:53:01.139868 localhost kernel: zbud: loaded Nov 16 11:53:01.139874 localhost kernel: pstore: using deflate compression Nov 16 11:53:01.139882 localhost kernel: Platform Keyring initialized Nov 16 11:53:01.139889 localhost kernel: NET: Registered protocol family 38 Nov 16 11:53:01.139895 localhost kernel: Key type asymmetric registered Nov 16 11:53:01.139901 localhost kernel: Asymmetric key parser 'x509' registered Nov 16 11:53:01.139907 localhost kernel: Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) Nov 16 11:53:01.139913 localhost kernel: io scheduler mq-deadline registered Nov 16 11:53:01.139937 localhost kernel: io scheduler kyber registered Nov 16 11:53:01.139943 localhost kernel: io scheduler bfq registered Nov 16 11:53:01.139949 localhost kernel: atomic64_test: passed for x86-64 platform with CX8 and with SSE Nov 16 11:53:01.139956 localhost kernel: shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 Nov 16 11:53:01.139980 localhost kernel: intel_idle: Please enable MWAIT in BIOS SETUP Nov 16 11:53:01.139987 localhost kernel: input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 Nov 16 11:53:01.139993 localhost kernel: ACPI: Power Button [PWRF] Nov 16 11:53:01.139999 localhost kernel: Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled Nov 16 11:53:01.140005 localhost kernel: 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A Nov 16 11:53:01.140012 localhost kernel: 00:04: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A Nov 16 11:53:01.140018 localhost kernel: Non-volatile memory driver v1.3 Nov 16 11:53:01.140024 localhost kernel: rdac: device handler registered Nov 16 11:53:01.140030 localhost kernel: hp_sw: device handler registered Nov 16 11:53:01.140039 localhost kernel: emc: device handler registered Nov 16 11:53:01.140060 localhost kernel: alua: device handler registered Nov 16 11:53:01.140068 localhost kernel: libphy: Fixed MDIO Bus: probed Nov 16 11:53:01.140075 localhost kernel: ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver Nov 16 11:53:01.140084 localhost kernel: ehci-pci: EHCI PCI platform driver Nov 16 11:53:01.140090 localhost kernel: ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver Nov 16 11:53:01.140097 localhost kernel: ohci-pci: OHCI PCI platform driver Nov 16 11:53:01.140103 localhost kernel: uhci_hcd: USB Universal Host Controller Interface driver Nov 16 11:53:01.140109 localhost kernel: usbcore: registered new interface driver usbserial_generic Nov 16 11:53:01.140132 localhost kernel: usbserial: USB Serial support registered for generic Nov 16 11:53:01.140138 localhost kernel: i8042: PNP: PS/2 Controller [PNP0303:PS2K,PNP0f03:PS2M] at 0x60,0x64 irq 1,12 Nov 16 11:53:01.140144 localhost kernel: serio: i8042 KBD port at 0x60,0x64 irq 1 Nov 16 11:53:01.140150 localhost kernel: serio: i8042 AUX port at 0x60,0x64 irq 12 Nov 16 11:53:01.140159 localhost kernel: mousedev: PS/2 mouse device common for all mice Nov 16 11:53:01.140275 localhost kernel: rtc_cmos 00:00: RTC can wake from S4 Nov 16 11:53:01.140437 localhost kernel: rtc_cmos 00:00: registered as rtc0 Nov 16 11:53:01.140555 localhost kernel: rtc_cmos 00:00: alarms up to one month, 114 bytes nvram Nov 16 11:53:01.140566 localhost kernel: intel_pstate: CPU model not supported Nov 16 11:53:01.140573 localhost kernel: input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 Nov 16 11:53:01.140580 localhost kernel: hid: raw HID events driver (C) Jiri Kosina Nov 16 11:53:01.140587 localhost kernel: usbcore: registered new interface driver usbhid Nov 16 11:53:01.140594 localhost kernel: usbhid: USB HID core driver Nov 16 11:53:01.140605 localhost kernel: drop_monitor: Initializing network drop monitor service Nov 16 11:53:01.140612 localhost kernel: Initializing XFRM netlink socket Nov 16 11:53:01.140619 localhost kernel: NET: Registered protocol family 10 Nov 16 11:53:01.140625 localhost kernel: Segment Routing with IPv6 Nov 16 11:53:01.140649 localhost kernel: NET: Registered protocol family 17 Nov 16 11:53:01.140655 localhost kernel: mpls_gso: MPLS GSO support Nov 16 11:53:01.140662 localhost kernel: AVX2 version of gcm_enc/dec engaged. Nov 16 11:53:01.140668 localhost kernel: AES CTR mode by8 optimization enabled Nov 16 11:53:01.140821 localhost kernel: psmouse serio1: trackpoint: failed to get extended button data, assuming 3 buttons Nov 16 11:53:01.140836 localhost kernel: registered taskstats version 1 Nov 16 11:53:01.140842 localhost kernel: Loading compiled-in X.509 certificates Nov 16 11:53:01.140849 localhost kernel: Loaded X.509 cert 'Red Hat Enterprise Linux kernel signing key: ccb9b4751382772d18fe86de326a0645bd233a71' Nov 16 11:53:01.140855 localhost kernel: Loaded X.509 cert 'Red Hat Enterprise Linux Driver Update Program (key 3): bf57f3e87362bc7229d9f465321773dfd1f77a80' Nov 16 11:53:01.140862 localhost kernel: Loaded X.509 cert 'Red Hat Enterprise Linux kpatch signing key: 4d38fd864ebe18c5f0b72e3852e2014c3a676fc8' Nov 16 11:53:01.140868 localhost kernel: zswap: loaded using pool lzo/zbud Nov 16 11:53:01.140874 localhost kernel: page_owner is disabled Nov 16 11:53:01.140880 localhost kernel: Key type big_key registered Nov 16 11:53:01.140886 localhost kernel: Key type encrypted registered Nov 16 11:53:01.140912 localhost kernel: ima: No TPM chip found, activating TPM-bypass! Nov 16 11:53:01.140919 localhost kernel: ima: Allocated hash algorithm: sha256 Nov 16 11:53:01.140925 localhost kernel: ima: No architecture policies found Nov 16 11:53:01.140932 localhost kernel: evm: Initialising EVM extended attributes: Nov 16 11:53:01.140938 localhost kernel: evm: security.selinux Nov 16 11:53:01.140944 localhost kernel: evm: security.ima Nov 16 11:53:01.140951 localhost kernel: evm: security.capability Nov 16 11:53:01.140957 localhost kernel: evm: HMAC attrs: 0x1 Nov 16 11:53:01.141083 localhost kernel: rtc_cmos 00:00: setting system clock to 2022-11-16 11:53:00 UTC (1668599580) Nov 16 11:53:01.141114 localhost kernel: Unstable clock detected, switching default tracing clock to "global" If you want to keep using the local clock, then add: "trace_clock=local" on the kernel command line Nov 16 11:53:01.141121 localhost kernel: Freeing unused decrypted memory: 2036K Nov 16 11:53:01.141127 localhost kernel: Freeing unused kernel memory: 2480K Nov 16 11:53:01.141134 localhost kernel: Write protecting the kernel read-only data: 22528k Nov 16 11:53:01.141140 localhost kernel: Freeing unused kernel memory: 2012K Nov 16 11:53:01.141147 localhost kernel: Freeing unused kernel memory: 476K Nov 16 11:53:01.141176 localhost systemd-journald[388]: Missed 10 kernel messages Nov 16 11:53:01.141193 localhost kernel: Loading iSCSI transport class v2.0-870. Nov 16 11:53:01.141200 localhost systemd-journald[388]: Journal started Nov 16 11:53:01.141242 localhost systemd-journald[388]: Runtime journal (/run/log/journal/5bd686cc3f7e4a69b544a82ce52123a1) is 8.0M, max 801.1M, 793.1M free. Nov 16 11:53:01.424887 localhost systemd[1]: Started Create list of required static device nodes for the current kernel. Nov 16 11:53:01.431042 localhost systemd[1]: systemd-vconsole-setup.service: Succeeded. Nov 16 11:53:01.431326 localhost systemd[1]: Started Setup Virtual Console. Nov 16 11:53:01.454532 localhost systemd[1]: memstrack.service: Succeeded. Nov 16 11:53:01.455331 localhost systemd[1]: Started CoreOS: Touch /run/agetty.reload. Nov 16 11:53:01.477594 localhost systemd[1]: Starting dracut ask for additional cmdline parameters... Nov 16 11:53:01.503988 localhost systemd-journald[388]: Missed 6 kernel messages Nov 16 11:53:01.503988 localhost kernel: fuse: init (API version 7.31) Nov 16 11:53:01.492789 localhost systemd[1]: Starting Create Static Device Nodes in /dev... Nov 16 11:53:01.503926 localhost systemd-modules-load[406]: Inserted module 'fuse' Nov 16 11:53:01.504000 localhost systemd-modules-load[406]: Module 'msr' is builtin Nov 16 11:53:01.505114 localhost systemd[1]: Started Load Kernel Modules. Nov 16 11:53:01.518019 localhost systemd[1]: Started dracut ask for additional cmdline parameters. Nov 16 11:53:01.538052 localhost systemd[1]: Starting dracut cmdline hook... Nov 16 11:53:01.547650 localhost systemd[1]: Starting Apply Kernel Variables... Nov 16 11:53:01.554711 localhost dracut-cmdline[424]: dracut-410.84.202211032138-0 dracut-049-137.git20220131.el8_4.1 Nov 16 11:53:01.558339 localhost dracut-cmdline[424]: Using kernel command line parameters: rd.driver.pre=dm_multipath BOOT_IMAGE=(hd0,gpt3)/ostree/rhcos-7e6edd7ff4609c0b1e388de010a2ee1ae8883be1667b7adf26f17cf24f0b3760/vmlinuz-4.18.0-305.65.1.el8_4.x86_64 random.trust_cpu=on console=tty0 console=ttyS0,115200n8 ostree=/ostree/boot.0/rhcos/7e6edd7ff4609c0b1e388de010a2ee1ae8883be1667b7adf26f17cf24f0b3760/0 ignition.platform.id=azure root=UUID=566864e3-acbe-4961-8488-233c4e33ce96 rw rootflags=prjquota boot=UUID=4a20a463-024a-4bb3-afd8-31decc06c38d Nov 16 11:53:01.565919 localhost systemd[1]: Started Create Static Device Nodes in /dev. Nov 16 11:53:01.622001 localhost systemd[1]: Started Apply Kernel Variables. Nov 16 11:53:01.724325 localhost systemd-journald[388]: Missed 5 kernel messages Nov 16 11:53:01.724325 localhost kernel: iscsi: registered transport (tcp) Nov 16 11:53:01.774565 localhost kernel: iscsi: registered transport (qla4xxx) Nov 16 11:53:01.774638 localhost kernel: QLogic iSCSI HBA Driver Nov 16 11:53:01.792319 localhost kernel: libcxgbi:libcxgbi_init_module: Chelsio iSCSI driver library libcxgbi v0.9.1-ko (Apr. 2015) Nov 16 11:53:01.838874 localhost kernel: Chelsio T4-T6 iSCSI Driver cxgb4i v0.9.5-ko (Apr. 2015) Nov 16 11:53:01.838928 localhost kernel: iscsi: registered transport (cxgb4i) Nov 16 11:53:01.852370 localhost kernel: cnic: QLogic cnicDriver v2.5.22 (July 20, 2015) Nov 16 11:53:01.871808 localhost kernel: QLogic NetXtreme II iSCSI Driver bnx2i v2.7.10.1 (Jul 16, 2014) Nov 16 11:53:01.871869 localhost kernel: iscsi: registered transport (bnx2i) Nov 16 11:53:01.893238 localhost kernel: iscsi: registered transport (be2iscsi) Nov 16 11:53:01.893287 localhost kernel: In beiscsi_module_init, tt=00000000dde2648e Nov 16 11:53:02.101529 localhost systemd[1]: Started dracut cmdline hook. Nov 16 11:53:02.114432 localhost systemd[1]: Starting dracut pre-udev hook... Nov 16 11:53:02.177328 localhost systemd-journald[388]: Missed 2 kernel messages Nov 16 11:53:02.177328 localhost kernel: device-mapper: uevent: version 1.0.3 Nov 16 11:53:02.188317 localhost kernel: device-mapper: ioctl: 4.43.0-ioctl (2020-10-01) initialised: dm-devel@redhat.com Nov 16 11:53:02.259599 localhost systemd[1]: Started dracut pre-udev hook. Nov 16 11:53:02.270721 localhost systemd[1]: Starting udev Kernel Device Manager... Nov 16 11:53:02.374328 localhost systemd[1]: Started udev Kernel Device Manager. Nov 16 11:53:02.387594 localhost systemd[1]: Starting dracut pre-trigger hook... Nov 16 11:53:02.417037 localhost dracut-pre-trigger[592]: rd.md=0: removing MD RAID activation Nov 16 11:53:02.639151 localhost systemd[1]: Started dracut pre-trigger hook. Nov 16 11:53:02.649473 localhost systemd[1]: Starting udev Coldplug all Devices... Nov 16 11:53:02.743189 localhost systemd[1]: Mounting Kernel Configuration File System... Nov 16 11:53:02.753960 localhost systemd[1]: Mounted Kernel Configuration File System. Nov 16 11:53:02.841503 localhost systemd[1]: Started udev Coldplug all Devices. Nov 16 11:53:02.851995 localhost systemd[1]: Starting udev Wait for Complete Device Initialization... Nov 16 11:53:02.899335 localhost systemd-journald[388]: Missed 11 kernel messages Nov 16 11:53:02.899335 localhost kernel: hv_vmbus: Vmbus version:4.0 Nov 16 11:53:02.907346 localhost kernel: libata version 3.00 loaded. Nov 16 11:53:02.921131 localhost systemd-udevd[666]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:53:02.935170 localhost kernel: ata_piix 0000:00:07.1: version 2.13 Nov 16 11:53:02.935385 localhost kernel: ata_piix 0000:00:07.1: Hyper-V Virtual Machine detected, ATA device ignore set Nov 16 11:53:02.935552 localhost systemd-journald[388]: Missed 1 kernel messages Nov 16 11:53:02.935552 localhost kernel: scsi host0: ata_piix Nov 16 11:53:02.945477 localhost kernel: scsi host1: ata_piix Nov 16 11:53:02.945700 localhost kernel: ata1: PATA max UDMA/33 cmd 0x1f0 ctl 0x3f6 bmdma 0xffa0 irq 14 Nov 16 11:53:02.954152 localhost kernel: ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0xffa8 irq 15 Nov 16 11:53:02.971213 localhost kernel: hv_vmbus: registering driver hv_storvsc Nov 16 11:53:02.978512 localhost kernel: hv_vmbus: registering driver hyperv_keyboard Nov 16 11:53:02.979598 localhost kernel: hv_vmbus: registering driver hid_hyperv Nov 16 11:53:02.979631 localhost kernel: scsi host2: storvsc_host_t Nov 16 11:53:02.979681 localhost kernel: scsi host4: storvsc_host_t Nov 16 11:53:02.983325 localhost kernel: scsi host5: storvsc_host_t Nov 16 11:53:02.986363 localhost kernel: scsi host3: storvsc_host_t Nov 16 11:53:02.986423 localhost kernel: input: Microsoft Vmbus HID-compliant Mouse as /devices/0006:045E:0621.0001/input/input4 Nov 16 11:53:02.986446 localhost kernel: hid-generic 0006:045E:0621.0001: input: HID v0.01 Mouse [Microsoft Vmbus HID-compliant Mouse] on Nov 16 11:53:02.996393 localhost kernel: scsi: waiting for bus probes to complete ... Nov 16 11:53:03.001312 localhost kernel: scsi: waiting for bus probes to complete ... Nov 16 11:53:03.030227 localhost kernel: scsi 2:0:0:0: Direct-Access Msft Virtual Disk 1.0 PQ: 0 ANSI: 5 Nov 16 11:53:03.030396 localhost kernel: scsi 3:0:1:0: Direct-Access Msft Virtual Disk 1.0 PQ: 0 ANSI: 5 Nov 16 11:53:03.064463 localhost kernel: hv_vmbus: registering driver hv_netvsc Nov 16 11:53:03.092494 localhost kernel: scsi 2:0:0:0: Attached scsi generic sg0 type 0 Nov 16 11:53:03.092924 localhost kernel: scsi 3:0:1:0: Attached scsi generic sg1 type 0 Nov 16 11:53:03.099320 localhost kernel: sd 2:0:0:0: [sda] 536870912 512-byte logical blocks: (275 GB/256 GiB) Nov 16 11:53:03.099589 localhost kernel: sd 2:0:0:0: [sda] 4096-byte physical blocks Nov 16 11:53:03.099867 localhost kernel: sd 2:0:0:0: [sda] Write Protect is off Nov 16 11:53:03.100067 localhost kernel: sd 2:0:0:0: [sda] Mode Sense: 0f 00 10 00 Nov 16 11:53:03.100335 localhost kernel: sd 3:0:1:0: [sdb] 67108864 512-byte logical blocks: (34.4 GB/32.0 GiB) Nov 16 11:53:03.100528 localhost kernel: sd 3:0:1:0: [sdb] 4096-byte physical blocks Nov 16 11:53:03.100710 localhost kernel: sd 3:0:1:0: [sdb] Write Protect is off Nov 16 11:53:03.100897 localhost kernel: sd 3:0:1:0: [sdb] Mode Sense: 0f 00 10 00 Nov 16 11:53:03.101093 localhost kernel: sd 2:0:0:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA Nov 16 11:53:03.101273 localhost kernel: sd 3:0:1:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA Nov 16 11:53:03.177597 localhost systemd-udevd[671]: Using default interface naming scheme 'rhel-8.0'. Nov 16 11:53:03.422463 localhost systemd-journald[388]: Missed 1 kernel messages Nov 16 11:53:03.422463 localhost kernel: ata2.01: NODEV after polling detection Nov 16 11:53:03.475349 localhost kernel: ata1.01: host indicates ignore ATA devices, ignored Nov 16 11:53:03.475394 localhost kernel: sda: sda1 sda2 sda3 sda4 Nov 16 11:53:03.476353 localhost kernel: sd 2:0:0:0: [sda] Attached SCSI disk Nov 16 11:53:03.483432 localhost kernel: sdb: sdb1 Nov 16 11:53:03.490308 localhost kernel: ata2.00: ATAPI: Virtual CD, , max MWDMA2 Nov 16 11:53:03.499442 localhost kernel: sd 3:0:1:0: [sdb] Attached SCSI disk Nov 16 11:53:03.502356 localhost kernel: ata1.00: host indicates ignore ATA devices, ignored Nov 16 11:53:03.502390 localhost kernel: psmouse serio1: trackpoint: IBM TrackPoint firmware: 0x01, buttons: 3/3 Nov 16 11:53:03.503348 localhost kernel: input: TPPS/2 IBM TrackPoint as /devices/platform/i8042/serio1/input/input3 Nov 16 11:53:03.503382 localhost kernel: input: AT Translated Set 2 keyboard as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A03:00/device:07/VMBUS:01/d34b2567-b9b6-42b9-8778-0a4ec0b955bf/serio2/input/input5 Nov 16 11:53:03.616366 localhost kernel: scsi 1:0:0:0: CD-ROM Msft Virtual CD/ROM 1.0 PQ: 0 ANSI: 5 Nov 16 11:53:03.627136 localhost kernel: scsi 1:0:0:0: Attached scsi generic sg2 type 5 Nov 16 11:53:03.674555 localhost kernel: sr 1:0:0:0: [sr0] scsi3-mmc drive: 0x/0x tray Nov 16 11:53:03.674865 localhost kernel: cdrom: Uniform CD-ROM driver Revision: 3.20 Nov 16 11:53:03.686308 localhost kernel: sr 1:0:0:0: Attached scsi CD-ROM sr0 Nov 16 11:53:03.764986 localhost systemd[1]: Found device Virtual_Disk root. Nov 16 11:53:03.774857 localhost systemd[1]: Found device Virtual_Disk root. Nov 16 11:53:03.833439 localhost systemd[1]: Reached target Initrd Root Device. Nov 16 11:53:03.903832 localhost systemd-udevd[671]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:53:04.274508 localhost systemd[1]: Started udev Wait for Complete Device Initialization. Nov 16 11:53:04.287659 localhost systemd[1]: Starting Device-Mapper Multipath Device Controller... Nov 16 11:53:04.311373 localhost systemd[1]: Started Device-Mapper Multipath Device Controller. Nov 16 11:53:04.324153 localhost systemd[1]: Starting Open-iSCSI... Nov 16 11:53:04.331220 localhost systemd[1]: Reached target Local File Systems (Pre). Nov 16 11:53:04.342849 localhost systemd[1]: Reached target Local File Systems. Nov 16 11:53:04.355483 localhost systemd[1]: Starting Create Volatile Files and Directories... Nov 16 11:53:04.372427 localhost multipathd[733]: --------start up-------- Nov 16 11:53:04.372427 localhost multipathd[733]: read /etc/multipath.conf Nov 16 11:53:04.388380 localhost multipathd[733]: /etc/multipath.conf does not exist, blacklisting all devices. Nov 16 11:53:04.388380 localhost multipathd[733]: You can run "/sbin/mpathconf --enable" to create Nov 16 11:53:04.388380 localhost multipathd[733]: /etc/multipath.conf. See man mpathconf(8) for more details Nov 16 11:53:04.388380 localhost multipathd[733]: path checkers start up Nov 16 11:53:04.388380 localhost multipathd[733]: /etc/multipath.conf does not exist, blacklisting all devices. Nov 16 11:53:04.388380 localhost multipathd[733]: You can run "/sbin/mpathconf --enable" to create Nov 16 11:53:04.388380 localhost multipathd[733]: /etc/multipath.conf. See man mpathconf(8) for more details Nov 16 11:53:04.406655 localhost systemd[1]: Started Open-iSCSI. Nov 16 11:53:04.457168 localhost iscsid[734]: iscsid: can't open InitiatorName configuration file /etc/iscsi/initiatorname.iscsi Nov 16 11:53:04.457168 localhost iscsid[734]: iscsid: Warning: InitiatorName file /etc/iscsi/initiatorname.iscsi does not exist or does not contain a properly formatted InitiatorName. If using software iscsi (iscsi_tcp or ib_iser) or partial offload (bnx2i or cxgbi iscsi), you may not be able to log into or discover targets. Please create a file /etc/iscsi/initiatorname.iscsi that contains a sting with the format: InitiatorName=iqn.yyyy-mm.[:identifier]. Nov 16 11:53:04.457168 localhost iscsid[734]: Example: InitiatorName=iqn.2001-04.com.redhat:fc6. Nov 16 11:53:04.457168 localhost iscsid[734]: If using hardware iscsi like qla4xxx this message can be ignored. Nov 16 11:53:04.457168 localhost iscsid[734]: iscsid: can't open InitiatorAlias configuration file /etc/iscsi/initiatorname.iscsi Nov 16 11:53:04.442417 localhost systemd[1]: Started Create Volatile Files and Directories. Nov 16 11:53:04.516841 localhost systemd[1]: Reached target System Initialization. Nov 16 11:53:04.548833 localhost systemd[1]: Reached target Basic System. Nov 16 11:53:04.565100 localhost systemd[1]: Starting CoreOS Enable Network... Nov 16 11:53:04.797547 localhost systemd[1]: Started CoreOS Enable Network. Nov 16 11:53:04.808888 localhost systemd[1]: Starting dracut initqueue hook... Nov 16 11:53:05.201080 localhost NetworkManager[778]: [1668599585.2010] NetworkManager (version 1.30.0-16.el8_4) is starting... (for the first time) Nov 16 11:53:05.201121 localhost NetworkManager[778]: [1668599585.2010] Read config: /etc/NetworkManager/NetworkManager.conf Nov 16 11:53:05.226161 localhost systemd-journald[388]: Missed 34 kernel messages Nov 16 11:53:05.226161 localhost kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 11:53:05.202708 localhost NetworkManager[778]: [1668599585.2026] auth[0x55aa7259eea0]: create auth-manager: D-Bus connection not available. Polkit is disabled and only root will be authorized. Nov 16 11:53:05.203645 localhost NetworkManager[778]: [1668599585.2036] manager[0x55aa725cd020]: monitoring kernel firmware directory '/lib/firmware'. Nov 16 11:53:05.266154 localhost systemd-journald[388]: Missed 2 kernel messages Nov 16 11:53:05.266154 localhost kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 11:53:05.203785 localhost NetworkManager[778]: [1668599585.2037] hostname: hostname: hostnamed not used as proxy creation failed with: Could not connect: No such file or directory Nov 16 11:53:05.287662 localhost systemd-journald[388]: Missed 1 kernel messages Nov 16 11:53:05.287662 localhost kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 11:53:05.203925 localhost NetworkManager[778]: [1668599585.2039] dns-mgr[0x55aa725bc1d0]: init: dns=default,systemd-resolved rc-manager=symlink Nov 16 11:53:05.215744 localhost NetworkManager[778]: [1668599585.2157] Loaded device plugin: NMTeamFactory (/usr/lib64/NetworkManager/1.30.0-16.el8_4/libnm-device-plugin-team.so) Nov 16 11:53:05.215758 localhost NetworkManager[778]: [1668599585.2157] manager: rfkill: Wi-Fi enabled by radio killswitch; enabled by state file Nov 16 11:53:05.215766 localhost NetworkManager[778]: [1668599585.2157] manager: rfkill: WWAN enabled by radio killswitch; enabled by state file Nov 16 11:53:05.215773 localhost NetworkManager[778]: [1668599585.2157] manager: Networking is enabled by state file Nov 16 11:53:05.215788 localhost NetworkManager[778]: [1668599585.2157] dhcp-init: Using DHCP client 'internal' Nov 16 11:53:05.216105 localhost NetworkManager[778]: [1668599585.2160] ifcfg-rh: dbus: don't use D-Bus for com.redhat.ifcfgrh1 service Nov 16 11:53:05.216116 localhost NetworkManager[778]: [1668599585.2161] settings: Loaded settings plugin: ifcfg-rh ("/usr/lib64/NetworkManager/1.30.0-16.el8_4/libnm-settings-plugin-ifcfg-rh.so") Nov 16 11:53:05.216133 localhost NetworkManager[778]: [1668599585.2161] settings: Loaded settings plugin: keyfile (internal) Nov 16 11:53:05.217364 localhost NetworkManager[778]: [1668599585.2173] device (lo): carrier: link connected Nov 16 11:53:05.217398 localhost NetworkManager[778]: [1668599585.2173] manager: (lo): new Generic device (/org/freedesktop/NetworkManager/Devices/1) Nov 16 11:53:05.217822 localhost NetworkManager[778]: [1668599585.2178] manager: (eth0): new Ethernet device (/org/freedesktop/NetworkManager/Devices/2) Nov 16 11:53:05.217942 localhost NetworkManager[778]: [1668599585.2179] device (eth0): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external') Nov 16 11:53:05.294757 localhost NetworkManager[778]: [1668599585.2947] device (eth0): carrier: link connected Nov 16 11:53:05.295256 localhost NetworkManager[778]: [1668599585.2952] sleep-monitor-sd: failed to acquire D-Bus proxy: Could not connect: No such file or directory Nov 16 11:53:05.295456 localhost NetworkManager[778]: [1668599585.2954] device (eth0): state change: unavailable -> disconnected (reason 'none', sys-iface-state: 'managed') Nov 16 11:53:05.295774 localhost NetworkManager[778]: [1668599585.2957] policy: auto-activating connection 'Wired Connection' (57a7b822-94c8-4a42-8bfc-ecb46b108a8a) Nov 16 11:53:05.296069 localhost NetworkManager[778]: [1668599585.2960] device (eth0): Activation: starting connection 'Wired Connection' (57a7b822-94c8-4a42-8bfc-ecb46b108a8a) Nov 16 11:53:05.296092 localhost NetworkManager[778]: [1668599585.2960] device (eth0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed') Nov 16 11:53:05.296117 localhost NetworkManager[778]: [1668599585.2961] manager: NetworkManager state is now CONNECTING Nov 16 11:53:05.296149 localhost NetworkManager[778]: [1668599585.2961] device (eth0): state change: prepare -> config (reason 'none', sys-iface-state: 'managed') Nov 16 11:53:05.296287 localhost NetworkManager[778]: [1668599585.2962] device (eth0): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed') Nov 16 11:53:05.296415 localhost NetworkManager[778]: [1668599585.2964] dhcp4 (eth0): activation: beginning transaction (timeout in 90 seconds) Nov 16 11:53:05.348844 localhost NetworkManager[778]: [1668599585.3488] dhcp4 (eth0): state changed unknown -> bound, address=10.0.1.4 Nov 16 11:53:05.363902 localhost NetworkManager[778]: [1668599585.3638] device (eth0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'managed') Nov 16 11:53:05.363963 localhost NetworkManager[778]: [1668599585.3639] device (eth0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'managed') Nov 16 11:53:05.363993 localhost NetworkManager[778]: [1668599585.3639] device (eth0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'managed') Nov 16 11:53:05.364317 localhost NetworkManager[778]: [1668599585.3643] manager: NetworkManager state is now CONNECTED_LOCAL Nov 16 11:53:05.364700 localhost NetworkManager[778]: [1668599585.3646] manager: NetworkManager state is now CONNECTED_SITE Nov 16 11:53:05.364713 localhost NetworkManager[778]: [1668599585.3647] policy: set 'Wired Connection' (eth0) as default for IPv4 routing and DNS Nov 16 11:53:05.365371 localhost NetworkManager[778]: [1668599585.3653] device (eth0): Activation: successful, device activated. Nov 16 11:53:05.365457 localhost NetworkManager[778]: [1668599585.3654] manager: NetworkManager state is now CONNECTED_GLOBAL Nov 16 11:53:05.365870 localhost NetworkManager[778]: [1668599585.3658] manager: startup complete Nov 16 11:53:05.365887 localhost NetworkManager[778]: [1668599585.3658] quitting now that startup is complete Nov 16 11:53:05.574554 localhost NetworkManager[778]: [1668599585.5745] dhcp4 (eth0): canceled DHCP transaction Nov 16 11:53:05.574576 localhost NetworkManager[778]: [1668599585.5745] dhcp4 (eth0): state changed bound -> done Nov 16 11:53:05.574611 localhost NetworkManager[778]: [1668599585.5746] device (eth0): DHCPv4: trying to acquire a new lease within 180 seconds Nov 16 11:53:05.574672 localhost NetworkManager[778]: [1668599585.5746] manager: NetworkManager state is now CONNECTED_SITE Nov 16 11:53:05.574770 localhost NetworkManager[778]: [1668599585.5747] exiting (success) Nov 16 11:53:05.609499 localhost systemd[1]: Started dracut initqueue hook. Nov 16 11:53:05.654649 localhost systemd[1]: Starting dracut pre-mount hook... Nov 16 11:53:05.685966 localhost systemd[1]: Reached target Remote File Systems (Pre). Nov 16 11:53:05.717258 localhost systemd[1]: Reached target Remote File Systems. Nov 16 11:53:05.750003 localhost systemd[1]: Started dracut pre-mount hook. Nov 16 11:53:05.791805 localhost systemd[1]: Starting File System Check on /dev/disk/by-uuid/566864e3-acbe-4961-8488-233c4e33ce96... Nov 16 11:53:05.869718 localhost systemd-fsck[803]: /usr/sbin/fsck.xfs: XFS file system. Nov 16 11:53:05.876810 localhost systemd[1]: Started File System Check on /dev/disk/by-uuid/566864e3-acbe-4961-8488-233c4e33ce96. Nov 16 11:53:05.892694 localhost systemd[1]: Mounting /sysroot... Nov 16 11:53:06.021313 localhost systemd-journald[388]: Missed 48 kernel messages Nov 16 11:53:06.021313 localhost kernel: SGI XFS with ACLs, security attributes, quota, no debug enabled Nov 16 11:53:06.031367 localhost kernel: XFS (sda4): Mounting V5 Filesystem Nov 16 11:53:06.110369 localhost kernel: XFS (sda4): Ending clean mount Nov 16 11:53:06.233446 localhost systemd[1]: Mounted /sysroot. Nov 16 11:53:06.242249 localhost systemd[1]: Starting OSTree Prepare OS/... Nov 16 11:53:06.294082 localhost ostree-prepare-root[821]: preparing sysroot at /sysroot Nov 16 11:53:06.295402 localhost ostree-prepare-root[821]: Resolved OSTree target to: /sysroot/ostree/deploy/rhcos/deploy/8d68051abeecf4991283052b4e6d38a5bad50e3c8e7b678973d5c39d30145ab0.0 Nov 16 11:53:06.295675 localhost ostree-prepare-root[821]: filesystem at /sysroot currently writable: 1 Nov 16 11:53:06.295680 localhost ostree-prepare-root[821]: sysroot.readonly configuration value: 1 Nov 16 11:53:06.305528 localhost systemd[1]: sysroot-ostree-deploy-rhcos-deploy-8d68051abeecf4991283052b4e6d38a5bad50e3c8e7b678973d5c39d30145ab0.0.mount: Succeeded. Nov 16 11:53:06.306031 localhost systemd[1]: Started OSTree Prepare OS/. Nov 16 11:53:06.321584 localhost systemd[1]: Reached target Initrd Root File System. Nov 16 11:53:06.334226 localhost systemd[1]: Reached target Subsequent (Not Ignition) boot complete. Nov 16 11:53:06.340962 localhost systemd[1]: Starting Reload Configuration from the Real Root... Nov 16 11:53:06.412886 localhost systemd[1]: Reloading. Nov 16 11:53:06.790007 localhost systemd[1]: Stopping Device-Mapper Multipath Device Controller... Nov 16 11:53:06.801433 localhost multipathd[733]: exit (signal) Nov 16 11:53:06.801433 localhost multipathd[733]: --------shut down------- Nov 16 11:53:06.801917 localhost systemd[1]: multipathd.service: Succeeded. Nov 16 11:53:06.802232 localhost systemd[1]: Stopped Device-Mapper Multipath Device Controller. Nov 16 11:53:06.818771 localhost systemd[1]: initrd-parse-etc.service: Succeeded. Nov 16 11:53:06.819088 localhost systemd[1]: Started Reload Configuration from the Real Root. Nov 16 11:53:06.837809 localhost systemd[1]: Reached target Initrd File Systems. Nov 16 11:53:06.855693 localhost systemd[1]: Reached target Initrd Default Target. Nov 16 11:53:06.866956 localhost systemd[1]: Starting dracut pre-pivot and cleanup hook... Nov 16 11:53:07.043684 localhost dracut-pre-pivot[902]: Nov 16 11:53:07 | /etc/multipath.conf does not exist, blacklisting all devices. Nov 16 11:53:07.043684 localhost dracut-pre-pivot[902]: Nov 16 11:53:07 | You can run "/sbin/mpathconf --enable" to create Nov 16 11:53:07.043684 localhost dracut-pre-pivot[902]: Nov 16 11:53:07 | /etc/multipath.conf. See man mpathconf(8) for more details Nov 16 11:53:07.062313 localhost systemd[1]: Started dracut pre-pivot and cleanup hook. Nov 16 11:53:07.071440 localhost systemd[1]: Starting Cleaning Up and Shutting Down Daemons... Nov 16 11:53:07.148763 localhost systemd[1]: dracut-pre-pivot.service: Succeeded. Nov 16 11:53:07.149135 localhost systemd[1]: Stopped dracut pre-pivot and cleanup hook. Nov 16 11:53:07.166604 localhost systemd[1]: Stopped target Remote File Systems. Nov 16 11:53:07.177269 localhost systemd[1]: Stopped target Initrd Default Target. Nov 16 11:53:07.187529 localhost systemd[1]: Stopped target Timers. Nov 16 11:53:07.196150 localhost systemd[1]: Stopped target Initrd Root Device. Nov 16 11:53:07.206967 localhost systemd[1]: clevis-luks-askpass.path: Succeeded. Nov 16 11:53:07.207058 localhost systemd[1]: Stopped Forward Password Requests to Clevis Directory Watch. Nov 16 11:53:07.227341 localhost systemd[1]: dracut-pre-mount.service: Succeeded. Nov 16 11:53:07.227844 localhost systemd[1]: Stopped dracut pre-mount hook. Nov 16 11:53:07.245673 localhost systemd[1]: coreos-touch-run-agetty.service: Succeeded. Nov 16 11:53:07.246111 localhost systemd[1]: Stopped CoreOS: Touch /run/agetty.reload. Nov 16 11:53:07.264609 localhost systemd[1]: Stopped target Subsequent (Not Ignition) boot complete. Nov 16 11:53:07.283464 localhost systemd[1]: Stopped target Ignition Subsequent Boot Disk Setup. Nov 16 11:53:07.294835 localhost systemd[1]: Stopped target Remote File Systems (Pre). Nov 16 11:53:07.320983 localhost systemd[1]: dracut-initqueue.service: Succeeded. Nov 16 11:53:07.321423 localhost systemd[1]: Stopped dracut initqueue hook. Nov 16 11:53:07.342115 localhost systemd[1]: coreos-enable-network.service: Succeeded. Nov 16 11:53:07.342463 localhost systemd[1]: Stopped CoreOS Enable Network. Nov 16 11:53:07.372016 localhost systemd[1]: Stopped target Basic System. Nov 16 11:53:07.381148 localhost systemd[1]: Stopped target System Initialization. Nov 16 11:53:07.394914 localhost systemd[1]: Stopped target Swap. Nov 16 11:53:07.403784 localhost systemd[1]: systemd-tmpfiles-setup.service: Succeeded. Nov 16 11:53:07.404135 localhost systemd[1]: Stopped Create Volatile Files and Directories. Nov 16 11:53:07.421468 localhost systemd[1]: systemd-udev-settle.service: Succeeded. Nov 16 11:53:07.421801 localhost systemd[1]: Stopped udev Wait for Complete Device Initialization. Nov 16 11:53:07.445702 localhost systemd[1]: systemd-sysctl.service: Succeeded. Nov 16 11:53:07.446015 localhost systemd[1]: Stopped Apply Kernel Variables. Nov 16 11:53:07.461495 localhost systemd[1]: Stopped target Local File Systems. Nov 16 11:53:07.472331 localhost systemd[1]: Stopped target Local File Systems (Pre). Nov 16 11:53:07.484080 localhost systemd[1]: Stopped target Local Encrypted Volumes. Nov 16 11:53:07.494438 localhost systemd[1]: systemd-modules-load.service: Succeeded. Nov 16 11:53:07.494761 localhost systemd[1]: Stopped Load Kernel Modules. Nov 16 11:53:07.513238 localhost systemd[1]: Stopped target Paths. Nov 16 11:53:07.520057 localhost systemd[1]: systemd-ask-password-console.path: Succeeded. Nov 16 11:53:07.523391 localhost systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. Nov 16 11:53:07.533186 localhost systemd[1]: Stopped target Slices. Nov 16 11:53:07.552384 localhost systemd[1]: systemd-udev-trigger.service: Succeeded. Nov 16 11:53:07.552690 localhost systemd[1]: Stopped udev Coldplug all Devices. Nov 16 11:53:07.571014 localhost systemd[1]: Stopping Open-iSCSI... Nov 16 11:53:07.578522 localhost iscsid[734]: iscsid: iscsid shutting down. Nov 16 11:53:07.578551 localhost systemd[1]: dracut-pre-trigger.service: Succeeded. Nov 16 11:53:07.578865 localhost systemd[1]: Stopped dracut pre-trigger hook. Nov 16 11:53:07.596197 localhost systemd[1]: Stopping udev Kernel Device Manager... Nov 16 11:53:07.614570 localhost systemd[1]: Stopped target Sockets. Nov 16 11:53:07.624300 localhost systemd[1]: iscsid.service: Succeeded. Nov 16 11:53:07.624624 localhost systemd[1]: Stopped Open-iSCSI. Nov 16 11:53:07.638245 localhost systemd[1]: systemd-udevd.service: Succeeded. Nov 16 11:53:07.638578 localhost systemd[1]: Stopped udev Kernel Device Manager. Nov 16 11:53:07.656035 localhost systemd[1]: systemd-tmpfiles-setup-dev.service: Succeeded. Nov 16 11:53:07.656336 localhost systemd[1]: Stopped Create Static Device Nodes in /dev. Nov 16 11:53:07.678034 localhost systemd[1]: kmod-static-nodes.service: Succeeded. Nov 16 11:53:07.678428 localhost systemd[1]: Stopped Create list of required static device nodes for the current kernel. Nov 16 11:53:07.700505 localhost systemd[1]: dracut-pre-udev.service: Succeeded. Nov 16 11:53:07.700838 localhost systemd[1]: Stopped dracut pre-udev hook. Nov 16 11:53:07.717947 localhost systemd[1]: systemd-udevd-control.socket: Succeeded. Nov 16 11:53:07.718341 localhost systemd[1]: Closed udev Control Socket. Nov 16 11:53:07.736026 localhost systemd[1]: systemd-udevd-kernel.socket: Succeeded. Nov 16 11:53:07.736468 localhost systemd[1]: Closed udev Kernel Socket. Nov 16 11:53:07.752879 localhost systemd[1]: Starting Cleanup udevd DB... Nov 16 11:53:07.761445 localhost systemd[1]: Stopping iSCSI UserSpace I/O driver... Nov 16 11:53:07.764842 localhost systemd[1]: dracut-cmdline.service: Succeeded. Nov 16 11:53:07.765152 localhost systemd[1]: Stopped dracut cmdline hook. Nov 16 11:53:07.780815 localhost systemd[1]: dracut-cmdline-ask.service: Succeeded. Nov 16 11:53:07.781111 localhost systemd[1]: Stopped dracut ask for additional cmdline parameters. Nov 16 11:53:07.799587 localhost systemd[1]: iscsid.socket: Succeeded. Nov 16 11:53:07.799910 localhost systemd[1]: Closed Open-iSCSI iscsid Socket. Nov 16 11:53:07.817106 localhost systemd[1]: iscsiuio.service: Succeeded. Nov 16 11:53:07.817464 localhost systemd[1]: Stopped iSCSI UserSpace I/O driver. Nov 16 11:53:07.846719 localhost systemd[1]: initrd-cleanup.service: Succeeded. Nov 16 11:53:07.847053 localhost systemd[1]: Started Cleaning Up and Shutting Down Daemons. Nov 16 11:53:07.866139 localhost systemd[1]: initrd-udevadm-cleanup-db.service: Succeeded. Nov 16 11:53:07.866448 localhost systemd[1]: Started Cleanup udevd DB. Nov 16 11:53:07.884625 localhost systemd[1]: Reached target Switch Root. Nov 16 11:53:07.894560 localhost systemd[1]: Starting Switch Root... Nov 16 11:53:07.902522 localhost systemd[1]: iscsiuio.socket: Succeeded. Nov 16 11:53:07.902863 localhost systemd[1]: Closed Open-iSCSI iscsiuio Socket. Nov 16 11:53:07.963332 localhost systemd[1]: Switching root. Nov 16 11:53:07.982432 localhost systemd-journald[388]: Journal stopped Nov 16 11:53:08.705543 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted /sysroot. Nov 16 11:53:08.705572 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting OSTree Prepare OS/... Nov 16 11:53:08.705589 ropatil1611az-gs6zj-worker-westus-j4ngg ostree-prepare-root[821]: preparing sysroot at /sysroot Nov 16 11:53:08.705601 ropatil1611az-gs6zj-worker-westus-j4ngg ostree-prepare-root[821]: Resolved OSTree target to: /sysroot/ostree/deploy/rhcos/deploy/8d68051abeecf4991283052b4e6d38a5bad50e3c8e7b678973d5c39d30145ab0.0 Nov 16 11:53:08.705612 ropatil1611az-gs6zj-worker-westus-j4ngg ostree-prepare-root[821]: filesystem at /sysroot currently writable: 1 Nov 16 11:53:08.705622 ropatil1611az-gs6zj-worker-westus-j4ngg ostree-prepare-root[821]: sysroot.readonly configuration value: 1 Nov 16 11:53:08.705633 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-ostree-deploy-rhcos-deploy-8d68051abeecf4991283052b4e6d38a5bad50e3c8e7b678973d5c39d30145ab0.0.mount: Succeeded. Nov 16 11:53:08.705646 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started OSTree Prepare OS/. Nov 16 11:53:08.705657 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Initrd Root File System. Nov 16 11:53:08.705667 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Subsequent (Not Ignition) boot complete. Nov 16 11:53:08.705677 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Reload Configuration from the Real Root... Nov 16 11:53:08.705687 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reloading. Nov 16 11:53:08.705697 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Device-Mapper Multipath Device Controller... Nov 16 11:53:08.705707 ropatil1611az-gs6zj-worker-westus-j4ngg multipathd[733]: exit (signal) Nov 16 11:53:08.705722 ropatil1611az-gs6zj-worker-westus-j4ngg multipathd[733]: --------shut down------- Nov 16 11:53:08.705733 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: multipathd.service: Succeeded. Nov 16 11:53:08.705743 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Device-Mapper Multipath Device Controller. Nov 16 11:53:08.705752 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: initrd-parse-etc.service: Succeeded. Nov 16 11:53:08.705762 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Reload Configuration from the Real Root. Nov 16 11:53:08.705772 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Initrd File Systems. Nov 16 11:53:08.705785 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Initrd Default Target. Nov 16 11:53:08.705796 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting dracut pre-pivot and cleanup hook... Nov 16 11:53:08.705806 ropatil1611az-gs6zj-worker-westus-j4ngg dracut-pre-pivot[902]: Nov 16 11:53:07 | /etc/multipath.conf does not exist, blacklisting all devices. Nov 16 11:53:08.705817 ropatil1611az-gs6zj-worker-westus-j4ngg dracut-pre-pivot[902]: Nov 16 11:53:07 | You can run "/sbin/mpathconf --enable" to create Nov 16 11:53:08.705827 ropatil1611az-gs6zj-worker-westus-j4ngg dracut-pre-pivot[902]: Nov 16 11:53:07 | /etc/multipath.conf. See man mpathconf(8) for more details Nov 16 11:53:08.705837 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started dracut pre-pivot and cleanup hook. Nov 16 11:53:08.705847 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Cleaning Up and Shutting Down Daemons... Nov 16 11:53:08.705857 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-pre-pivot.service: Succeeded. Nov 16 11:53:08.705870 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut pre-pivot and cleanup hook. Nov 16 11:53:08.705880 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Remote File Systems. Nov 16 11:53:08.705891 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Initrd Default Target. Nov 16 11:53:08.705902 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Timers. Nov 16 11:53:08.705911 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Initrd Root Device. Nov 16 11:53:08.705921 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: clevis-luks-askpass.path: Succeeded. Nov 16 11:53:08.705931 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Forward Password Requests to Clevis Directory Watch. Nov 16 11:53:08.705941 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-pre-mount.service: Succeeded. Nov 16 11:53:08.705951 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut pre-mount hook. Nov 16 11:53:08.705965 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-touch-run-agetty.service: Succeeded. Nov 16 11:53:08.705975 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped CoreOS: Touch /run/agetty.reload. Nov 16 11:53:08.705985 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Subsequent (Not Ignition) boot complete. Nov 16 11:53:08.705999 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Ignition Subsequent Boot Disk Setup. Nov 16 11:53:08.706009 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Remote File Systems (Pre). Nov 16 11:53:08.706026 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-initqueue.service: Succeeded. Nov 16 11:53:08.706037 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut initqueue hook. Nov 16 11:53:08.706047 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: coreos-enable-network.service: Succeeded. Nov 16 11:53:08.706057 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped CoreOS Enable Network. Nov 16 11:53:08.706071 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Basic System. Nov 16 11:53:08.706082 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target System Initialization. Nov 16 11:53:08.706093 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Swap. Nov 16 11:53:08.706103 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-tmpfiles-setup.service: Succeeded. Nov 16 11:53:08.706114 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Create Volatile Files and Directories. Nov 16 11:53:08.706124 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-udev-settle.service: Succeeded. Nov 16 11:53:08.706137 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped udev Wait for Complete Device Initialization. Nov 16 11:53:08.706148 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-sysctl.service: Succeeded. Nov 16 11:53:08.706158 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Apply Kernel Variables. Nov 16 11:53:08.706168 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Local File Systems. Nov 16 11:53:08.706178 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Local File Systems (Pre). Nov 16 11:53:08.706188 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Local Encrypted Volumes. Nov 16 11:53:08.706203 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-modules-load.service: Succeeded. Nov 16 11:53:08.706213 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Load Kernel Modules. Nov 16 11:53:08.706223 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Paths. Nov 16 11:53:08.706243 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-ask-password-console.path: Succeeded. Nov 16 11:53:08.706253 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. Nov 16 11:53:08.706264 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Slices. Nov 16 11:53:08.706275 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-udev-trigger.service: Succeeded. Nov 16 11:53:08.706285 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped udev Coldplug all Devices. Nov 16 11:53:08.706331 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping Open-iSCSI... Nov 16 11:53:08.706342 ropatil1611az-gs6zj-worker-westus-j4ngg iscsid[734]: iscsid: iscsid shutting down. Nov 16 11:53:08.706356 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-pre-trigger.service: Succeeded. Nov 16 11:53:08.706366 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut pre-trigger hook. Nov 16 11:53:08.706379 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping udev Kernel Device Manager... Nov 16 11:53:08.706390 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Sockets. Nov 16 11:53:08.706401 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: iscsid.service: Succeeded. Nov 16 11:53:08.706411 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Open-iSCSI. Nov 16 11:53:08.706421 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-udevd.service: Succeeded. Nov 16 11:53:08.706431 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped udev Kernel Device Manager. Nov 16 11:53:08.706441 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-tmpfiles-setup-dev.service: Succeeded. Nov 16 11:53:08.706451 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Create Static Device Nodes in /dev. Nov 16 11:53:08.706461 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kmod-static-nodes.service: Succeeded. Nov 16 11:53:08.706472 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Create list of required static device nodes for the current kernel. Nov 16 11:53:08.706485 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-pre-udev.service: Succeeded. Nov 16 11:53:08.706495 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut pre-udev hook. Nov 16 11:53:08.706506 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-udevd-control.socket: Succeeded. Nov 16 11:53:08.706516 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Closed udev Control Socket. Nov 16 11:53:08.706527 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-udevd-kernel.socket: Succeeded. Nov 16 11:53:08.706537 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Closed udev Kernel Socket. Nov 16 11:53:08.706547 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Cleanup udevd DB... Nov 16 11:53:08.706574 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopping iSCSI UserSpace I/O driver... Nov 16 11:53:08.706585 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-cmdline.service: Succeeded. Nov 16 11:53:08.706601 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut cmdline hook. Nov 16 11:53:08.706611 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: dracut-cmdline-ask.service: Succeeded. Nov 16 11:53:08.706621 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped dracut ask for additional cmdline parameters. Nov 16 11:53:08.706631 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: iscsid.socket: Succeeded. Nov 16 11:53:08.706641 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Closed Open-iSCSI iscsid Socket. Nov 16 11:53:08.706651 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: iscsiuio.service: Succeeded. Nov 16 11:53:08.706661 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped iSCSI UserSpace I/O driver. Nov 16 11:53:08.706671 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: initrd-cleanup.service: Succeeded. Nov 16 11:53:08.706681 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Cleaning Up and Shutting Down Daemons. Nov 16 11:53:08.706694 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: initrd-udevadm-cleanup-db.service: Succeeded. Nov 16 11:53:08.706704 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Cleanup udevd DB. Nov 16 11:53:08.706714 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Switch Root. Nov 16 11:53:08.706723 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Switch Root... Nov 16 11:53:08.706733 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: iscsiuio.socket: Succeeded. Nov 16 11:53:08.706744 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Closed Open-iSCSI iscsiuio Socket. Nov 16 11:53:08.706753 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Switching root. Nov 16 11:53:08.706764 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: printk: systemd-journal: 6 output lines suppressed due to ratelimiting Nov 16 11:53:08.706774 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: printk: systemd: 21 output lines suppressed due to ratelimiting Nov 16 11:53:08.706787 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: audit: type=1404 audit(1668599588.042:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295 enabled=1 old-enabled=1 lsm=selinux res=1 Nov 16 11:53:08.706796 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: SELinux: policy capability network_peer_controls=1 Nov 16 11:53:08.706805 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: SELinux: policy capability open_perms=1 Nov 16 11:53:08.706817 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: SELinux: policy capability extended_socket_class=1 Nov 16 11:53:08.706827 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: SELinux: policy capability always_check_network=0 Nov 16 11:53:08.706837 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: SELinux: policy capability cgroup_seclabel=1 Nov 16 11:53:08.706846 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: SELinux: policy capability nnp_nosuid_transition=1 Nov 16 11:53:08.706855 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: audit: type=1403 audit(1668599588.243:3): auid=4294967295 ses=4294967295 lsm=selinux res=1 Nov 16 11:53:08.706868 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Successfully loaded SELinux policy in 203.367ms. Nov 16 11:53:08.706878 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Relabelled /dev, /run and /sys/fs/cgroup in 34.499ms. Nov 16 11:53:08.706889 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd 239 (239-45.el8_4.14) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) Nov 16 11:53:08.706905 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Detected virtualization microsoft. Nov 16 11:53:08.706915 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Detected architecture x86-64. Nov 16 11:53:08.706925 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Set hostname to . Nov 16 11:53:08.706938 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-platform-chrony: Updated chrony to use azure configuration /run/coreos-platform-chrony.conf Nov 16 11:53:08.706948 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: /usr/lib/systemd/system/bootupd.service:22: Unknown lvalue 'ProtectHostname' in section 'Service' Nov 16 11:53:08.706963 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journald.service: Succeeded. Nov 16 11:53:08.706973 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journald.service: Consumed 0 CPU time Nov 16 11:53:08.706983 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: initrd-switch-root.service: Succeeded. Nov 16 11:53:08.706996 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Switch Root. Nov 16 11:53:08.707007 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: initrd-switch-root.service: Consumed 0 CPU time Nov 16 11:53:08.707017 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. Nov 16 11:53:08.707027 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. Nov 16 11:53:08.707037 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped Journal Service. Nov 16 11:53:08.707050 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-journald.service: Consumed 0 CPU time Nov 16 11:53:08.707060 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Journal Service... Nov 16 11:53:08.707070 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting Kernel Debug File System... Nov 16 11:53:08.707083 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point. Nov 16 11:53:08.707093 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Swap. Nov 16 11:53:08.707103 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on udev Control Socket. Nov 16 11:53:08.707112 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Forward Password Requests to Wall Directory Watch. Nov 16 11:53:08.707125 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-journald[998]: Journal started Nov 16 11:53:08.707191 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-journald[998]: Runtime journal (/run/log/journal/17c3dc7b18784633a4bd61ccb76c02a9) is 8.0M, max 801.1M, 793.1M free. Nov 16 11:53:08.736364 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on Process Core Dump Socket. Nov 16 11:53:08.748332 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice system-getty.slice. Nov 16 11:53:08.760998 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Switch Root. Nov 16 11:53:08.772783 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Initrd File Systems. Nov 16 11:53:08.789316 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Dispatch Password Requests to Console Directory Watch. Nov 16 11:53:08.809376 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting POSIX Message Queue File System... Nov 16 11:53:08.822156 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Remote File Systems. Nov 16 11:53:08.836416 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Load Kernel Modules... Nov 16 11:53:08.851946 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Create list of required static device nodes for the current kernel... Nov 16 11:53:08.857974 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-modules-load[1003]: Module 'msr' is builtin Nov 16 11:53:08.866525 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting Huge Pages File System... Nov 16 11:53:08.875611 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped target Initrd Root File System. Nov 16 11:53:08.876937 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-modules-load[1003]: Inserted module 'ip_tables' Nov 16 11:53:08.889348 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice system-sshd\x2dkeygen.slice. Nov 16 11:53:08.907309 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ostree-prepare-root.service: Succeeded. Nov 16 11:53:08.907356 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped OSTree Prepare OS/. Nov 16 11:53:08.921374 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ostree-prepare-root.service: Consumed 0 CPU time Nov 16 11:53:08.921420 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Synchronize afterburn-sshkeys@.service template instances. Nov 16 11:53:08.952098 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on udev Kernel Socket. Nov 16 11:53:08.963803 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting udev Coldplug all Devices... Nov 16 11:53:08.976446 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on RPCbind Server Activation Socket. Nov 16 11:53:08.989055 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target RPC Port Mapper. Nov 16 11:53:09.002345 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on initctl Compatibility Named Pipe. Nov 16 11:53:09.015350 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Host and Network Name Lookups. Nov 16 11:53:09.031205 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Forward Password Requests to Clevis Directory Watch. Nov 16 11:53:09.044389 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Local Encrypted Volumes (Pre). Nov 16 11:53:09.056366 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Remote Encrypted Volumes. Nov 16 11:53:09.070347 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice User and Session Slice. Nov 16 11:53:09.081094 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Slices. Nov 16 11:53:09.093708 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting Temporary Directory (/tmp)... Nov 16 11:53:09.104687 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on LVM2 poll daemon socket. Nov 16 11:53:09.117359 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-fsck-root.service: Succeeded. Nov 16 11:53:09.117404 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Stopped File System Check on Root Device. Nov 16 11:53:09.137350 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-fsck-root.service: Consumed 0 CPU time Nov 16 11:53:09.153203 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Rebuild Hardware Database... Nov 16 11:53:09.163574 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Create System Users... Nov 16 11:53:09.180180 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on Device-mapper event daemon FIFOs. Nov 16 11:53:09.205707 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling... Nov 16 11:53:09.221076 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice system-serial\x2dgetty.slice. Nov 16 11:53:09.231350 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Local Encrypted Volumes. Nov 16 11:53:09.246348 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice system-systemd\x2dfsck.slice. Nov 16 11:53:09.264316 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-sysroot.mount: Succeeded. Nov 16 11:53:09.281720 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-sysroot.mount: Consumed 0 CPU time Nov 16 11:53:09.281834 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-sysroot-ostree-deploy-rhcos-var.mount: Succeeded. Nov 16 11:53:09.300752 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-sysroot-ostree-deploy-rhcos-var.mount: Consumed 0 CPU time Nov 16 11:53:09.300797 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-etc.mount: Succeeded. Nov 16 11:53:09.301350 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-etc.mount: Consumed 0 CPU time Nov 16 11:53:09.331954 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-usr.mount: Succeeded. Nov 16 11:53:09.343340 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: sysroot-usr.mount: Consumed 0 CPU time Nov 16 11:53:09.351027 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Journal Service. Nov 16 11:53:09.355270 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted Kernel Debug File System. Nov 16 11:53:09.360120 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted POSIX Message Queue File System. Nov 16 11:53:09.366080 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Load Kernel Modules. Nov 16 11:53:09.372391 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Create list of required static device nodes for the current kernel. Nov 16 11:53:09.379898 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted Huge Pages File System. Nov 16 11:53:09.387032 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted Temporary Directory (/tmp). Nov 16 11:53:09.392807 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Create System Users. Nov 16 11:53:09.401040 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Create Static Device Nodes in /dev... Nov 16 11:53:09.406916 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Apply Kernel Variables... Nov 16 11:53:09.411262 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting FUSE Control File System... Nov 16 11:53:09.419853 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Apply Kernel Variables. Nov 16 11:53:09.425554 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted FUSE Control File System. Nov 16 11:53:09.431908 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Create Static Device Nodes in /dev. Nov 16 11:53:09.489813 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started udev Coldplug all Devices. Nov 16 11:53:09.496206 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting udev Wait for Complete Device Initialization... Nov 16 11:53:09.515929 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Rebuild Hardware Database. Nov 16 11:53:09.522046 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting udev Kernel Device Manager... Nov 16 11:53:09.571886 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started udev Kernel Device Manager. Nov 16 11:53:09.641497 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_utils: Registering HyperV Utility Driver Nov 16 11:53:09.641574 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_vmbus: registering driver hv_utils Nov 16 11:53:09.648342 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_utils: Heartbeat IC version 3.0 Nov 16 11:53:09.648394 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_utils: Shutdown IC version 3.2 Nov 16 11:53:09.661530 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_utils: TimeSync IC version 4.0 Nov 16 11:53:09.424179 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_vmbus: registering driver hyperv_fb Nov 16 11:53:09.428418 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hyperv_fb: Synthvid Version major 3, minor 5 Nov 16 11:53:09.430399 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[1028]: Using default interface naming scheme 'rhel-8.0'. Nov 16 11:53:09.431616 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[1028]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:53:09.437831 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_vmbus: registering driver hv_balloon Nov 16 11:53:09.437910 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hyperv_fb: Screen resolution: 1024x768, Color depth: 32 Nov 16 11:53:09.455484 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hyperv_fb: Unable to allocate enough contiguous physical memory on Gen 1 VM. Using MMIO instead. Nov 16 11:53:09.456155 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_balloon: Using Dynamic Memory protocol version 2.0 Nov 16 11:53:09.462599 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: input: PC Speaker as /devices/platform/pcspkr/input/input6 Nov 16 11:53:09.463147 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: piix4_smbus 0000:00:07.3: SMBus base address uninitialized - upgrade BIOS or use force_addr=0xaddr Nov 16 11:53:09.464153 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: Console: switching to colour frame buffer device 128x48 Nov 16 11:53:09.522911 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[1039]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:53:09.554155 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer Nov 16 11:53:09.586545 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling. Nov 16 11:53:09.589147 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: EDAC sbridge: Seeking for: PCI ID 8086:6fa0 Nov 16 11:53:09.589219 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: EDAC sbridge: Ver: 1.1.2 Nov 16 11:53:09.793002 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started udev Wait for Complete Device Initialization. Nov 16 11:53:09.799622 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Local File Systems (Pre). Nov 16 11:53:09.806517 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting File System Check on /dev/disk/by-uuid/4a20a463-024a-4bb3-afd8-31decc06c38d... Nov 16 11:53:09.812392 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var.mount: Directory /var to mount over is not empty, mounting anyway. Nov 16 11:53:09.813646 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting /var... Nov 16 11:53:09.819884 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted /var. Nov 16 11:53:09.825271 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting OSTree Remount OS/ Bind Mounts... Nov 16 11:53:09.827798 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-fsck[1095]: boot: clean, 325/98304 files, 225955/393216 blocks Nov 16 11:53:09.835876 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started OSTree Remount OS/ Bind Mounts. Nov 16 11:53:09.843136 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started File System Check on /dev/disk/by-uuid/4a20a463-024a-4bb3-afd8-31decc06c38d. Nov 16 11:53:09.853678 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounting CoreOS Dynamic Mount for /boot... Nov 16 11:53:09.858963 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Flush Journal to Persistent Storage... Nov 16 11:53:09.864565 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Load/Save Random Seed... Nov 16 11:53:09.871523 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Load/Save Random Seed. Nov 16 11:53:09.877499 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-journald[998]: Time spent on flushing to /var is 22.448ms for 956 entries. Nov 16 11:53:09.877499 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-journald[998]: System journal (/var/log/journal/17c3dc7b18784633a4bd61ccb76c02a9) is 8.0M, max 4.0G, 3.9G free. Nov 16 11:53:10.337632 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: EXT4-fs (sda3): mounted filesystem with ordered data mode. Opts: (null) Nov 16 11:53:09.915802 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Flush Journal to Persistent Storage. Nov 16 11:53:09.966979 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Mounted CoreOS Dynamic Mount for /boot. Nov 16 11:53:09.973878 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Local File Systems. Nov 16 11:53:09.979037 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Run update-ca-trust... Nov 16 11:53:09.986618 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Restore /run/initramfs on shutdown... Nov 16 11:53:09.992687 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Create Volatile Files and Directories... Nov 16 11:53:09.998831 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Rebuild Journal Catalog... Nov 16 11:53:10.004369 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Rebuild Dynamic Linker Cache... Nov 16 11:53:10.020239 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Restore /run/initramfs on shutdown. Nov 16 11:53:10.022704 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1121]: [/usr/lib/tmpfiles.d/pkg-dbus-daemon.conf:1] Duplicate line for path "/var/lib/dbus", ignoring. Nov 16 11:53:10.027429 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1121]: [/usr/lib/tmpfiles.d/tmp.conf:12] Duplicate line for path "/var/tmp", ignoring. Nov 16 11:53:10.027614 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1121]: [/usr/lib/tmpfiles.d/var.conf:14] Duplicate line for path "/var/log", ignoring. Nov 16 11:53:10.027679 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Rebuild Journal Catalog. Nov 16 11:53:10.027756 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1121]: [/usr/lib/tmpfiles.d/var.conf:19] Duplicate line for path "/var/cache", ignoring. Nov 16 11:53:10.027763 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1121]: [/usr/lib/tmpfiles.d/var.conf:21] Duplicate line for path "/var/lib", ignoring. Nov 16 11:53:10.027768 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1121]: [/usr/lib/tmpfiles.d/var.conf:23] Duplicate line for path "/var/spool", ignoring. Nov 16 11:53:10.331855 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1121]: "/home" already exists and is not a directory. Nov 16 11:53:10.331979 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[1121]: "/srv" already exists and is not a directory. Nov 16 11:53:10.421791 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Create Volatile Files and Directories. Nov 16 11:53:10.433335 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Rebuild Dynamic Linker Cache. Nov 16 11:53:10.441255 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Update is Completed... Nov 16 11:53:10.446119 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting RHCOS Fix SELinux Labeling For /usr/local/sbin... Nov 16 11:53:10.448499 ropatil1611az-gs6zj-worker-westus-j4ngg chcon[1130]: changing security context of '/usr/local/sbin' Nov 16 11:53:10.452301 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Security Auditing Service... Nov 16 11:53:10.459941 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Update is Completed. Nov 16 11:53:10.464252 ropatil1611az-gs6zj-worker-westus-j4ngg auditd[1134]: No plugins found, not dispatching events Nov 16 11:53:10.465101 ropatil1611az-gs6zj-worker-westus-j4ngg auditd[1134]: Init complete, auditd 3.0 listening for events (startup state enable) Nov 16 11:53:10.471363 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started RHCOS Fix SELinux Labeling For /usr/local/sbin. Nov 16 11:53:10.489324 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: /sbin/augenrules: No change Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: No rules Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: enabled 1 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: failure 1 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: pid 1134 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: rate_limit 0 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: backlog_limit 8192 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: lost 0 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: backlog 4 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: backlog_wait_time 60000 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: enabled 1 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: failure 1 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: pid 1134 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: rate_limit 0 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: backlog_limit 8192 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: lost 0 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: backlog 0 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: backlog_wait_time 60000 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: enabled 1 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: failure 1 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: pid 1134 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: rate_limit 0 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: backlog_limit 8192 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: lost 0 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: backlog 0 Nov 16 11:53:10.493208 ropatil1611az-gs6zj-worker-westus-j4ngg augenrules[1138]: backlog_wait_time 60000 Nov 16 11:53:10.500425 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Security Auditing Service. Nov 16 11:53:10.506438 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Update UTMP about System Boot/Shutdown... Nov 16 11:53:10.520565 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Update UTMP about System Boot/Shutdown. Nov 16 11:53:11.526290 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Run update-ca-trust. Nov 16 11:53:11.532730 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target System Initialization. Nov 16 11:53:11.538516 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started daily update of the root trust anchor for DNSSEC. Nov 16 11:53:11.545060 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started OSTree Monitor Staged Deployment. Nov 16 11:53:11.550735 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Daily Cleanup of Temporary Directories. Nov 16 11:53:11.558065 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on bootupd.socket. Nov 16 11:53:11.564339 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Listening on D-Bus System Message Bus Socket. Nov 16 11:53:11.569545 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Sockets. Nov 16 11:53:11.574578 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Monitor console-login-helper-messages runtime issue snippets directory for changes. Nov 16 11:53:11.581118 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Paths. Nov 16 11:53:11.585595 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Basic System. Nov 16 11:53:11.592648 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Check if Ignition config is provided... Nov 16 11:53:11.598813 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting System Security Services Daemon... Nov 16 11:53:11.604329 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting CRI-O Auto Update Script... Nov 16 11:53:11.609322 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started D-Bus System Message Bus. Nov 16 11:53:11.616184 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target sshd-keygen.target. Nov 16 11:53:11.622615 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Generate SSH keys snippet for display via console-login-helper-messages... Nov 16 11:53:11.629892 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started irqbalance daemon. Nov 16 11:53:11.637271 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Open vSwitch Database Unit... Nov 16 11:53:11.644547 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Generate console-login-helper-messages issue snippet... Nov 16 11:53:11.650270 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Daily rotation of log files. Nov 16 11:53:11.655167 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Timers. Nov 16 11:53:11.662063 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting NTP client/server... Nov 16 11:53:11.667510 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Generation of shadow ID ranges for CRI-O... Nov 16 11:53:11.673150 ropatil1611az-gs6zj-worker-westus-j4ngg chronyd[1211]: chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG) Nov 16 11:53:11.674637 ropatil1611az-gs6zj-worker-westus-j4ngg chronyd[1211]: Frequency -7.372 +/- 0.470 ppm read from /var/lib/chrony/drift Nov 16 11:53:11.675332 ropatil1611az-gs6zj-worker-westus-j4ngg chronyd[1211]: Using right/UTC timezone to obtain leap second data Nov 16 11:53:11.676939 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Check if Ignition config is provided. Nov 16 11:53:11.687066 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Generate SSH keys snippet for display via console-login-helper-messages. Nov 16 11:53:11.697550 ropatil1611az-gs6zj-worker-westus-j4ngg chown[1218]: /usr/bin/chown: cannot access '/var/run/openvswitch': No such file or directory Nov 16 11:53:11.703298 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1176]: [system] Successfully activated service 'org.freedesktop.systemd1' Nov 16 11:53:11.710509 ropatil1611az-gs6zj-worker-westus-j4ngg sssd[sssd][1169]: Starting up Nov 16 11:53:11.712121 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started NTP client/server. Nov 16 11:53:11.738203 ropatil1611az-gs6zj-worker-westus-j4ngg sssd[be[implicit_files]][1235]: Starting up Nov 16 11:53:11.746620 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-subid.service: Succeeded. Nov 16 11:53:11.747397 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Generation of shadow ID ranges for CRI-O. Nov 16 11:53:11.753622 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-subid.service: Consumed 31ms CPU time Nov 16 11:53:11.766544 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1173]: time="2022-11-16 11:53:11.766212774Z" level=info msg="Starting CRI-O, version: 1.23.3-20.rhaos4.10.git89344de.el8, git: ()" Nov 16 11:53:11.807003 ropatil1611az-gs6zj-worker-westus-j4ngg sssd[nss][1282]: Starting up Nov 16 11:53:11.833440 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1229]: Starting ovsdb-server. Nov 16 11:53:11.844568 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vsctl[1287]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait -- init -- set Open_vSwitch . db-version=8.3.0 Nov 16 11:53:11.867462 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started System Security Services Daemon. Nov 16 11:53:11.872953 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target User and Group Name Lookups. Nov 16 11:53:11.877595 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vsctl[1292]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait set Open_vSwitch . ovs-version=2.16.5 "external-ids:system-id=\"45649f20-b8cb-4065-81f0-c8f1463b1647\"" "external-ids:rundir=\"/var/run/openvswitch\"" "system-type=\"rhcos\"" "system-version=\"4.10\"" Nov 16 11:53:11.880220 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Login Service... Nov 16 11:53:11.880850 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1229]: Configuring Open vSwitch system IDs. Nov 16 11:53:11.895068 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vsctl[1299]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait add Open_vSwitch . external-ids hostname=ropatil1611az-gs6zj-worker-westus-j4ngg.oom5dvpsls0ujnnamz30z0cwqd.dx.internal.cloudapp.net Nov 16 11:53:11.897175 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1229]: Enabling remote OVSDB managers. Nov 16 11:53:11.898043 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Open vSwitch Database Unit. Nov 16 11:53:11.904538 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Open vSwitch Delete Transient Ports... Nov 16 11:53:11.925054 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-logind[1293]: New seat seat0. Nov 16 11:53:11.927079 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-logind[1293]: Watching system buttons on /dev/input/event0 (Power Button) Nov 16 11:53:11.927339 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-logind[1293]: Watching system buttons on /dev/input/event4 (AT Translated Set 2 keyboard) Nov 16 11:53:11.927510 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-logind[1293]: Watching system buttons on /dev/input/event1 (AT Translated Set 2 keyboard) Nov 16 11:53:11.927586 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Open vSwitch Delete Transient Ports. Nov 16 11:53:11.936796 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Login Service. Nov 16 11:53:11.943739 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Open vSwitch Forwarding Unit... Nov 16 11:53:12.296156 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: openvswitch: Open vSwitch switching datapath Nov 16 11:53:12.296651 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1326]: Inserting openvswitch module. Nov 16 11:53:12.430900 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1326]: Starting ovs-vswitchd. Nov 16 11:53:12.443847 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vsctl[1375]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait add Open_vSwitch . external-ids hostname=ropatil1611az-gs6zj-worker-westus-j4ngg.oom5dvpsls0ujnnamz30z0cwqd.dx.internal.cloudapp.net Nov 16 11:53:12.481200 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-ctl[1326]: Enabling remote OVSDB managers. Nov 16 11:53:12.492488 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Open vSwitch Forwarding Unit. Nov 16 11:53:12.499670 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Open vSwitch... Nov 16 11:53:12.504027 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Open vSwitch. Nov 16 11:53:12.510224 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Network Manager... Nov 16 11:53:12.662794 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-metacopy\x2dcheck2889164436-merged.mount: Succeeded. Nov 16 11:53:12.663914 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599592.6636] NetworkManager (version 1.30.0-16.el8_4) is starting... (after a restart) Nov 16 11:53:12.663943 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599592.6639] Read config: /etc/NetworkManager/NetworkManager.conf (lib: 10-disable-default-plugins.conf, 20-client-id-from-mac.conf) (etc: 20-keyfiles.conf, sdn.conf) Nov 16 11:53:12.670021 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Network Manager. Nov 16 11:53:12.670256 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599592.6702] bus-manager: acquired D-Bus service "org.freedesktop.NetworkManager" Nov 16 11:53:12.675442 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Network. Nov 16 11:53:12.681739 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Afterburn (Check In)... Nov 16 11:53:12.766894 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting OpenSSH server daemon... Nov 16 11:53:12.771426 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599592.7713] manager[0x56490843c0c0]: monitoring kernel firmware directory '/lib/firmware'. Nov 16 11:53:12.772361 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Network Manager Wait Online... Nov 16 11:53:12.772605 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1176]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.8' (uid=0 pid=1379 comm="/usr/sbin/NetworkManager --no-daemon " label="system_u:system_r:NetworkManager_t:s0") Nov 16 11:53:12.777248 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: console-login-helper-messages-issuegen.service: Succeeded. Nov 16 11:53:12.778111 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Generate console-login-helper-messages issue snippet. Nov 16 11:53:12.782082 ropatil1611az-gs6zj-worker-westus-j4ngg sshd[1392]: Server listening on 0.0.0.0 port 22. Nov 16 11:53:12.782219 ropatil1611az-gs6zj-worker-westus-j4ngg sshd[1392]: Server listening on :: port 22. Nov 16 11:53:12.785280 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: console-login-helper-messages-issuegen.service: Consumed 23ms CPU time Nov 16 11:53:12.792790 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started OpenSSH server daemon. Nov 16 11:53:12.922033 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Hostname Service... Nov 16 11:53:12.927361 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Permit User Sessions... Nov 16 11:53:12.929558 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1173]: time="2022-11-16 11:53:12.929146047Z" level=info msg="Checking whether cri-o should wipe containers: version file /var/run/crio/version not found: open /var/run/crio/version: no such file or directory" Nov 16 11:53:12.929558 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1173]: time="2022-11-16 11:53:12.929540950Z" level=info msg="version file /var/lib/crio/version not found: open /var/lib/crio/version: no such file or directory: triggering wipe of images" Nov 16 11:53:12.933001 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-wipe.service: Succeeded. Nov 16 11:53:12.933674 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started CRI-O Auto Update Script. Nov 16 11:53:12.938729 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-wipe.service: Consumed 255ms CPU time Nov 16 11:53:12.939637 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Permit User Sessions. Nov 16 11:53:12.946736 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Getty on tty1. Nov 16 11:53:12.953109 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Serial Getty on ttyS0. Nov 16 11:53:12.958653 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Login Prompts. Nov 16 11:53:13.032644 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1176]: [system] Successfully activated service 'org.freedesktop.hostname1' Nov 16 11:53:13.032829 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Hostname Service. Nov 16 11:53:13.034472 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0343] hostname: hostname: using hostnamed Nov 16 11:53:13.034518 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0344] hostname: hostname changed from (none) to "ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:53:13.034909 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0348] dns-mgr[0x56490842c170]: init: dns=default,systemd-resolved rc-manager=symlink Nov 16 11:53:13.039432 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0394] Loaded device plugin: NMOvsFactory (/usr/lib64/NetworkManager/1.30.0-16.el8_4/libnm-device-plugin-ovs.so) Nov 16 11:53:13.040751 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0407] Loaded device plugin: NMTeamFactory (/usr/lib64/NetworkManager/1.30.0-16.el8_4/libnm-device-plugin-team.so) Nov 16 11:53:13.040831 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0408] manager: rfkill: Wi-Fi enabled by radio killswitch; enabled by state file Nov 16 11:53:13.040916 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0409] manager: rfkill: WWAN enabled by radio killswitch; enabled by state file Nov 16 11:53:13.040988 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0409] manager: Networking is enabled by state file Nov 16 11:53:13.041087 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0410] dhcp-init: Using DHCP client 'internal' Nov 16 11:53:13.041238 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1176]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested by ':1.8' (uid=0 pid=1379 comm="/usr/sbin/NetworkManager --no-daemon " label="system_u:system_r:NetworkManager_t:s0") Nov 16 11:53:13.041414 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0414] settings: Loaded settings plugin: keyfile (internal) Nov 16 11:53:13.042274 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0422] settings: Loaded settings plugin: ifcfg-rh ("/usr/lib64/NetworkManager/1.30.0-16.el8_4/libnm-settings-plugin-ifcfg-rh.so") Nov 16 11:53:13.045432 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0454] device (lo): carrier: link connected Nov 16 11:53:13.045557 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Network Manager Script Dispatcher Service... Nov 16 11:53:13.045687 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0456] manager: (lo): new Generic device (/org/freedesktop/NetworkManager/Devices/1) Nov 16 11:53:13.046570 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0465] device (eth0): carrier: link connected Nov 16 11:53:13.047032 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0470] manager: (eth0): new Ethernet device (/org/freedesktop/NetworkManager/Devices/2) Nov 16 11:53:13.048424 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0484] manager: (eth0): assume: will attempt to assume matching connection 'Wired Connection' (57a7b822-94c8-4a42-8bfc-ecb46b108a8a) (indicated) Nov 16 11:53:13.048498 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0484] device (eth0): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'assume') Nov 16 11:53:13.049502 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0494] device (eth0): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'assume') Nov 16 11:53:13.050474 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0504] device (eth0): Activation: starting connection 'Wired Connection' (57a7b822-94c8-4a42-8bfc-ecb46b108a8a) Nov 16 11:53:13.053557 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0535] device (eth0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'assume') Nov 16 11:53:13.054552 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0545] device (eth0): state change: prepare -> config (reason 'none', sys-iface-state: 'assume') Nov 16 11:53:13.056186 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0561] device (eth0): state change: config -> ip-config (reason 'none', sys-iface-state: 'assume') Nov 16 11:53:13.056622 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0566] dhcp4 (eth0): activation: beginning transaction (timeout in 90 seconds) Nov 16 11:53:13.057787 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1176]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher' Nov 16 11:53:13.058191 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Network Manager Script Dispatcher Service. Nov 16 11:53:13.067610 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0675] dhcp4 (eth0): state changed unknown -> bound, address=10.0.1.4 Nov 16 11:53:13.068932 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0689] device (eth0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'assume') Nov 16 11:53:13.070550 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0705] device (eth0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'assume') Nov 16 11:53:13.070774 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0707] device (eth0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'assume') Nov 16 11:53:13.071283 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0712] manager: NetworkManager state is now CONNECTED_LOCAL Nov 16 11:53:13.072383 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0723] manager: NetworkManager state is now CONNECTED_SITE Nov 16 11:53:13.072492 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0724] policy: set 'Wired Connection' (eth0) as default for IPv4 routing and DNS Nov 16 11:53:13.072901 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1176]: [system] Activating via systemd: service name='org.freedesktop.resolve1' unit='dbus-org.freedesktop.resolve1.service' requested by ':1.8' (uid=0 pid=1379 comm="/usr/sbin/NetworkManager --no-daemon " label="system_u:system_r:NetworkManager_t:s0") Nov 16 11:53:13.073274 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1176]: [system] Activation via systemd failed for unit 'dbus-org.freedesktop.resolve1.service': Unit dbus-org.freedesktop.resolve1.service not found. Nov 16 11:53:13.083290 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0832] device (eth0): Activation: successful, device activated. Nov 16 11:53:13.083933 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0839] manager: NetworkManager state is now CONNECTED_GLOBAL Nov 16 11:53:13.084519 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599593.0845] manager: startup complete Nov 16 11:53:13.087035 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Network Manager Wait Online. Nov 16 11:53:13.093566 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Configures OVS with proper host networking configuration... Nov 16 11:53:13.097055 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + touch /var/run/ovs-config-executed Nov 16 11:53:13.105376 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + NM_CONN_PATH=/etc/NetworkManager/system-connections Nov 16 11:53:13.105376 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + nm_config_changed=0 Nov 16 11:53:13.105376 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + MANAGED_NM_CONN_SUFFIX=-slave-ovs-clone Nov 16 11:53:13.105376 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + BRIDGE_METRIC=48 Nov 16 11:53:13.105376 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + BRIDGE1_METRIC=49 Nov 16 11:53:13.105376 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + trap handle_exit EXIT Nov 16 11:53:13.105376 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' '!' -f /etc/cno/mtu-migration/config ']' Nov 16 11:53:13.105376 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + echo 'Cleaning up left over mtu migration configuration' Nov 16 11:53:13.105376 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: Cleaning up left over mtu migration configuration Nov 16 11:53:13.105376 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + rm -rf /etc/cno/mtu-migration Nov 16 11:53:13.167123 ropatil1611az-gs6zj-worker-westus-j4ngg nm-dispatcher[1406]: Error: Device '' not found. Nov 16 11:53:13.167691 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + rpm -qa Nov 16 11:53:13.172724 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + grep -q openvswitch Nov 16 11:53:13.180632 ropatil1611az-gs6zj-worker-westus-j4ngg nm-dispatcher[1406]: Error: Device '' not found. Nov 16 11:53:13.218784 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Generate console-login-helper-messages issue snippet... Nov 16 11:53:13.271028 ropatil1611az-gs6zj-worker-westus-j4ngg nm-dispatcher[1406]: Error: Device '' not found. Nov 16 11:53:13.688689 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + print_state Nov 16 11:53:13.688689 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + echo 'Current device, connection, interface and routing state:' Nov 16 11:53:13.688689 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: Current device, connection, interface and routing state: Nov 16 11:53:13.709057 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + nmcli -g all device Nov 16 11:53:13.714388 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + grep -v unmanaged Nov 16 11:53:13.727188 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: eth0:ethernet:connected:full:full:/org/freedesktop/NetworkManager/Devices/2:Wired Connection:57a7b822-94c8-4a42-8bfc-ecb46b108a8a:/org/freedesktop/NetworkManager/ActiveConnection/1 Nov 16 11:53:13.740537 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + nmcli -g all connection Nov 16 11:53:13.761139 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: Wired Connection:57a7b822-94c8-4a42-8bfc-ecb46b108a8a:802-3-ethernet:1668599593:Wed Nov 16 11\:53\:13 2022:yes:0:no:/org/freedesktop/NetworkManager/Settings/1:yes:eth0:activated:/org/freedesktop/NetworkManager/ActiveConnection/1::/run/NetworkManager/system-connections/default_connection.nmconnection Nov 16 11:53:13.782560 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + ip -d address show Nov 16 11:53:13.876898 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 Nov 16 11:53:13.876898 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 minmtu 0 maxmtu 0 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 Nov 16 11:53:13.876898 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: inet 127.0.0.1/8 scope host lo Nov 16 11:53:13.876898 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: valid_lft forever preferred_lft forever Nov 16 11:53:13.876898 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: inet6 ::1/128 scope host Nov 16 11:53:13.876898 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: valid_lft forever preferred_lft forever Nov 16 11:53:13.876898 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: 2: eth0: mtu 1500 qdisc mq state UP group default qlen 1000 Nov 16 11:53:13.876898 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: link/ether 00:0d:3a:30:a6:f2 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 65521 numtxqueues 64 numrxqueues 64 gso_max_size 62780 gso_max_segs 65535 Nov 16 11:53:13.876898 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: inet 10.0.1.4/24 brd 10.0.1.255 scope global noprefixroute eth0 Nov 16 11:53:13.876898 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: valid_lft forever preferred_lft forever Nov 16 11:53:13.876898 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: inet6 fe80::20d:3aff:fe30:a6f2/64 scope link noprefixroute Nov 16 11:53:13.876898 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: valid_lft forever preferred_lft forever Nov 16 11:53:13.975108 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + ip route show Nov 16 11:53:13.980555 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: default via 10.0.1.1 dev eth0 proto dhcp metric 100 Nov 16 11:53:13.980555 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: 10.0.1.0/24 dev eth0 proto kernel scope link src 10.0.1.4 metric 100 Nov 16 11:53:13.980555 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: 168.63.129.16 via 10.0.1.1 dev eth0 proto dhcp metric 100 Nov 16 11:53:13.980555 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: 169.254.169.254 via 10.0.1.1 dev eth0 proto dhcp metric 100 Nov 16 11:53:13.978233 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vsctl[1496]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=30 --if-exists del-br br-ex Nov 16 11:53:14.008936 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + ip -6 route show Nov 16 11:53:14.014976 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: ::1 dev lo proto kernel metric 256 pref medium Nov 16 11:53:14.014976 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: fe80::/64 dev eth0 proto kernel metric 100 pref medium Nov 16 11:53:14.027855 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' OpenShiftSDN == OVNKubernetes ']' Nov 16 11:53:14.027855 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' OpenShiftSDN == OpenShiftSDN ']' Nov 16 11:53:14.027855 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + rollback_nm Nov 16 11:53:14.044070 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: ++ get_bridge_physical_interface ovs-if-phys0 Nov 16 11:53:14.044070 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: ++ local bridge_interface=ovs-if-phys0 Nov 16 11:53:14.044070 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: ++ local physical_interface= Nov 16 11:53:14.063352 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: +++ nmcli -g connection.interface-name conn show ovs-if-phys0 Nov 16 11:53:14.063352 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: +++ echo '' Nov 16 11:53:14.076145 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: ++ physical_interface= Nov 16 11:53:14.076145 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: ++ echo '' Nov 16 11:53:14.087057 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + phys0= Nov 16 11:53:14.092185 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: ++ get_bridge_physical_interface ovs-if-phys1 Nov 16 11:53:14.092185 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: ++ local bridge_interface=ovs-if-phys1 Nov 16 11:53:14.092185 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: ++ local physical_interface= Nov 16 11:53:14.110113 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: +++ nmcli -g connection.interface-name conn show ovs-if-phys1 Nov 16 11:53:14.110113 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: +++ echo '' Nov 16 11:53:14.122440 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: ++ physical_interface= Nov 16 11:53:14.122440 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: ++ echo '' Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + phys1= Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + remove_all_ovn_bridges Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + echo 'Reverting any previous OVS configuration' Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: Reverting any previous OVS configuration Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + remove_ovn_bridges br-ex phys0 Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + bridge_name=br-ex Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + port_name=phys0 Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + update_nm_conn_files br-ex phys0 Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + bridge_name=br-ex Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + port_name=phys0 Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + ovs_port=ovs-port-br-ex Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + ovs_interface=ovs-if-br-ex Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + default_port_name=ovs-port-phys0 Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + bridge_interface_name=ovs-if-phys0 Nov 16 11:53:14.133151 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + MANAGED_NM_CONN_FILES=($(echo "${NM_CONN_PATH}"/{"$bridge_name","$ovs_interface","$ovs_port","$bridge_interface_name","$default_port_name"}{,.nmconnection})) Nov 16 11:53:14.224736 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: ++ echo /etc/NetworkManager/system-connections/br-ex /etc/NetworkManager/system-connections/br-ex.nmconnection /etc/NetworkManager/system-connections/ovs-if-br-ex /etc/NetworkManager/system-connections/ovs-if-br-ex.nmconnection /etc/NetworkManager/system-connections/ovs-port-br-ex /etc/NetworkManager/system-connections/ovs-port-br-ex.nmconnection /etc/NetworkManager/system-connections/ovs-if-phys0 /etc/NetworkManager/system-connections/ovs-if-phys0.nmconnection /etc/NetworkManager/system-connections/ovs-port-phys0 /etc/NetworkManager/system-connections/ovs-port-phys0.nmconnection Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + shopt -s nullglob Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + MANAGED_NM_CONN_FILES+=(${NM_CONN_PATH}/*${MANAGED_NM_CONN_SUFFIX}.nmconnection ${NM_CONN_PATH}/*${MANAGED_NM_CONN_SUFFIX}) Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + shopt -u nullglob Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + rm_nm_conn_files Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + for file in "${MANAGED_NM_CONN_FILES[@]}" Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' -f /etc/NetworkManager/system-connections/br-ex ']' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + for file in "${MANAGED_NM_CONN_FILES[@]}" Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' -f /etc/NetworkManager/system-connections/br-ex.nmconnection ']' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + for file in "${MANAGED_NM_CONN_FILES[@]}" Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' -f /etc/NetworkManager/system-connections/ovs-if-br-ex ']' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + for file in "${MANAGED_NM_CONN_FILES[@]}" Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' -f /etc/NetworkManager/system-connections/ovs-if-br-ex.nmconnection ']' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + for file in "${MANAGED_NM_CONN_FILES[@]}" Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' -f /etc/NetworkManager/system-connections/ovs-port-br-ex ']' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + for file in "${MANAGED_NM_CONN_FILES[@]}" Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' -f /etc/NetworkManager/system-connections/ovs-port-br-ex.nmconnection ']' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + for file in "${MANAGED_NM_CONN_FILES[@]}" Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' -f /etc/NetworkManager/system-connections/ovs-if-phys0 ']' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + for file in "${MANAGED_NM_CONN_FILES[@]}" Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' -f /etc/NetworkManager/system-connections/ovs-if-phys0.nmconnection ']' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + for file in "${MANAGED_NM_CONN_FILES[@]}" Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' -f /etc/NetworkManager/system-connections/ovs-port-phys0 ']' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + for file in "${MANAGED_NM_CONN_FILES[@]}" Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' -f /etc/NetworkManager/system-connections/ovs-port-phys0.nmconnection ']' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + ovs-vsctl --timeout=30 --if-exists del-br br-ex Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' -d /sys/class/net/br-ex1 ']' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + echo 'OVS configuration successfully reverted' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: OVS configuration successfully reverted Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + reload_profiles_nm '' '' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' 0 -eq 0 ']' Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + return Nov 16 11:53:14.258282 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + ovs-vsctl --timeout=30 --if-exists del-br br-int -- --if-exists del-br br-local Nov 16 11:53:14.241455 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: console-login-helper-messages-issuegen.service: Succeeded. Nov 16 11:53:14.376488 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vsctl[1497]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=30 --if-exists del-br br-int -- --if-exists del-br br-local Nov 16 11:53:14.242106 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Generate console-login-helper-messages issue snippet. Nov 16 11:53:14.266144 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: console-login-helper-messages-issuegen.service: Consumed 20ms CPU time Nov 16 11:53:14.482999 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + handle_exit Nov 16 11:53:14.482999 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + e=0 Nov 16 11:53:14.482999 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + '[' 0 -eq 0 ']' Nov 16 11:53:14.482999 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + print_state Nov 16 11:53:14.501102 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + echo 'Current device, connection, interface and routing state:' Nov 16 11:53:14.501102 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: Current device, connection, interface and routing state: Nov 16 11:53:14.515007 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + nmcli -g all device Nov 16 11:53:14.521066 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + grep -v unmanaged Nov 16 11:53:14.521066 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: eth0:ethernet:connected:full:full:/org/freedesktop/NetworkManager/Devices/2:Wired Connection:57a7b822-94c8-4a42-8bfc-ecb46b108a8a:/org/freedesktop/NetworkManager/ActiveConnection/1 Nov 16 11:53:14.538607 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + nmcli -g all connection Nov 16 11:53:14.546036 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: Wired Connection:57a7b822-94c8-4a42-8bfc-ecb46b108a8a:802-3-ethernet:1668599593:Wed Nov 16 11\:53\:13 2022:yes:0:no:/org/freedesktop/NetworkManager/Settings/1:yes:eth0:activated:/org/freedesktop/NetworkManager/ActiveConnection/1::/run/NetworkManager/system-connections/default_connection.nmconnection Nov 16 11:53:14.566018 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + ip -d address show Nov 16 11:53:14.571983 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 Nov 16 11:53:14.571983 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 minmtu 0 maxmtu 0 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 Nov 16 11:53:14.571983 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: inet 127.0.0.1/8 scope host lo Nov 16 11:53:14.571983 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: valid_lft forever preferred_lft forever Nov 16 11:53:14.571983 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: inet6 ::1/128 scope host Nov 16 11:53:14.571983 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: valid_lft forever preferred_lft forever Nov 16 11:53:14.571983 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: 2: eth0: mtu 1500 qdisc mq state UP group default qlen 1000 Nov 16 11:53:14.571983 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: link/ether 00:0d:3a:30:a6:f2 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 65521 numtxqueues 64 numrxqueues 64 gso_max_size 62780 gso_max_segs 65535 Nov 16 11:53:14.571983 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: inet 10.0.1.4/24 brd 10.0.1.255 scope global noprefixroute eth0 Nov 16 11:53:14.571983 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: valid_lft forever preferred_lft forever Nov 16 11:53:14.571983 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: inet6 fe80::20d:3aff:fe30:a6f2/64 scope link noprefixroute Nov 16 11:53:14.571983 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: valid_lft forever preferred_lft forever Nov 16 11:53:14.571629 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ovs-configuration.service: Succeeded. Nov 16 11:53:14.686255 ropatil1611az-gs6zj-worker-westus-j4ngg rpc.statd[1531]: Version 2.3.3 starting Nov 16 11:53:14.722704 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + ip route show Nov 16 11:53:14.572337 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Configures OVS with proper host networking configuration. Nov 16 11:53:14.686267 ropatil1611az-gs6zj-worker-westus-j4ngg rpc.statd[1531]: Flags: TI-RPC Nov 16 11:53:14.728515 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: default via 10.0.1.1 dev eth0 proto dhcp metric 100 Nov 16 11:53:14.728515 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: 10.0.1.0/24 dev eth0 proto kernel scope link src 10.0.1.4 metric 100 Nov 16 11:53:14.728515 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: 168.63.129.16 via 10.0.1.1 dev eth0 proto dhcp metric 100 Nov 16 11:53:14.728515 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: 169.254.169.254 via 10.0.1.1 dev eth0 proto dhcp metric 100 Nov 16 11:53:14.757766 ropatil1611az-gs6zj-worker-westus-j4ngg mco-hostname[1521]: waiting for non-localhost hostname to be assigned Nov 16 11:53:14.757766 ropatil1611az-gs6zj-worker-westus-j4ngg mco-hostname[1521]: node identified as ropatil1611az-gs6zj-worker-westus-j4ngg Nov 16 11:53:14.579600 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: ovs-configuration.service: Consumed 298ms CPU time Nov 16 11:53:14.758195 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + ip -6 route show Nov 16 11:53:14.581382 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Wait for a non-localhost hostname... Nov 16 11:53:14.764210 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: ::1 dev lo proto kernel metric 256 pref medium Nov 16 11:53:14.764210 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: fe80::/64 dev eth0 proto kernel metric 100 pref medium Nov 16 11:53:14.593722 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Wait for a non-localhost hostname. Nov 16 11:53:14.777939 ropatil1611az-gs6zj-worker-westus-j4ngg configure-ovs.sh[1425]: + exit 0 Nov 16 11:53:14.612820 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Network is Online. Nov 16 11:53:14.783189 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.752678090Z" level=info msg="Starting CRI-O, version: 1.23.3-20.rhaos4.10.git89344de.el8, git: ()" Nov 16 11:53:14.783189 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.752833291Z" level=info msg="Node configuration value for hugetlb cgroup is true" Nov 16 11:53:14.783189 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.752847191Z" level=info msg="Node configuration value for pid cgroup is true" Nov 16 11:53:14.783189 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.752940792Z" level=info msg="Node configuration value for memoryswap cgroup is true" Nov 16 11:53:14.783189 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.752951592Z" level=info msg="Node configuration value for cgroup v2 is false" Nov 16 11:53:14.783189 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.762641163Z" level=info msg="Node configuration value for systemd CollectMode is true" Nov 16 11:53:14.783189 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.770466021Z" level=info msg="Node configuration value for systemd AllowedCPUs is true" Nov 16 11:53:14.783189 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.772541536Z" level=info msg="Using default capabilities: CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_FSETID, CAP_FOWNER, CAP_SETGID, CAP_SETUID, CAP_SETPCAP, CAP_NET_BIND_SERVICE, CAP_KILL" Nov 16 11:53:14.627520 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting NFS status monitor for NFSv2/3 locking.... Nov 16 11:53:14.633712 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Dynamically sets the system reserved for the kubelet... Nov 16 11:53:14.641857 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: iscsi.service: Unit cannot be reloaded because it is inactive. Nov 16 11:53:14.643821 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Dynamically sets the system reserved for the kubelet. Nov 16 11:53:14.662392 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting RPC Bind... Nov 16 11:53:14.682305 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Container Runtime Interface for OCI (CRI-O)... Nov 16 11:53:14.693833 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started RPC Bind. Nov 16 11:53:14.708906 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started NFS status monitor for NFSv2/3 locking.. Nov 16 11:53:14.819889 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.819851385Z" level=info msg="Conmon does support the --sync option" Nov 16 11:53:14.824667 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.824579620Z" level=info msg="Conmon does support the --log-global-size-max option" Nov 16 11:53:14.825306 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.825278125Z" level=info msg="No seccomp profile specified, using the internal default" Nov 16 11:53:14.825306 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.825303225Z" level=info msg="AppArmor is disabled by the system or at CRI-O build-time" Nov 16 11:53:14.825385 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.825312025Z" level=info msg="No blockio config file specified, blockio not configured" Nov 16 11:53:14.825385 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.825319325Z" level=info msg="RDT not available in the host system" Nov 16 11:53:14.825650 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.825611227Z" level=info msg="Updated default CNI network name to " Nov 16 11:53:14.964190 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.963918047Z" level=warning msg="Error encountered when checking whether cri-o should wipe images: version file /var/lib/crio/version not found: open /var/lib/crio/version: no such file or directory" Nov 16 11:53:14.965375 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:14.965293357Z" level=info msg="Serving metrics on :9537 via HTTP" Nov 16 11:53:14.981456 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Container Runtime Interface for OCI (CRI-O). Nov 16 11:53:14.989838 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Kubernetes Kubelet... Nov 16 11:53:15.231860 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 16 11:53:15.231860 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 16 11:53:15.231860 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Flag --cloud-provider has been deprecated, will be removed in 1.24 or later, in favor of removing cloud provider code from Kubelet. Nov 16 11:53:15.231860 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Flag --cloud-config has been deprecated, will be removed in 1.24 or later, in favor of removing cloud provider code from Kubelet. Nov 16 11:53:15.231860 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 16 11:53:15.232676 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.232068 1563 server.go:198] "Warning: For remote container runtime, --pod-infra-container-image is ignored in kubelet, which should be set in that remote runtime instead" Nov 16 11:53:15.234027 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 16 11:53:15.234027 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 16 11:53:15.234027 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Flag --cloud-provider has been deprecated, will be removed in 1.24 or later, in favor of removing cloud provider code from Kubelet. Nov 16 11:53:15.234027 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Flag --cloud-config has been deprecated, will be removed in 1.24 or later, in favor of removing cloud provider code from Kubelet. Nov 16 11:53:15.234027 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 16 11:53:15.234191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.234046 1563 feature_gate.go:223] unrecognized feature gate: NodeDisruptionExclusion Nov 16 11:53:15.234191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.234054 1563 feature_gate.go:223] unrecognized feature gate: ServiceNodeExclusion Nov 16 11:53:15.234191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.234061 1563 feature_gate.go:223] unrecognized feature gate: SupportPodPidsLimit Nov 16 11:53:15.234191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.234067 1563 feature_gate.go:223] unrecognized feature gate: LegacyNodeRoleBehavior Nov 16 11:53:15.234191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.234145 1563 feature_gate.go:223] unrecognized feature gate: SupportPodPidsLimit Nov 16 11:53:15.234191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.234153 1563 feature_gate.go:223] unrecognized feature gate: LegacyNodeRoleBehavior Nov 16 11:53:15.234191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.234158 1563 feature_gate.go:223] unrecognized feature gate: NodeDisruptionExclusion Nov 16 11:53:15.234191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.234163 1563 feature_gate.go:223] unrecognized feature gate: ServiceNodeExclusion Nov 16 11:53:15.234380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234252 1563 flags.go:64] FLAG: --add-dir-header="false" Nov 16 11:53:15.234380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234271 1563 flags.go:64] FLAG: --address="0.0.0.0" Nov 16 11:53:15.234380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234277 1563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 16 11:53:15.234380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234287 1563 flags.go:64] FLAG: --alsologtostderr="false" Nov 16 11:53:15.234380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234291 1563 flags.go:64] FLAG: --anonymous-auth="true" Nov 16 11:53:15.234380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234297 1563 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 16 11:53:15.234380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234301 1563 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 16 11:53:15.234380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234305 1563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 16 11:53:15.234380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234311 1563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 16 11:53:15.234380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234315 1563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 16 11:53:15.234380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234319 1563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 16 11:53:15.234380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234325 1563 flags.go:64] FLAG: --azure-container-registry-config="" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234528 1563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234542 1563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234547 1563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234553 1563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234558 1563 flags.go:64] FLAG: --cgroup-root="" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234563 1563 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234567 1563 flags.go:64] FLAG: --client-ca-file="" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234571 1563 flags.go:64] FLAG: --cloud-config="/etc/kubernetes/cloud.conf" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234576 1563 flags.go:64] FLAG: --cloud-provider="azure" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234580 1563 flags.go:64] FLAG: --cluster-dns="[]" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234586 1563 flags.go:64] FLAG: --cluster-domain="" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234590 1563 flags.go:64] FLAG: --cni-bin-dir="/opt/cni/bin" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234597 1563 flags.go:64] FLAG: --cni-cache-dir="/var/lib/cni/cache" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234602 1563 flags.go:64] FLAG: --cni-conf-dir="/etc/cni/net.d" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234606 1563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234611 1563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234616 1563 flags.go:64] FLAG: --container-log-max-files="5" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234621 1563 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234643 1563 flags.go:64] FLAG: --container-runtime="remote" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234670 1563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234675 1563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234697 1563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234701 1563 flags.go:64] FLAG: --contention-profiling="false" Nov 16 11:53:15.234701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234706 1563 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234710 1563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234716 1563 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234720 1563 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234728 1563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234749 1563 flags.go:64] FLAG: --docker="unix:///var/run/docker.sock" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234754 1563 flags.go:64] FLAG: --docker-endpoint="unix:///var/run/docker.sock" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234776 1563 flags.go:64] FLAG: --docker-env-metadata-whitelist="" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234780 1563 flags.go:64] FLAG: --docker-only="false" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234807 1563 flags.go:64] FLAG: --docker-root="/var/lib/docker" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234812 1563 flags.go:64] FLAG: --docker-tls="false" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234833 1563 flags.go:64] FLAG: --docker-tls-ca="ca.pem" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234838 1563 flags.go:64] FLAG: --docker-tls-cert="cert.pem" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234858 1563 flags.go:64] FLAG: --docker-tls-key="key.pem" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234863 1563 flags.go:64] FLAG: --dynamic-config-dir="" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234868 1563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234872 1563 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234878 1563 flags.go:64] FLAG: --enable-load-reader="false" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234882 1563 flags.go:64] FLAG: --enable-server="true" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234886 1563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234894 1563 flags.go:64] FLAG: --event-burst="10" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234898 1563 flags.go:64] FLAG: --event-qps="5" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234902 1563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234906 1563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 16 11:53:15.235315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234910 1563 flags.go:64] FLAG: --eviction-hard="imagefs.available<15%,memory.available<100Mi,nodefs.available<10%,nodefs.inodesFree<5%" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234923 1563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234927 1563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234934 1563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234939 1563 flags.go:64] FLAG: --eviction-soft="" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234943 1563 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234947 1563 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234951 1563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234955 1563 flags.go:64] FLAG: --experimental-check-node-capabilities-before-mount="false" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234959 1563 flags.go:64] FLAG: --experimental-dockershim-root-directory="/var/lib/dockershim" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234969 1563 flags.go:64] FLAG: --experimental-kernel-memcg-notification="false" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234974 1563 flags.go:64] FLAG: --experimental-logging-sanitization="false" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234978 1563 flags.go:64] FLAG: --experimental-mounter-path="" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234982 1563 flags.go:64] FLAG: --fail-swap-on="true" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234986 1563 flags.go:64] FLAG: --feature-gates="" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234992 1563 flags.go:64] FLAG: --file-check-frequency="20s" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.234996 1563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235000 1563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235004 1563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235009 1563 flags.go:64] FLAG: --healthz-port="10248" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235013 1563 flags.go:64] FLAG: --help="false" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235017 1563 flags.go:64] FLAG: --hostname-override="" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235021 1563 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235025 1563 flags.go:64] FLAG: --http-check-frequency="20s" Nov 16 11:53:15.235892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235029 1563 flags.go:64] FLAG: --image-credential-provider-bin-dir="" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235033 1563 flags.go:64] FLAG: --image-credential-provider-config="" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235037 1563 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235041 1563 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235045 1563 flags.go:64] FLAG: --image-pull-progress-deadline="1m0s" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235049 1563 flags.go:64] FLAG: --image-service-endpoint="" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235053 1563 flags.go:64] FLAG: --iptables-drop-bit="15" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235057 1563 flags.go:64] FLAG: --iptables-masquerade-bit="14" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235061 1563 flags.go:64] FLAG: --keep-terminated-pod-volumes="false" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235065 1563 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235086 1563 flags.go:64] FLAG: --kube-api-burst="10" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235090 1563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235094 1563 flags.go:64] FLAG: --kube-api-qps="5" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235099 1563 flags.go:64] FLAG: --kube-reserved="" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235103 1563 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235107 1563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235112 1563 flags.go:64] FLAG: --kubelet-cgroups="" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235116 1563 flags.go:64] FLAG: --lock-file="" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235141 1563 flags.go:64] FLAG: --log-backtrace-at=":0" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235149 1563 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235153 1563 flags.go:64] FLAG: --log-dir="" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235158 1563 flags.go:64] FLAG: --log-file="" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235162 1563 flags.go:64] FLAG: --log-file-max-size="1800" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235166 1563 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235171 1563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235178 1563 flags.go:64] FLAG: --log-json-split-stream="false" Nov 16 11:53:15.236483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235182 1563 flags.go:64] FLAG: --logging-format="text" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235186 1563 flags.go:64] FLAG: --logtostderr="true" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235190 1563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235195 1563 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235199 1563 flags.go:64] FLAG: --manifest-url="" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235203 1563 flags.go:64] FLAG: --manifest-url-header="" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235209 1563 flags.go:64] FLAG: --master-service-namespace="default" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235213 1563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235217 1563 flags.go:64] FLAG: --max-open-files="1000000" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235223 1563 flags.go:64] FLAG: --max-pods="110" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235228 1563 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235232 1563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235236 1563 flags.go:64] FLAG: --memory-manager-policy="None" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235241 1563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235245 1563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235251 1563 flags.go:64] FLAG: --network-plugin="" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235256 1563 flags.go:64] FLAG: --network-plugin-mtu="0" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235260 1563 flags.go:64] FLAG: --node-ip="" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235264 1563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhcos" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235274 1563 flags.go:64] FLAG: --node-status-max-images="50" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235278 1563 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235282 1563 flags.go:64] FLAG: --non-masquerade-cidr="10.0.0.0/8" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235286 1563 flags.go:64] FLAG: --one-output="false" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235290 1563 flags.go:64] FLAG: --oom-score-adj="-999" Nov 16 11:53:15.237222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235301 1563 flags.go:64] FLAG: --pod-cidr="" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235305 1563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235312 1563 flags.go:64] FLAG: --pod-manifest-path="" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235316 1563 flags.go:64] FLAG: --pod-max-pids="-1" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235321 1563 flags.go:64] FLAG: --pods-per-core="0" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235325 1563 flags.go:64] FLAG: --port="10250" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235339 1563 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235345 1563 flags.go:64] FLAG: --provider-id="" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235350 1563 flags.go:64] FLAG: --qos-reserved="" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235354 1563 flags.go:64] FLAG: --read-only-port="10255" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235359 1563 flags.go:64] FLAG: --really-crash-for-testing="false" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235363 1563 flags.go:64] FLAG: --register-node="true" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235367 1563 flags.go:64] FLAG: --register-schedulable="true" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235380 1563 flags.go:64] FLAG: --register-with-taints="" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235385 1563 flags.go:64] FLAG: --registry-burst="10" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235389 1563 flags.go:64] FLAG: --registry-qps="5" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235393 1563 flags.go:64] FLAG: --reserved-cpus="" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235397 1563 flags.go:64] FLAG: --reserved-memory="" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235402 1563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235407 1563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235411 1563 flags.go:64] FLAG: --rotate-certificates="false" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235415 1563 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235419 1563 flags.go:64] FLAG: --runonce="false" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235424 1563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235428 1563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 16 11:53:15.237750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235432 1563 flags.go:64] FLAG: --seccomp-default="false" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235436 1563 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235440 1563 flags.go:64] FLAG: --skip-headers="false" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235444 1563 flags.go:64] FLAG: --skip-log-headers="false" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235448 1563 flags.go:64] FLAG: --stderrthreshold="2" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235452 1563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235457 1563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235462 1563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235471 1563 flags.go:64] FLAG: --storage-driver-password="root" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235475 1563 flags.go:64] FLAG: --storage-driver-secure="false" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235479 1563 flags.go:64] FLAG: --storage-driver-table="stats" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235484 1563 flags.go:64] FLAG: --storage-driver-user="root" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235488 1563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235492 1563 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235496 1563 flags.go:64] FLAG: --system-cgroups="" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235500 1563 flags.go:64] FLAG: --system-reserved="cpu=500m,memory=1Gi" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235507 1563 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235511 1563 flags.go:64] FLAG: --tls-cert-file="" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235514 1563 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235520 1563 flags.go:64] FLAG: --tls-min-version="" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235524 1563 flags.go:64] FLAG: --tls-private-key-file="" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235529 1563 flags.go:64] FLAG: --topology-manager-policy="none" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235534 1563 flags.go:64] FLAG: --topology-manager-scope="container" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235538 1563 flags.go:64] FLAG: --v="2" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235548 1563 flags.go:64] FLAG: --version="false" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235554 1563 flags.go:64] FLAG: --vmodule="" Nov 16 11:53:15.238301 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235559 1563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 16 11:53:15.238851 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.235564 1563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 16 11:53:15.247724 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 11:53:15.253817 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.253788 1563 mount_linux.go:222] Detected OS with systemd Nov 16 11:53:15.254120 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.254088 1563 server.go:446] "Kubelet version" kubeletVersion="v1.23.12+7566c4d" Nov 16 11:53:15.254252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.254207 1563 feature_gate.go:223] unrecognized feature gate: SupportPodPidsLimit Nov 16 11:53:15.254252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.254221 1563 feature_gate.go:223] unrecognized feature gate: LegacyNodeRoleBehavior Nov 16 11:53:15.254252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.254228 1563 feature_gate.go:223] unrecognized feature gate: NodeDisruptionExclusion Nov 16 11:53:15.254252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.254234 1563 feature_gate.go:223] unrecognized feature gate: ServiceNodeExclusion Nov 16 11:53:15.254356 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.254241 1563 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true CSIMigrationAWS:false CSIMigrationAzureDisk:false CSIMigrationAzureFile:false CSIMigrationGCE:false CSIMigrationOpenStack:false CSIMigrationvSphere:false DownwardAPIHugePages:true PodSecurity:true RotateKubeletServerCertificate:true]} Nov 16 11:53:15.254356 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.254328 1563 feature_gate.go:223] unrecognized feature gate: NodeDisruptionExclusion Nov 16 11:53:15.254356 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.254338 1563 feature_gate.go:223] unrecognized feature gate: ServiceNodeExclusion Nov 16 11:53:15.254356 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.254344 1563 feature_gate.go:223] unrecognized feature gate: SupportPodPidsLimit Nov 16 11:53:15.254356 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.254350 1563 feature_gate.go:223] unrecognized feature gate: LegacyNodeRoleBehavior Nov 16 11:53:15.254493 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.254355 1563 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true CSIMigrationAWS:false CSIMigrationAzureDisk:false CSIMigrationAzureFile:false CSIMigrationGCE:false CSIMigrationOpenStack:false CSIMigrationvSphere:false DownwardAPIHugePages:true PodSecurity:true RotateKubeletServerCertificate:true]} Nov 16 11:53:15.254665 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:15.254537 1563 plugins.go:132] WARNING: azure built-in cloud provider is now deprecated. The Azure provider is deprecated and will be removed in a future release. Please use https://github.com/kubernetes-sigs/cloud-provider-azure Nov 16 11:53:15.255866 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.255793 1563 azure_auth.go:94] azure: using managed identity extension to retrieve access token Nov 16 11:53:15.258379 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rcd23012634d04223baec71ccf2310ffe.scope: Succeeded. Nov 16 11:53:15.259071 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rcd23012634d04223baec71ccf2310ffe.scope: Consumed 801us CPU time Nov 16 11:53:15.267584 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.267500 1563 azure.go:454] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=6, jitter=1.000000 Nov 16 11:53:15.268361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268316 1563 azure_interfaceclient.go:63] Azure InterfacesClient (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268356 1563 azure_interfaceclient.go:66] Azure InterfacesClient (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268377 1563 azure_vmsizeclient.go:63] Azure VirtualMachineSizesClient (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268382 1563 azure_vmsizeclient.go:66] Azure VirtualMachineSizesClient (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268569 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268552 1563 azure_snapshotclient.go:64] Azure SnapshotClient (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268569 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268561 1563 azure_snapshotclient.go:67] Azure SnapshotClient (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268639 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268573 1563 azure_storageaccountclient.go:69] Azure StorageAccountClient (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268639 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268578 1563 azure_storageaccountclient.go:72] Azure StorageAccountClient (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268639 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268593 1563 azure_diskclient.go:69] Azure DisksClient (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268639 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268598 1563 azure_diskclient.go:72] Azure DisksClient (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268639 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268606 1563 azure_vmclient.go:64] Azure VirtualMachine client (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268639 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268627 1563 azure_vmclient.go:67] Azure VirtualMachine client (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268639 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268636 1563 azure_vmssclient.go:64] Azure VirtualMachineScaleSetClient (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268639 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268641 1563 azure_vmssclient.go:67] Azure VirtualMachineScaleSetClient (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268650 1563 azure_vmssvmclient.go:65] Azure vmssVM client (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268655 1563 azure_vmssvmclient.go:68] Azure vmssVM client (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268663 1563 azure_routeclient.go:62] Azure RoutesClient (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268667 1563 azure_routeclient.go:65] Azure RoutesClient (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268689 1563 azure_subnetclient.go:63] Azure SubnetsClient (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268695 1563 azure_subnetclient.go:66] Azure SubnetsClient (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268703 1563 azure_routetableclient.go:62] Azure RouteTablesClient (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268723 1563 azure_routetableclient.go:65] Azure RouteTablesClient (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268731 1563 azure_loadbalancerclient.go:64] Azure LoadBalancersClient (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268736 1563 azure_loadbalancerclient.go:67] Azure LoadBalancersClient (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268743 1563 azure_securitygroupclient.go:64] Azure SecurityGroupsClient (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268752 1563 azure_securitygroupclient.go:67] Azure SecurityGroupsClient (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268759 1563 azure_publicipclient.go:64] Azure PublicIPAddressesClient (read ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.268870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.268802 1563 azure_publicipclient.go:67] Azure PublicIPAddressesClient (write ops) using rate limit config: QPS=1, bucket=5 Nov 16 11:53:15.269229 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.269011 1563 server.go:585] "Successfully initialized cloud provider" cloudProvider="azure" cloudConfigFile="/etc/kubernetes/cloud.conf" Nov 16 11:53:15.269229 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.269047 1563 server.go:1039] "Cloud provider determined current node" nodeName="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:53:15.269229 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.269058 1563 server.go:874] "Client rotation is on, will bootstrap in background" Nov 16 11:53:15.271634 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.271572 1563 bootstrap.go:100] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Nov 16 11:53:15.271929 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.271891 1563 server.go:931] "Starting client certificate rotation" Nov 16 11:53:15.272099 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.272077 1563 certificate_manager.go:270] kubernetes.io/kube-apiserver-client-kubelet: Certificate rotation is enabled Nov 16 11:53:15.272272 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.272231 1563 certificate_manager.go:270] kubernetes.io/kube-apiserver-client-kubelet: Rotating certificates Nov 16 11:53:15.275028 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.274966 1563 dynamic_cafile_content.go:118] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 16 11:53:15.275253 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.275211 1563 dynamic_cafile_content.go:156] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 16 11:53:15.276325 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:15.276280 1563 manager.go:165] cAdvisor running in container: "/sys/fs/cgroup/cpu,cpuacct/system.slice/kubelet.service" Nov 16 11:53:20.283558 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.283418 1563 fs.go:137] Filesystem UUIDs: map[4a20a463-024a-4bb3-afd8-31decc06c38d:/dev/sda3 566864e3-acbe-4961-8488-233c4e33ce96:/dev/sda4 6CFE4825FE47E644:/dev/sdb1 ACDE-4FE1:/dev/sda2] Nov 16 11:53:20.283558 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.283484 1563 fs.go:138] Filesystem partitions: map[/dev/sda3:{mountpoint:/boot major:8 minor:3 fsType:ext4 blockSize:0} /dev/sda4:{mountpoint:/var major:8 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /sys/fs/cgroup:{mountpoint:/sys/fs/cgroup major:0 minor:25 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:43 fsType:tmpfs blockSize:0}] Nov 16 11:53:20.283558 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.283576 1563 nvidia.go:54] NVIDIA GPU metrics disabled Nov 16 11:53:20.294537 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.294256 1563 manager.go:214] Machine: {Timestamp:2022-11-16 11:53:20.293725036 +0000 UTC m=+5.230500658 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:8 NumSockets:1 CpuFrequency:2294686 MemoryCapacity:16802365440 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:17c3dc7b18784633a4bd61ccb76c02a9 SystemUUID:f63520c0-c4ce-5743-af3a-5ffe95225017 BootID:02e2874c-844e-4381-b533-ab7db72975cc Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:8401182720 Type:vfs Inodes:2051070 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:8401182720 Type:vfs Inodes:2051070 HasInodes:true} {Device:/sys/fs/cgroup DeviceMajor:0 DeviceMinor:25 Capacity:8401182720 Type:vfs Inodes:2051070 HasInodes:true} {Device:/dev/sda4 DeviceMajor:8 DeviceMinor:4 Capacity:274329481216 Type:vfs Inodes:133955008 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:43 Capacity:8401182720 Type:vfs Inodes:2051070 HasInodes:true} {Device:/dev/sda3 DeviceMajor:8 DeviceMinor:3 Capacity:379452416 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[8:0:{Name:sda Major:8 Minor:0 Size:274877906944 Scheduler:mq-deadline} 8:16:{Name:sdb Major:8 Minor:16 Size:34359738368 Scheduler:mq-deadline}] NetworkDevices:[{Name:eth0 MacAddress:00:0d:3a:30:a6:f2 Speed:40000 Mtu:1500}] Topology:[{Id:0 Memory:16802365440 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:262144 Type:Unified Level:2}] UncoreCaches:[] SocketID:0} {Id:1 Threads:[1] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:262144 Type:Unified Level:2}] UncoreCaches:[] SocketID:0} {Id:2 Threads:[2] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:262144 Type:Unified Level:2}] UncoreCaches:[] SocketID:0} {Id:3 Threads:[3] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:262144 Type:Unified Level:2}] UncoreCaches:[] SocketID:0} {Id:4 Threads:[4] Caches:[{Id:4 Size:32768 Type:Data Level:1} {Id:4 Size:32768 Type:Instruction Level:1} {Id:4 Size:262144 Type:Unified Level:2}] UncoreCaches:[] SocketID:0} {Id:5 Threads:[5] Caches:[{Id:5 Size:32768 Type:Data Level:1} {Id:5 Size:32768 Type:Instruction Level:1} {Id:5 Size:262144 Type:Unified Level:2}] UncoreCaches:[] SocketID:0} {Id:6 Threads:[6] Caches:[{Id:6 Size:32768 Type:Data Level:1} {Id:6 Size:32768 Type:Instruction Level:1} {Id:6 Size:262144 Type:Unified Level:2}] UncoreCaches:[] SocketID:0} {Id:7 Threads:[7] Caches:[{Id:7 Size:32768 Type:Data Level:1} {Id:7 Size:32768 Type:Instruction Level:1} {Id:7 Size:262144 Type:Unified Level:2}] UncoreCaches:[] SocketID:0}] Caches:[{Id:0 Size:52428800 Type:Unified Level:3}]}] CloudProvider:Azure InstanceType:Unknown InstanceID:f63520c0-c4ce-5743-af3a-5ffe95225017} Nov 16 11:53:20.294537 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.294480 1563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 16 11:53:20.294705 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.294667 1563 manager.go:230] Version: {KernelVersion:4.18.0-305.65.1.el8_4.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 410.84.202211032138-0 (Ootpa) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 16 11:53:20.295711 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.295626 1563 container_manager_linux.go:281] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 16 11:53:20.295789 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.295746 1563 container_manager_linux.go:286] "Creating Container Manager object based on Node Config" nodeConfig={RuntimeCgroupsName:/system.slice/crio.service SystemCgroupsName:/system.slice KubeletCgroupsName: ContainerRuntime:remote CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:systemd KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[cpu:{i:{value:500 scale:-3} d:{Dec:} s:500m Format:DecimalSI} memory:{i:{value:1073741824 scale:0} d:{Dec:} s:1Gi Format:BinarySI}] HardEvictionThresholds:[{Signal:imagefs.available Operator:LessThan Value:{Quantity: Percentage:0.15} GracePeriod:0s MinReclaim:} {Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:} {Signal:nodefs.available Operator:LessThan Value:{Quantity: Percentage:0.1} GracePeriod:0s MinReclaim:} {Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity: Percentage:0.05} GracePeriod:0s MinReclaim:}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerPolicyOptions:map[] ExperimentalTopologyManagerScope:container ExperimentalCPUManagerReconcilePeriod:10s ExperimentalMemoryManagerPolicy:None ExperimentalMemoryManagerReservedMemory:[] ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none} Nov 16 11:53:20.295789 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.295778 1563 topology_manager.go:133] "Creating topology manager with policy per scope" topologyPolicyName="none" topologyScopeName="container" Nov 16 11:53:20.295789 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.295792 1563 container_manager_linux.go:321] "Creating device plugin manager" devicePluginEnabled=true Nov 16 11:53:20.296193 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.296169 1563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 16 11:53:20.296459 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.296436 1563 state_mem.go:36] "Initialized new in-memory state store" Nov 16 11:53:20.296579 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.296560 1563 util_unix.go:104] "Using this format as endpoint is deprecated, please consider using full url format." deprecatedFormat="/var/run/crio/crio.sock" fullURLFormat="unix:///var/run/crio/crio.sock" Nov 16 11:53:20.301879 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.301798 1563 remote_runtime.go:115] "Using CRI v1 runtime API" Nov 16 11:53:20.301879 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.301843 1563 util_unix.go:104] "Using this format as endpoint is deprecated, please consider using full url format." deprecatedFormat="/var/run/crio/crio.sock" fullURLFormat="unix:///var/run/crio/crio.sock" Nov 16 11:53:20.303387 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.303360 1563 remote_image.go:91] "Using CRI v1 image API" Nov 16 11:53:20.303387 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.303404 1563 server.go:1039] "Cloud provider determined current node" nodeName="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:53:20.303518 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.303422 1563 server.go:1180] "Using root directory" path="/var/lib/kubelet" Nov 16 11:53:20.303996 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.303938 1563 kubelet.go:428] "Attempting to sync node with API server" Nov 16 11:53:20.303996 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.303975 1563 kubelet.go:290] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 16 11:53:20.303996 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.304001 1563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 16 11:53:20.304112 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.304015 1563 kubelet.go:301] "Adding apiserver pod source" Nov 16 11:53:20.304112 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.304032 1563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 16 11:53:20.305531 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.305447 1563 kuberuntime_manager.go:249] "Container runtime initialized" containerRuntime="cri-o" version="1.23.3-20.rhaos4.10.git89344de.el8" apiVersion="v1" Nov 16 11:53:20.307180 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307089 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/vsphere-volume" Nov 16 11:53:20.307180 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307122 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 16 11:53:20.307180 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307174 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/rbd" Nov 16 11:53:20.307180 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307185 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/aws-ebs" Nov 16 11:53:20.307357 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307211 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/gce-pd" Nov 16 11:53:20.307357 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307220 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/cinder" Nov 16 11:53:20.307357 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307239 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/azure-disk" Nov 16 11:53:20.307357 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307250 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/azure-file" Nov 16 11:53:20.307478 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307441 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 16 11:53:20.307478 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307473 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 16 11:53:20.307545 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307483 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 16 11:53:20.307545 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307507 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 16 11:53:20.307545 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307517 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 16 11:53:20.307545 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307528 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 16 11:53:20.307545 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307541 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/glusterfs" Nov 16 11:53:20.307672 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307550 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/quobyte" Nov 16 11:53:20.307672 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307558 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/cephfs" Nov 16 11:53:20.307672 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307567 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 16 11:53:20.307672 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307581 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 16 11:53:20.307672 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307595 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/flocker" Nov 16 11:53:20.307672 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307603 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 16 11:53:20.307672 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307612 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 16 11:53:20.307672 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307621 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 16 11:53:20.307672 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307630 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/storageos" Nov 16 11:53:20.307877 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.307856 1563 plugins.go:643] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 16 11:53:20.308085 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.308062 1563 server.go:1244] "Started kubelet" Nov 16 11:53:20.308712 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.308591 1563 server.go:150] "Starting to listen" address="0.0.0.0" port=10250 Nov 16 11:53:20.309069 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:20.309032 1563 kubelet.go:1367] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Nov 16 11:53:20.309151 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes Kubelet. Nov 16 11:53:20.309992 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.309958 1563 server.go:410] "Adding debug handlers to kubelet server" Nov 16 11:53:20.310520 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.310458 1563 certificate_manager.go:270] kubernetes.io/kubelet-serving: Certificate rotation is enabled Nov 16 11:53:20.310520 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.310496 1563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Nov 16 11:53:20.310913 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.310888 1563 volume_manager.go:289] "The desired_state_of_world populator starts" Nov 16 11:53:20.310955 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.310914 1563 volume_manager.go:291] "Starting Kubelet Volume Manager" Nov 16 11:53:20.311081 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.311047 1563 desired_state_of_world_populator.go:147] "Desired state populator starts to run" Nov 16 11:53:20.311213 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:20.311158065Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=2bd1c636-49ef-41e3-a2cc-05b51f07d020 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:53:20.311901 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:53:20.311870170Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40 not found" id=2bd1c636-49ef-41e3-a2cc-05b51f07d020 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:53:20.312316 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:20.312287 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:53:20.315499 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.315480 1563 factory.go:153] Registering CRI-O factory Nov 16 11:53:20.315636 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.315610 1563 factory.go:55] Registering systemd factory Nov 16 11:53:20.315714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.315705 1563 factory.go:103] Registering Raw factory Nov 16 11:53:20.315815 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.315794 1563 manager.go:1203] Started watching for new ooms in manager Nov 16 11:53:20.316480 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.316408 1563 manager.go:304] Starting recovery of all containers Nov 16 11:53:20.370546 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.370513 1563 manager.go:309] Recovery completed Nov 16 11:53:20.411772 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.411715 1563 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach" Nov 16 11:53:20.411945 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:20.411715 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:20.411945 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.411854 1563 kubelet_node_status.go:424] "Adding label from cloud provider" labelKey="beta.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:53:20.411945 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.411887 1563 kubelet_node_status.go:426] "Adding node label from cloud provider" labelKey="node.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:53:20.411945 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.411910 1563 kubelet_node_status.go:437] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/zone" labelValue="0" Nov 16 11:53:20.411945 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.411921 1563 kubelet_node_status.go:439] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/zone" labelValue="0" Nov 16 11:53:20.411945 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.411932 1563 kubelet_node_status.go:443] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/region" labelValue="westus" Nov 16 11:53:20.411945 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.411942 1563 kubelet_node_status.go:445] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/region" labelValue="westus" Nov 16 11:53:20.412698 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.412673 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientMemory" Nov 16 11:53:20.412753 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.412715 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasNoDiskPressure" Nov 16 11:53:20.412753 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.412730 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientPID" Nov 16 11:53:20.412819 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.412765 1563 kubelet_node_status.go:72] "Attempting to register node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:53:20.430980 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.430946 1563 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach" Nov 16 11:53:20.430980 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.430980 1563 kubelet_node_status.go:424] "Adding label from cloud provider" labelKey="beta.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:53:20.430980 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.430991 1563 kubelet_node_status.go:426] "Adding node label from cloud provider" labelKey="node.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:53:20.431165 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.431003 1563 kubelet_node_status.go:437] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/zone" labelValue="0" Nov 16 11:53:20.431165 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.431013 1563 kubelet_node_status.go:439] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/zone" labelValue="0" Nov 16 11:53:20.431165 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.431022 1563 kubelet_node_status.go:443] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/region" labelValue="westus" Nov 16 11:53:20.431165 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.431031 1563 kubelet_node_status.go:445] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/region" labelValue="westus" Nov 16 11:53:20.431506 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.431483 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientMemory" Nov 16 11:53:20.432173 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.432152 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasNoDiskPressure" Nov 16 11:53:20.432243 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.432186 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientPID" Nov 16 11:53:20.432852 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.432822 1563 cpu_manager.go:214] "Starting CPU manager" policy="none" Nov 16 11:53:20.432909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.432868 1563 cpu_manager.go:215] "Reconciling" reconcilePeriod="10s" Nov 16 11:53:20.433003 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.432980 1563 state_mem.go:36] "Initialized new in-memory state store" Nov 16 11:53:20.447051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.447016 1563 policy_none.go:49] "None policy: Start" Nov 16 11:53:20.447688 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.447649 1563 memory_manager.go:168] "Starting memorymanager" policy="None" Nov 16 11:53:20.447688 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.447691 1563 state_mem.go:35] "Initializing new in-memory state store" Nov 16 11:53:20.456723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.456656 1563 container_manager_linux.go:449] "Updating kernel flag" flag="kernel/panic" expectedValue=10 actualValue=0 Nov 16 11:53:20.456949 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.456890 1563 container_manager_linux.go:449] "Updating kernel flag" flag="vm/overcommit_memory" expectedValue=1 actualValue=0 Nov 16 11:53:20.462093 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods.slice. Nov 16 11:53:20.476320 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable.slice. Nov 16 11:53:20.482310 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-besteffort.slice. Nov 16 11:53:20.485783 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.485707 1563 manager.go:247] "Starting Device Plugin manager" Nov 16 11:53:20.485783 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.485778 1563 manager.go:610] "Failed to read data from checkpoint" checkpoint="kubelet_internal_checkpoint" err="checkpoint is not found" Nov 16 11:53:20.486048 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.486025 1563 manager.go:289] "Serving device plugin registration server on socket" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 16 11:53:20.486232 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:20.486203 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:20.486232 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.486216 1563 plugin_watcher.go:52] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 16 11:53:20.486372 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.486314 1563 plugin_manager.go:112] "The desired_state_of_world populator (plugin watcher) starts" Nov 16 11:53:20.486372 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.486341 1563 plugin_manager.go:114] "Starting Kubelet Plugin Manager" Nov 16 11:53:20.503083 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.503056 1563 kubelet_network_linux.go:57] "Initialized protocol iptables rules." protocol=IPv4 Nov 16 11:53:20.512061 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:20.512038 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:20.585926 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.585840 1563 kubelet_network_linux.go:57] "Initialized protocol iptables rules." protocol=IPv6 Nov 16 11:53:20.585926 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.585880 1563 status_manager.go:161] "Starting to sync pod status with apiserver" Nov 16 11:53:20.585926 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.585927 1563 kubelet.go:2051] "Starting kubelet main sync loop" Nov 16 11:53:20.586086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:20.585988 1563 kubelet.go:2075] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Nov 16 11:53:20.612232 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:20.612154 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:20.686589 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:20.686504 1563 kubelet.go:2137] "SyncLoop ADD" source="file" pods=[] Nov 16 11:53:20.712852 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:20.712781 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:20.814034 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:20.813961 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:20.915228 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:20.915080 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:21.016189 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:21.016086 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:21.117086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:21.116972 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:21.217957 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:21.217745 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:21.318029 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:21.317956 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:21.418909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:21.418824 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:21.520006 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:21.519889 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:21.620618 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:21.620519 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:21.721574 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:21.721480 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:21.822037 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:21.821793 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:21.922635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:21.922551 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:22.023640 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:22.023539 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:22.124640 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:22.124567 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:22.225535 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:22.225457 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:22.326306 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:22.326195 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:22.427381 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:22.427245 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:22.527668 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:22.527588 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:22.628167 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:22.628082 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:22.729085 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:22.728935 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:22.829099 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:22.829009 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:22.929557 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:22.929473 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:23.030504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:23.030344 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:23.131318 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:23.131212 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:23.232422 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:23.232282 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:23.333068 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:23.332866 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:23.433656 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:23.433571 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:23.534614 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:23.534538 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:23.635218 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:23.635118 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:23.736040 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:23.735928 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:23.747799 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: NetworkManager-dispatcher.service: Succeeded. Nov 16 11:53:23.748619 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: NetworkManager-dispatcher.service: Consumed 180ms CPU time Nov 16 11:53:23.836984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:23.836873 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:23.937674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:23.937540 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:24.038383 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:24.038294 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:24.139121 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:24.138976 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:24.239610 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:24.239473 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:24.340428 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:24.340336 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:24.441246 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:24.441104 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:24.542557 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:24.542346 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:24.642749 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:24.642599 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:24.743612 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:24.743494 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:24.844770 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:24.844691 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:24.945502 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:24.945393 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:25.046305 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:25.046218 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:25.147424 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:25.147249 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:25.247667 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:25.247573 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:25.348414 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:25.348297 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:25.449200 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:25.448987 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:25.488399 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:25.488289 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:53:25.550117 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:25.549995 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:25.651291 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:25.651204 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:25.751991 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:25.751829 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:25.853098 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:25.852986 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:25.953715 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:25.953600 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:26.054714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:26.054458 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:26.155420 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:26.155289 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:26.255996 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:26.255954 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:26.356191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:26.356043 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:26.457083 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:26.457006 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:26.558085 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:26.558002 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:26.658771 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:26.658619 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:26.759461 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:26.759365 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:26.860616 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:26.860504 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:26.961516 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:26.961378 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:27.062153 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:27.062062 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:27.162798 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:27.162729 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:27.263715 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:27.263466 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:27.364297 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:27.364174 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:27.465161 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:27.464989 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:27.566325 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:27.566163 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:27.666426 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:27.666349 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:27.767156 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:27.767045 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:27.867256 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:27.867187 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:27.967808 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:27.967687 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:28.068528 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:28.068450 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:28.169322 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:28.169165 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:28.269893 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:28.269810 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:28.370939 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:28.370871 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:28.471591 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:28.471471 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:28.572643 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:28.572562 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:28.673578 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:28.673503 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:28.774407 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:28.774207 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:28.874996 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:28.874919 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:28.975640 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:28.975509 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:29.076431 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:29.076245 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:29.177285 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:29.177116 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:29.278142 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:29.278030 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:29.379022 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:29.378930 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:29.479814 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:29.479700 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:29.580733 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:29.580614 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:29.681478 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:29.681261 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:29.781471 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:29.781362 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:29.882270 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:29.882144 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:29.982972 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:29.982811 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:30.084120 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:30.084028 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:30.184919 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:30.184795 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:30.285661 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:30.285500 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:30.312231 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:30.312060 1563 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": context deadline exceeded Nov 16 11:53:30.385690 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:30.385612 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:30.486549 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:30.486461 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:30.486622 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:30.486552 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:30.488954 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:30.488923 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:53:30.586946 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:30.586859 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:30.687238 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:30.687115 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:30.788178 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:30.788057 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:30.889206 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:30.889045 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:30.989916 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:30.989794 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:31.090064 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:31.089977 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:31.190733 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:31.190638 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:31.291759 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:31.291671 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:31.392878 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:31.392802 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:31.493847 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:31.493676 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:31.594880 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:31.594840 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:31.695991 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:31.695856 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:31.796275 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:31.796099 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:31.897031 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:31.896887 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:31.997958 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:31.997824 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:32.098458 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:32.098308 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:32.199086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:32.199019 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:32.300170 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:32.300063 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:32.400660 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:32.400444 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:32.501856 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:32.501744 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:32.602074 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:32.601983 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:32.702518 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:32.702254 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:32.803206 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:32.803075 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:32.903960 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:32.903915 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:33.004625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:33.004523 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:33.105037 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:33.104932 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:33.205478 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:33.205360 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:33.306817 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:33.306613 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:33.406826 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:33.406717 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:33.507097 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:33.506993 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:33.607160 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:33.607040 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:33.708308 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:33.708190 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:33.809485 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:33.809378 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:33.910591 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:33.910410 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:34.011612 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:34.011530 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:34.112488 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:34.112363 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:34.213366 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:34.213241 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:34.313381 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:34.313302 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:34.414550 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:34.414433 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:34.514961 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:34.514790 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:34.615177 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:34.615085 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:34.715376 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:34.715300 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:34.815605 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:34.815409 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:34.915881 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:34.915785 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:35.016898 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:35.016771 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:35.118073 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:35.118022 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:35.218706 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:35.218642 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:35.320028 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:35.319946 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:35.420731 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:35.420513 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:35.465391 ropatil1611az-gs6zj-worker-westus-j4ngg chronyd[1211]: Selected source PHC0 Nov 16 11:53:35.465633 ropatil1611az-gs6zj-worker-westus-j4ngg chronyd[1211]: System clock TAI offset set to 37 seconds Nov 16 11:53:35.490071 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:35.489954 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:53:35.521647 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:35.521494 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:35.621935 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:35.621825 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:35.723149 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:35.722936 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:35.823735 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:35.823648 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:35.924812 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:35.924686 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:36.025095 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:36.024865 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:36.125996 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:36.125921 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:36.227022 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:36.226944 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:36.331333 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:36.328055 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:36.428631 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:36.428499 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:36.528773 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:36.528671 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:36.629329 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:36.629303 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:36.730245 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:36.730219 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:36.830834 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:36.830727 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:36.931020 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:36.930876 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:37.031742 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:37.031653 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:37.131896 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:37.131813 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:37.232081 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:37.231924 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:37.333056 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:37.332963 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:37.433960 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:37.433893 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:37.534459 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:37.534348 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:37.634635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:37.634551 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:37.735684 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:37.735615 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:37.835838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:37.835789 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:37.936481 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:37.936415 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:38.037428 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:38.037357 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:38.138026 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:38.137846 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:38.238454 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:38.238332 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:38.339110 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:38.338996 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:38.439493 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:38.439313 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:38.539486 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:38.539408 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:38.640244 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:38.640162 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:38.741370 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:38.741227 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:38.841772 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:38.841674 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:38.942769 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:38.942689 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:39.043866 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:39.043714 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:39.144385 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:39.144261 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:39.245494 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:39.245384 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:39.346507 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:39.346440 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:39.446565 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:39.446498 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:39.547503 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:39.547418 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:39.648374 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:39.648226 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:39.748795 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:39.748729 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:39.849649 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:39.849578 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:39.950793 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:39.950677 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:40.051745 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:40.051660 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:40.152884 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:40.152697 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:40.253531 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:40.253393 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:40.353771 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:40.353706 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:40.454841 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:40.454770 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:40.487144 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:40.487049 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:40.490775 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:40.490744 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:53:40.514091 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:40.513948 1563 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:53:40.555658 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:40.555594 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:40.656118 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:40.656045 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:40.757109 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:40.757031 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:40.858251 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:40.858182 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:40.959096 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:40.958971 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:41.059917 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:41.059822 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:41.161046 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:41.160862 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:41.261981 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:41.261915 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:41.363146 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:41.363057 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:41.464374 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:41.464177 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:41.535468 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1391]: Nov 16 11:53:41.534 WARN Failed to get fabric address from DHCP: maximum number of retries (60) reached Nov 16 11:53:41.535468 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1391]: Nov 16 11:53:41.535 INFO using fallback address Nov 16 11:53:41.546510 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1391]: Nov 16 11:53:41.546 INFO Fetching http://168.63.129.16/?comp=versions: Attempt #1 Nov 16 11:53:41.550170 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1391]: Nov 16 11:53:41.550 INFO Fetch successful Nov 16 11:53:41.550590 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1391]: Nov 16 11:53:41.550 INFO Fetching http://168.63.129.16/machine/?comp=goalstate: Attempt #1 Nov 16 11:53:41.552146 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1391]: Nov 16 11:53:41.552 INFO Fetch successful Nov 16 11:53:41.561525 ropatil1611az-gs6zj-worker-westus-j4ngg afterburn[1391]: Nov 16 11:53:41.561 INFO Posting http://168.63.129.16/machine/?comp=health: Attempt #1 Nov 16 11:53:41.564555 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:41.564490 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:41.568749 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Afterburn (Check In). Nov 16 11:53:41.569093 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Multi-User System. Nov 16 11:53:41.569143 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reached target Graphical Interface. Nov 16 11:53:41.570553 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Update UTMP about System Runlevel Changes... Nov 16 11:53:41.578814 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-update-utmp-runlevel.service: Succeeded. Nov 16 11:53:41.579905 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Update UTMP about System Runlevel Changes. Nov 16 11:53:41.580065 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Startup finished in 4.649s (kernel) + 7.402s (initrd) + 33.783s (userspace) = 45.835s. Nov 16 11:53:41.580252 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-update-utmp-runlevel.service: Consumed 5ms CPU time Nov 16 11:53:41.665007 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:41.664912 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:41.765971 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:41.765821 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:41.866305 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:41.866204 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:41.966840 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:41.966732 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:42.068007 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:42.067867 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:42.168929 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:42.168818 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:42.269868 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:42.269755 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:42.370897 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:42.370802 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:42.471794 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:42.471729 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:42.573033 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:42.572926 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:42.674065 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:42.673873 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:42.774933 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:42.774826 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:42.876067 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:42.875959 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:42.976759 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:42.976651 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:43.074691 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-hostnamed.service: Succeeded. Nov 16 11:53:43.075659 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-hostnamed.service: Consumed 52ms CPU time Nov 16 11:53:43.077532 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:43.077446 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:43.178173 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:43.178070 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:43.278604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:43.278427 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:43.378641 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:43.378584 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:43.479603 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:43.479540 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:43.579934 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:43.579704 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:43.680151 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:43.680073 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:43.781221 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:43.781193 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:43.882086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:43.882009 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:43.983215 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:43.983107 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:44.084211 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:44.084103 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:44.185360 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:44.185159 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:44.285424 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:44.285305 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:44.385893 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:44.385826 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:44.487112 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:44.486918 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:44.587057 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:44.587012 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:44.688163 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:44.688066 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:44.789218 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:44.789059 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:44.890259 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:44.890119 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:44.991198 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:44.991107 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:45.092451 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:45.092352 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:45.193448 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:45.193315 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:45.275602 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:45.275525 1563 certificate_manager.go:471] kubernetes.io/kube-apiserver-client-kubelet: Failed while requesting a signed certificate from the control plane: cannot create certificate signing request: Post "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/certificates.k8s.io/v1/certificatesigningrequests": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:53:45.294108 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:45.294045 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:45.394927 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:45.394710 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:45.492490 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:45.492454 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:53:45.495839 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:45.495722 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:45.596541 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:45.596413 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:45.697782 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:45.697664 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:45.798232 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:45.798119 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:45.899205 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:45.899091 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:46.000375 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:46.000199 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:46.101282 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:46.101186 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:46.202317 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:46.202228 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:46.303575 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:46.303330 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:46.404230 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:46.404088 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:46.504561 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:46.504490 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:46.604764 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:46.604676 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:46.704933 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:46.704823 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:46.806001 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:46.805937 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:46.906793 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:46.906656 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:47.007535 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:47.007434 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:47.108465 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:47.108373 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:47.209466 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:47.209313 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:47.310500 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:47.310384 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:47.374644 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:47.374580 1563 certificate_manager.go:270] kubernetes.io/kube-apiserver-client-kubelet: Rotating certificates Nov 16 11:53:47.411501 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:47.411438 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:47.512247 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:47.512053 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:47.612523 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:47.612379 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:47.712587 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:47.712511 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:47.813755 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:47.813612 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:47.914824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:47.914722 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:48.015986 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:48.015866 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:48.117049 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:48.116937 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:48.217936 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:48.217869 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:48.319082 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:48.319012 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:48.419916 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:48.419702 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:48.520006 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:48.519882 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:48.621176 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:48.621069 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:48.721729 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:48.721588 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:48.822518 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:48.822444 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:48.922665 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:48.922522 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:49.023306 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:49.023186 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:49.124283 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:49.124162 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:49.224683 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:49.224595 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:49.325099 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:49.324942 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:49.425788 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:49.425675 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:49.526713 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:49.526640 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:49.627365 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:49.627259 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:49.727723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:49.727645 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:49.828872 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:49.828781 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:49.929893 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:49.929770 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:50.030931 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.030846 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:50.131930 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.131807 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:50.232976 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.232794 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:50.306588 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:50.306458 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.Service: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:53:50.306588 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:50.306502 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.Node: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dropatil1611az-gs6zj-worker-westus-j4ngg&limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:53:50.306588 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.306590 1563 trace.go:205] Trace[1477312949]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:53:20.304) (total time: 30001ms): Nov 16 11:53:50.306588 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1477312949]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30001ms (11:53:50.306) Nov 16 11:53:50.306588 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1477312949]: [30.001589046s] [30.001589046s] END Nov 16 11:53:50.306883 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.306624 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:53:50.306883 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.306593 1563 trace.go:205] Trace[233010891]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:53:20.304) (total time: 30001ms): Nov 16 11:53:50.306883 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[233010891]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dropatil1611az-gs6zj-worker-westus-j4ngg&limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30001ms (11:53:50.306) Nov 16 11:53:50.306883 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[233010891]: [30.001615246s] [30.001615246s] END Nov 16 11:53:50.306883 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.306657 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dropatil1611az-gs6zj-worker-westus-j4ngg&limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:53:50.309250 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.309213 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/ropatil1611az-gs6zj-worker-westus-j4ngg": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:53:50.310611 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.310401 1563 event.go:276] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef0b7d27e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 308015742, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 308015742, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'Post "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp 10.0.0.4:6443: i/o timeout'(may retry after sleeping) Nov 16 11:53:50.311635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:50.311536 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.CSIDriver: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:53:50.311635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.311634 1563 trace.go:205] Trace[2045263124]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:53:20.311) (total time: 30000ms): Nov 16 11:53:50.311635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[2045263124]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30000ms (11:53:50.311) Nov 16 11:53:50.311635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[2045263124]: [30.000391035s] [30.000391035s] END Nov 16 11:53:50.311831 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.311667 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:53:50.333463 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.333393 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:50.414000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.413869 1563 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp 10.0.0.4:6443: i/o timeout" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:53:50.434492 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.434413 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:50.487992 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.487790 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:50.493542 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.493439 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:53:50.534951 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.534842 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:50.586665 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:53:50.586492 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.RuntimeClass: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:53:50.586665 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.586663 1563 trace.go:205] Trace[387141448]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:53:20.586) (total time: 30000ms): Nov 16 11:53:50.586665 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[387141448]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30000ms (11:53:50.586) Nov 16 11:53:50.586665 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[387141448]: [30.000464618s] [30.000464618s] END Nov 16 11:53:50.586893 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.586705 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:53:50.615087 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.615003 1563 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach" Nov 16 11:53:50.615087 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.615059 1563 kubelet_node_status.go:424] "Adding label from cloud provider" labelKey="beta.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:53:50.615087 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.615085 1563 kubelet_node_status.go:426] "Adding node label from cloud provider" labelKey="node.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:53:50.615087 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.615097 1563 kubelet_node_status.go:437] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/zone" labelValue="0" Nov 16 11:53:50.615288 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.615106 1563 kubelet_node_status.go:439] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/zone" labelValue="0" Nov 16 11:53:50.615288 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.615115 1563 kubelet_node_status.go:443] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/region" labelValue="westus" Nov 16 11:53:50.615288 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.615124 1563 kubelet_node_status.go:445] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/region" labelValue="westus" Nov 16 11:53:50.616066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.615996 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientMemory" Nov 16 11:53:50.616066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.616055 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasNoDiskPressure" Nov 16 11:53:50.616066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.616086 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientPID" Nov 16 11:53:50.616254 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:53:50.616156 1563 kubelet_node_status.go:72] "Attempting to register node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:53:50.635161 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.635065 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:50.735673 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.735606 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:50.836278 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.836229 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:50.916288 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.916234 1563 controller.go:144] failed to ensure lease exists, will retry in 800ms, error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:53:50.936981 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:50.936880 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:51.037984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:51.037824 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:51.139019 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:51.138858 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:51.239806 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:51.239715 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:51.340554 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:51.340512 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:51.441576 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:51.441395 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:51.541894 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:51.541763 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:51.642998 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:51.642933 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:51.744122 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:51.743943 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:51.845295 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:51.845150 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:51.945488 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:51.945387 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:52.046454 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:52.046302 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:52.147502 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:52.147374 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:52.248407 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:52.248297 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:52.349091 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:52.349007 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:52.449274 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:52.449215 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:52.549448 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:52.549361 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:52.649862 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:52.649688 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:52.750781 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:52.750699 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:52.851853 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:52.851761 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:52.952936 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:52.952731 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:53.053871 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:53.053791 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:53.154053 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:53.153973 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:53.255030 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:53.254833 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:53.355894 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:53.355793 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:53.456839 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:53.456727 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:53.557302 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:53.557118 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:53.657904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:53.657774 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:53.758922 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:53.758802 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:53.858970 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:53.858879 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:53.959905 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:53.959816 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:54.060918 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:54.060831 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:54.161892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:54.161756 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:54.262779 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:54.262679 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:54.363743 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:54.363637 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:54.464894 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:54.464690 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:54.565583 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:54.565507 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:54.666122 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:54.666045 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:54.767116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:54.766967 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:54.867345 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:54.867222 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:54.968310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:54.968197 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:55.069373 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:55.069123 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:55.169396 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:55.169286 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:55.269819 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:55.269771 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:55.370004 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:55.369928 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:55.470939 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:55.470808 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:55.494778 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:55.494712 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:53:55.571087 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:55.570953 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:55.671776 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:55.671587 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:55.772235 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:55.772104 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:55.872601 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:55.872557 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:55.973226 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:55.973000 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:56.073983 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:56.073913 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:56.174990 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:56.174883 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:56.275809 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:56.275595 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:56.376763 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:56.376588 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:56.477661 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:56.477547 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:56.578833 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:56.578625 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:56.679728 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:56.679600 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:56.780624 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:56.780541 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:56.881468 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:56.881392 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:56.982521 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:56.982408 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:57.083414 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:57.083340 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:57.184436 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:57.184297 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:57.284912 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:57.284810 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:57.385461 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:57.385407 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:57.486528 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:57.486310 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:57.580698 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: hv_balloon: Max. dynamic memory size: 16384 MB Nov 16 11:53:57.586529 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:57.586460 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:57.686886 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:57.686821 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:57.787711 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:57.787545 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:57.888474 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:57.888358 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:57.989310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:57.989272 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:58.090394 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:58.090271 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:58.191306 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:58.191230 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:58.291907 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:58.291805 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:58.392781 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:58.392655 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:58.493745 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:58.493666 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:58.594346 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:58.594243 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:58.694556 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:58.694377 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:58.795102 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:58.795016 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:58.895722 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:58.895648 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:58.996014 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:58.995872 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:59.096827 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:59.096679 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:59.197709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:59.197620 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:59.298360 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:59.298251 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:59.398569 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:59.398463 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:59.499518 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:59.499410 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:59.600679 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:59.600605 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:59.701218 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:59.701122 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:59.802161 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:59.802091 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:53:59.903109 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:53:59.902987 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:00.003306 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:00.003171 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:00.103458 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:00.103349 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:00.204742 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:00.204582 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:00.305617 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:00.305523 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:00.406150 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:00.406057 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:00.488716 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:00.488611 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:00.496278 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:00.496245 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:54:00.506634 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:00.506601 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:00.606956 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:00.606921 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:00.708031 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:00.707927 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:00.808248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:00.808101 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:00.909337 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:00.909205 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:01.010284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:01.010178 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:01.111333 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:01.111177 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:01.211633 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:01.211510 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:01.312051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:01.311939 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:01.412201 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:01.412057 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:01.513106 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:01.513056 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:01.613836 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:01.613714 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:01.714485 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:01.714362 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:01.717785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:01.717733 1563 controller.go:144] failed to ensure lease exists, will retry in 1.6s, error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:54:01.815179 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:01.815107 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:01.915394 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:01.915281 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:02.016580 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:02.016413 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:02.117372 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:02.117327 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:02.218450 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:02.218376 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:02.319062 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:02.318903 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:02.419146 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:02.419036 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:02.519217 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:02.519106 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:02.619602 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:02.619529 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:02.720039 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:02.719962 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:02.820404 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:02.820332 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:02.921560 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:02.921371 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:03.022597 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:03.022502 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:03.122914 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:03.122809 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:03.223905 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:03.223778 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:03.324176 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:03.324018 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:03.424820 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:03.424692 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:03.525209 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:03.525087 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:03.625412 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:03.625263 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:03.726556 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:03.726441 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:03.827498 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:03.827287 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:03.928518 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:03.928434 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:04.029477 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:04.029405 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:04.130342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:04.130294 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:04.231492 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:04.231366 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:04.332547 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:04.332435 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:04.432858 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:04.432639 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:04.533260 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:04.533214 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:04.634166 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:04.634098 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:04.734426 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:04.734272 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:04.835467 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:04.835396 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:04.936313 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:04.936208 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:05.037233 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:05.037070 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:05.138236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:05.138189 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:05.238817 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:05.238740 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:05.339020 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:05.338911 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:05.439090 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:05.438994 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:05.498209 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:05.498174 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:54:05.539200 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:05.539108 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:05.640487 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:05.640343 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:05.741659 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:05.741558 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:05.842053 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:05.841949 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:05.942622 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:05.942446 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:06.043376 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:06.043313 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:06.143729 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:06.143628 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:06.244906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:06.244679 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:06.345545 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:06.345454 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:06.445726 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:06.445612 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:06.545907 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:06.545765 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:06.646382 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:06.646292 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:06.746713 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:06.746638 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:06.847897 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:06.847785 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:06.948806 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:06.948766 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:07.049723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:07.049626 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:07.150886 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:07.150734 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:07.251815 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:07.251705 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:07.352575 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:07.352460 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:07.452774 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:07.452619 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:07.553574 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:07.553531 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:07.653774 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:07.653707 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:07.754252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:07.754089 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:07.854539 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:07.854442 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:07.954967 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:07.954854 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:08.055942 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:08.055812 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:08.157064 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:08.156938 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:08.257161 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:08.257093 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:08.357508 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:08.357430 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:08.458628 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:08.458534 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:08.559675 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:08.559607 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:08.660372 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:08.660209 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:08.761202 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:08.761121 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:08.862474 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:08.862362 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:08.963549 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:08.963360 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:09.064397 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:09.064326 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:09.164942 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:09.164844 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:09.266243 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:09.266056 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:09.366514 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:09.366358 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:09.466990 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:09.466900 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:09.568190 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:09.567981 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:09.668397 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:09.668305 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:09.768856 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:09.768760 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:09.869542 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:09.869417 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:09.970474 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:09.970393 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:10.070680 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:10.070537 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:10.171522 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:10.171381 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:10.272466 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:10.272384 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:10.373585 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:10.373501 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:10.474769 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:10.474593 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:10.488915 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:10.488842 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:10.499703 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:10.499672 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:54:10.575833 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:10.575712 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:10.676632 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:10.676523 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:10.776912 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:10.776804 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:10.877293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:10.877190 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:10.977771 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:10.977726 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:11.078931 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:11.078792 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:11.179806 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:11.179703 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:11.279942 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:11.279853 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:11.380913 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:11.380810 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:11.481932 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:11.481832 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:11.582353 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:11.582263 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:11.683118 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:11.682983 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:11.783160 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:11.783069 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:11.883358 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:11.883242 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:11.984338 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:11.984192 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:12.084888 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:12.084787 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:12.185823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:12.185747 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:12.286219 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:12.286074 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:12.386949 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:12.386866 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:12.487594 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:12.487506 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:12.588555 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:12.588469 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:12.689358 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:12.689289 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:12.789512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:12.789413 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:12.890008 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:12.889892 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:12.990932 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:12.990781 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:13.091921 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:13.091830 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:13.192969 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:13.192818 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:13.294103 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:13.294057 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:13.319210 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:13.319079 1563 controller.go:144] failed to ensure lease exists, will retry in 3.2s, error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:54:13.395142 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:13.395069 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:13.496397 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:13.496245 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:13.596607 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:13.596534 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:13.697867 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:13.697740 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:13.798034 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:13.797899 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:13.898289 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:13.898256 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:13.998435 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:13.998403 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:14.099230 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:14.099147 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:14.200090 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:14.200049 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:14.300419 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:14.300340 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:14.401336 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:14.401189 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:14.502239 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:14.502165 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:14.602673 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:14.602628 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:14.703210 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:14.703037 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:14.804267 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:14.804160 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:14.905174 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:14.905084 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:15.005756 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:15.005581 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:15.106268 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:15.106115 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:15.207302 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:15.207263 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:15.307932 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:15.307780 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:15.409064 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:15.408943 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:15.500525 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:15.500440 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:54:15.509907 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:15.509821 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:15.610447 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:15.610331 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:15.711531 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:15.711386 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:15.811686 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:15.811582 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:15.911821 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:15.911691 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:16.012495 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:16.012407 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:16.112791 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:16.112716 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:16.213880 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:16.213674 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:16.314496 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:16.314424 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:16.415205 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:16.415118 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:16.515834 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:16.515677 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:16.616159 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:16.616094 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:16.716909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:16.716843 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:16.817045 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:16.816911 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:16.918013 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:16.917916 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:17.019161 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:17.019020 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:17.120156 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:17.120082 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:17.221148 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:17.221105 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:17.321463 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:17.321421 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:17.375606 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:17.375492 1563 certificate_manager.go:471] kubernetes.io/kube-apiserver-client-kubelet: Failed while requesting a signed certificate from the control plane: cannot create certificate signing request: Post "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/certificates.k8s.io/v1/certificatesigningrequests": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:17.422473 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:17.422434 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:17.522655 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:17.522494 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:17.623256 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:17.623147 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:17.723666 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:17.723451 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:17.824794 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:17.824668 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:17.925742 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:17.925559 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:18.026944 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:18.026676 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:18.127645 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:18.127556 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:18.228544 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:18.228495 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:18.329414 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:18.329301 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:18.429468 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:18.429359 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:18.530204 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:18.530042 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:18.630568 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:18.630484 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:18.731305 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:18.731214 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:18.831630 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:18.831559 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:18.932895 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:18.932673 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:19.033879 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:19.033795 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:19.134818 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:19.134730 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:19.235979 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:19.235815 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:19.336723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:19.336616 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:19.437317 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:19.437224 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:19.538035 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:19.537858 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:19.638495 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:19.638404 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:19.739059 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:19.738986 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:19.839331 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:19.839216 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:19.940471 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:19.940357 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:20.040799 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:20.040710 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:20.141250 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:20.141086 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:20.241712 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:20.241591 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:20.342680 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:20.342570 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:20.443224 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:20.443094 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:20.489928 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:20.489826 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:20.501714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:20.501659 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:54:20.543905 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:20.543791 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:20.617511 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:20.617405 1563 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp 10.0.0.4:6443: i/o timeout" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:54:20.644686 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:20.644604 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:20.745717 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:20.745559 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:20.845848 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:20.845773 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:20.946804 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:20.946723 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:21.017904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.017757 1563 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach" Nov 16 11:54:21.017904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.017817 1563 kubelet_node_status.go:424] "Adding label from cloud provider" labelKey="beta.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:54:21.017904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.017831 1563 kubelet_node_status.go:426] "Adding node label from cloud provider" labelKey="node.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:54:21.017904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.017845 1563 kubelet_node_status.go:437] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/zone" labelValue="0" Nov 16 11:54:21.017904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.017854 1563 kubelet_node_status.go:439] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/zone" labelValue="0" Nov 16 11:54:21.017904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.017865 1563 kubelet_node_status.go:443] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/region" labelValue="westus" Nov 16 11:54:21.017904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.017876 1563 kubelet_node_status.go:445] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/region" labelValue="westus" Nov 16 11:54:21.018849 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.018823 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientMemory" Nov 16 11:54:21.018916 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.018864 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasNoDiskPressure" Nov 16 11:54:21.018916 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.018882 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientPID" Nov 16 11:54:21.018916 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.018911 1563 kubelet_node_status.go:72] "Attempting to register node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:54:21.047446 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.047347 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:21.132488 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:54:21.132369 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.Node: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dropatil1611az-gs6zj-worker-westus-j4ngg&limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:21.132488 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.132475 1563 trace.go:205] Trace[1692896356]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:53:51.131) (total time: 30000ms): Nov 16 11:54:21.132488 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1692896356]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dropatil1611az-gs6zj-worker-westus-j4ngg&limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30000ms (11:54:21.132) Nov 16 11:54:21.132488 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1692896356]: [30.000639533s] [30.000639533s] END Nov 16 11:54:21.132488 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.132494 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dropatil1611az-gs6zj-worker-westus-j4ngg&limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:21.147539 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.147466 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:21.247687 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.247617 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:21.310355 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.310241 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/ropatil1611az-gs6zj-worker-westus-j4ngg": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:21.312723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:54:21.312600 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.Service: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:21.312723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.312689 1563 trace.go:205] Trace[651120468]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:53:51.311) (total time: 30000ms): Nov 16 11:54:21.312723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[651120468]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30000ms (11:54:21.312) Nov 16 11:54:21.312723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[651120468]: [30.000873781s] [30.000873781s] END Nov 16 11:54:21.312723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.312724 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:21.348420 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.348312 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:21.448906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.448825 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:21.549151 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.549044 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:21.560892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:54:21.560665 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.RuntimeClass: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:21.560892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.560798 1563 trace.go:205] Trace[452580429]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:53:51.559) (total time: 30001ms): Nov 16 11:54:21.560892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[452580429]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30001ms (11:54:21.560) Nov 16 11:54:21.560892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[452580429]: [30.001688103s] [30.001688103s] END Nov 16 11:54:21.560892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.560850 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:21.609392 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:54:21.609293 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.CSIDriver: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:21.609472 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.609402 1563 trace.go:205] Trace[1396379879]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:53:51.608) (total time: 30000ms): Nov 16 11:54:21.609472 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1396379879]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30000ms (11:54:21.609) Nov 16 11:54:21.609472 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1396379879]: [30.000963946s] [30.000963946s] END Nov 16 11:54:21.609472 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.609423 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:21.649297 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.649225 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:21.671629 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:21.671545 1563 certificate_manager.go:270] kubernetes.io/kube-apiserver-client-kubelet: Rotating certificates Nov 16 11:54:21.750516 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.750417 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:21.851609 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.851532 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:21.952238 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:21.952147 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:22.053413 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:22.053287 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:22.154591 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:22.154425 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:22.255444 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:22.255316 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:22.355622 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:22.355553 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:22.456704 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:22.456551 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:22.557661 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:22.557575 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:22.658070 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:22.657973 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:22.758403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:22.758145 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:22.858417 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:22.858313 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:22.958937 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:22.958825 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:23.059951 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:23.059779 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:23.160172 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:23.160098 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:23.260715 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:23.260643 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:23.361897 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:23.361760 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:23.462887 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:23.462811 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:23.563228 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:23.563111 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:23.663777 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:23.663634 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:23.764562 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:23.764459 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:23.864915 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:23.864853 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:23.965069 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:23.964921 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:24.065945 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:24.065842 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:24.166907 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:24.166815 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:24.268035 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:24.267805 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:24.368633 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:24.368539 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:24.469704 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:24.469588 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:24.570378 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:24.570204 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:24.670582 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:24.670520 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:24.771282 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:24.771198 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:24.871730 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:24.871660 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:24.972806 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:24.972669 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:25.073715 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:25.073591 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:25.174396 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:25.174240 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:25.274793 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:25.274635 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:25.374941 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:25.374844 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:25.476012 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:25.475774 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:25.502695 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:25.502665 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:54:25.575984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:25.575942 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:25.676213 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:25.676097 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:25.776804 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:25.776649 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:25.877618 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:25.877481 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:25.977769 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:25.977644 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:26.078706 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:26.078528 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:26.179580 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:26.179483 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:26.280000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:26.279882 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:26.380988 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:26.380852 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:26.481503 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:26.481358 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:26.521386 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:26.521300 1563 controller.go:144] failed to ensure lease exists, will retry in 6.4s, error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:54:26.582324 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:26.582252 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:26.682928 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:26.682756 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:26.783718 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:26.783541 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:26.884506 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:26.884390 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:26.985580 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:26.985409 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:27.086466 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:27.086412 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:27.187368 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:27.187326 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:27.287889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:27.287677 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:27.388510 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:27.388430 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:27.489626 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:27.489521 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:27.590368 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:27.590280 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:27.691396 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:27.691262 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:27.792460 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:27.792344 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:27.893270 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:27.893106 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:27.994164 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:27.994023 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:28.094917 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:28.094830 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:28.196064 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:28.195909 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:28.297165 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:28.297076 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:28.397861 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:28.397726 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:28.498761 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:28.498621 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:28.598988 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:28.598869 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:28.699484 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:28.699389 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:28.800180 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:28.800020 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:28.901290 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:28.901114 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:28.992643 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:28.992433 1563 event.go:276] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef0b7d27e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 308015742, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 308015742, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'Post "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp 10.0.0.4:6443: i/o timeout'(may retry after sleeping) Nov 16 11:54:29.001721 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:29.001640 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:29.102777 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:29.102682 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:29.203737 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:29.203644 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:29.304255 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:29.304118 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:29.404715 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:29.404488 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:29.505795 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:29.505658 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:29.606478 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:29.606378 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:29.707211 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:29.706948 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:29.807690 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:29.807578 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:29.908422 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:29.908343 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:30.009605 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:30.009455 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:30.109965 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:30.109857 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:30.210855 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:30.210772 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:30.311468 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:30.311205 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:30.411617 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:30.411497 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:30.490874 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:30.490763 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:30.503587 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:30.503547 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:54:30.512037 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:30.511925 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:30.612943 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:30.612774 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:30.713971 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:30.713859 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:30.814445 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:30.814276 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:30.915626 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:30.915419 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:31.016541 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:31.016476 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:31.117696 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:31.117591 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:31.219007 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:31.218809 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:31.319671 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:31.319589 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:31.420616 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:31.420551 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:31.521626 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:31.521448 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:31.622041 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:31.621915 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:31.722439 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:31.722367 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:31.822899 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:31.822762 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:31.923858 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:31.923823 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:32.024906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:32.024839 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:32.125294 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:32.125200 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:32.226395 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:32.226286 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:32.327293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:32.327256 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:32.428071 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:32.427887 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:32.529178 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:32.529138 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:32.629898 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:32.629789 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:32.730177 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:32.729920 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:32.830583 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:32.830471 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:32.930770 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:32.930676 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:33.031892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:33.031715 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:33.132912 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:33.132831 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:33.234004 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:33.233894 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:33.334688 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:33.334640 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:33.435591 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:33.435467 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:33.536697 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:33.536598 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:33.636896 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:33.636732 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:33.736911 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:33.736820 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:33.837559 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:33.837455 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:33.937946 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:33.937815 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:34.038989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:34.038918 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:34.140012 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:34.139924 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:34.241178 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:34.241024 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:34.341783 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:34.341733 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:34.442002 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:34.441863 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:34.542653 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:34.542503 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:34.643249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:34.643159 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:34.743672 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:34.743589 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:34.844323 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:34.844228 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:34.945341 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:34.945254 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:35.046180 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:35.046074 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:35.147262 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:35.147093 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:35.247604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:35.247537 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:35.348443 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:35.348342 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:35.448895 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:35.448765 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:35.505485 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:35.505446 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:54:35.549769 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:35.549666 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:35.649996 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:35.649889 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:35.751207 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:35.751062 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:35.851287 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:35.851199 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:35.952180 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:35.952059 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:36.052768 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:36.052546 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:36.153539 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:36.153435 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:36.254582 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:36.254430 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:36.355538 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:36.355493 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:36.456290 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:36.456220 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:36.556748 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:36.556658 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:36.657312 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:36.657106 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:36.757508 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:36.757454 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:36.858677 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:36.858599 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:36.959817 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:36.959615 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:37.060855 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:37.060786 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:37.161808 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:37.161711 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:37.262276 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:37.262091 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:37.363218 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:37.363109 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:37.463370 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:37.463302 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:37.563726 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:37.563553 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:37.664016 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:37.663963 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:37.764484 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:37.764395 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:37.864782 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:37.864710 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:37.965335 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:37.965285 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:38.065546 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:38.065455 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:38.166858 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:38.166712 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:38.267843 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:38.267721 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:38.368714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:38.368599 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:38.469932 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:38.469745 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:38.570029 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:38.569894 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:38.670646 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:38.670483 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:38.770758 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:38.770614 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:38.871433 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:38.871382 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:38.971936 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:38.971802 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:39.072472 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:39.072322 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:39.173413 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:39.173368 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:39.274340 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:39.274179 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:39.374822 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:39.374742 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:39.475896 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:39.475804 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:39.576930 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:39.576848 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:39.678155 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:39.677951 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:39.778281 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:39.778170 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:39.878931 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:39.878809 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:39.979939 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:39.979767 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:40.080285 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:40.080221 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:40.180752 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:40.180662 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:40.281155 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:40.280970 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:40.381625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:40.381507 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:40.482420 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:40.482326 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:40.491802 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:40.491696 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:40.506425 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:40.506386 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:54:40.582886 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:40.582735 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:40.683816 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:40.683689 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:40.784169 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:40.784084 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:40.885031 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:40.884938 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:40.986009 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:40.985931 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:41.087160 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:41.087053 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:41.188193 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:41.188043 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:41.289246 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:41.289160 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:41.390192 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:41.390038 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:41.491323 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:41.491019 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:41.591801 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:41.591717 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:41.692385 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:41.692295 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:41.793557 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:41.793397 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:41.894259 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:41.894177 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:41.995425 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:41.995307 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:42.096114 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:42.096038 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:42.197087 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:42.197003 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:42.297261 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:42.297104 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:42.398101 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:42.397884 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:42.499067 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:42.498957 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:42.600089 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:42.599983 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:42.700616 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:42.700378 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:42.801559 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:42.801484 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:42.902139 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:42.902045 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:42.922761 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:42.922650 1563 controller.go:144] failed to ensure lease exists, will retry in 7s, error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:54:43.002375 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:43.002211 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:43.103322 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:43.103261 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:43.204248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:43.204180 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:43.304935 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:43.304700 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:43.405361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:43.405296 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:43.505549 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:43.505458 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:43.606429 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:43.606340 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:43.707462 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:43.707341 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:43.808175 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:43.808084 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:43.908729 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:43.908578 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:44.009609 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:44.009473 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:44.110029 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:44.109884 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:44.211029 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:44.210882 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:44.311062 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:44.310962 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:44.411292 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:44.411202 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:44.512369 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:44.512173 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:44.612931 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:44.612854 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:44.713808 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:44.713725 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:44.814640 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:44.814533 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:44.915574 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:44.915484 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:45.016505 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:45.016401 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:45.117161 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:45.117022 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:45.218392 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:45.218288 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:45.319040 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:45.318932 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:45.419865 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:45.419714 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:45.508259 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:45.508196 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:54:45.519946 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:45.519854 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:45.620503 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:45.620438 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:45.721496 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:45.721394 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:45.822640 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:45.822512 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:45.923516 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:45.923382 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:46.024542 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:46.024359 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:46.125483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:46.125370 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:46.226577 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:46.226476 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:46.326778 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:46.326663 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:46.427400 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:46.427321 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:46.528281 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:46.528249 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:46.629428 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:46.629308 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:46.729604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:46.729511 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:46.830057 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:46.829970 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:46.931201 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:46.931076 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:47.031570 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:47.031489 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:47.131997 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:47.131934 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:47.233017 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:47.232844 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:47.333367 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:47.333281 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:47.433971 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:47.433895 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:47.534948 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:47.534825 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:47.635386 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:47.635305 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:47.735972 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:47.735884 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:47.837008 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:47.836932 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:47.937938 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:47.937855 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:48.038903 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:48.038820 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:48.139770 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:48.139638 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:48.240019 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:48.239972 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:48.340777 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:48.340596 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:48.441549 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:48.441352 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:48.542415 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:48.542325 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:48.642510 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:48.642442 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:48.743346 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:48.743161 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:48.844336 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:48.844273 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:48.945449 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:48.945363 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:49.046028 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:49.045904 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:49.146438 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:49.146366 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:49.247590 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:49.247500 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:49.348118 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:49.348037 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:49.448639 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:49.448564 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:49.548987 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:49.548919 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:49.649391 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:49.649226 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:49.750492 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:49.750421 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:49.851449 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:49.851383 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:49.951893 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:49.951821 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:50.052874 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:50.052783 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:50.153041 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:50.152962 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:50.253621 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:50.253494 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:50.354314 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:50.354280 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:50.455236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:50.455167 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:50.492468 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:50.492428 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:50.509517 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:50.509417 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:54:50.555596 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:50.555520 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:50.656241 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:50.656168 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:50.756772 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:50.756701 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:50.857146 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:50.857029 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:50.957478 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:50.957328 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:51.019562 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:51.019474 1563 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp 10.0.0.4:6443: i/o timeout" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:54:51.057714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:51.057676 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:51.158512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:51.158365 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:51.259003 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:51.258930 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:51.359959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:51.359829 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:51.460440 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:51.460217 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:51.560813 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:51.560719 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:51.661926 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:51.661855 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:51.673421 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:51.673335 1563 certificate_manager.go:471] kubernetes.io/kube-apiserver-client-kubelet: Failed while requesting a signed certificate from the control plane: cannot create certificate signing request: Post "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/certificates.k8s.io/v1/certificatesigningrequests": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:51.762578 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:51.762471 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:51.819663 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:51.819579 1563 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach" Nov 16 11:54:51.819663 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:51.819633 1563 kubelet_node_status.go:424] "Adding label from cloud provider" labelKey="beta.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:54:51.819663 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:51.819660 1563 kubelet_node_status.go:426] "Adding node label from cloud provider" labelKey="node.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:54:51.819663 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:51.819671 1563 kubelet_node_status.go:437] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/zone" labelValue="0" Nov 16 11:54:51.819824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:51.819680 1563 kubelet_node_status.go:439] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/zone" labelValue="0" Nov 16 11:54:51.819824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:51.819688 1563 kubelet_node_status.go:443] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/region" labelValue="westus" Nov 16 11:54:51.819824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:51.819697 1563 kubelet_node_status.go:445] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/region" labelValue="westus" Nov 16 11:54:51.820383 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:51.820358 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientMemory" Nov 16 11:54:51.820452 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:51.820393 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasNoDiskPressure" Nov 16 11:54:51.820452 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:51.820408 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientPID" Nov 16 11:54:51.820452 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:51.820450 1563 kubelet_node_status.go:72] "Attempting to register node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:54:51.863509 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:51.863439 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:51.964435 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:51.964356 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:52.064869 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:52.064714 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:52.165733 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:52.165666 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:52.266838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:52.266753 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:52.310669 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:52.310543 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/ropatil1611az-gs6zj-worker-westus-j4ngg": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:52.367828 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:52.367735 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:52.468397 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:52.468291 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:52.569094 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:52.569020 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:52.669523 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:52.669397 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:52.770032 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:52.769968 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:52.870296 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:52.870195 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:52.970800 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:52.970659 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:53.071703 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.071583 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:53.149929 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:54:53.149853 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.Service: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:53.149929 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:53.149934 1563 trace.go:205] Trace[1678641392]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:54:23.148) (total time: 30001ms): Nov 16 11:54:53.149929 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1678641392]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30001ms (11:54:53.149) Nov 16 11:54:53.149929 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1678641392]: [30.001265873s] [30.001265873s] END Nov 16 11:54:53.150188 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.149966 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:53.159243 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:54:53.159176 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.Node: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dropatil1611az-gs6zj-worker-westus-j4ngg&limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:53.159400 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:53.159266 1563 trace.go:205] Trace[349156515]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:54:23.158) (total time: 30000ms): Nov 16 11:54:53.159400 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[349156515]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dropatil1611az-gs6zj-worker-westus-j4ngg&limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30000ms (11:54:53.159) Nov 16 11:54:53.159400 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[349156515]: [30.00098097s] [30.00098097s] END Nov 16 11:54:53.159400 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.159298 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dropatil1611az-gs6zj-worker-westus-j4ngg&limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:53.172362 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.172284 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:53.273425 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.273305 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:53.374215 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.374143 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:53.474627 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.474538 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:53.574838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.574734 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:53.661121 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:54:53.661043 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.CSIDriver: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:53.661121 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:53.661120 1563 trace.go:205] Trace[1528137284]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:54:23.659) (total time: 30001ms): Nov 16 11:54:53.661121 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1528137284]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30001ms (11:54:53.661) Nov 16 11:54:53.661121 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1528137284]: [30.001133192s] [30.001133192s] END Nov 16 11:54:53.661342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.661174 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:53.675385 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.675345 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:53.775833 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.775745 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:53.876936 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.876853 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:53.944066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:54:53.943983 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.RuntimeClass: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:53.944066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:53.944060 1563 trace.go:205] Trace[1763879094]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:54:23.942) (total time: 30001ms): Nov 16 11:54:53.944066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1763879094]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30001ms (11:54:53.943) Nov 16 11:54:53.944066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1763879094]: [30.001126375s] [30.001126375s] END Nov 16 11:54:53.944066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.944075 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:54:53.977254 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:53.977159 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:54.077727 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:54.077602 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:54.178321 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:54.178165 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:54.278699 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:54.278601 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:54.379613 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:54.379527 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:54.480698 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:54.480443 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:54.580940 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:54.580828 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:54.681499 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:54.681383 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:54.782260 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:54.781969 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:54.883036 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:54.882934 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:54.983965 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:54.983903 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:55.084587 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:55.084377 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:55.185500 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:55.185409 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:55.285725 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:55.285588 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:55.386270 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:55.386056 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:55.487271 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:55.487102 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:55.511089 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:55.511043 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:54:55.588482 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:55.588382 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:55.688765 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:55.688556 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:55.789234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:55.789109 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:55.890070 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:55.890004 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:55.991073 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:55.990869 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:56.091887 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:56.091833 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:56.192268 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:56.192210 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:56.292551 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:56.292336 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:56.392668 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:56.392576 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:56.493813 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:56.493747 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:56.594997 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:56.594888 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:56.695418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:56.695285 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:56.796020 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:56.795924 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:56.897048 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:56.896822 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:56.997284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:56.997189 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:57.098305 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:57.098176 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:57.199489 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:57.199270 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:57.299783 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:57.299687 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:57.400669 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:57.400567 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:57.501245 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:57.501041 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:57.601964 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:57.601845 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:57.702436 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:57.702371 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:57.802919 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:57.802715 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:57.903680 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:57.903559 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:58.004647 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:58.004567 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:58.105578 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:58.105506 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:58.206730 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:58.206627 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:58.307267 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:58.307122 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:58.408434 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:58.408209 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:58.509314 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:58.509210 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:58.609864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:58.609756 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:58.710463 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:58.710276 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:58.811033 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:58.810919 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:58.911940 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:58.911839 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:59.012874 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:59.012718 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:59.113984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:59.113890 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:59.214399 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:59.214278 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:59.315654 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:59.315467 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:59.415794 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:59.415732 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:59.517030 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:59.516907 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:59.617670 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:59.617581 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:59.718807 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:59.718706 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:59.819605 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:59.819490 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:59.920675 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:59.920487 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:54:59.922745 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:54:59.922710 1563 certificate_manager.go:270] kubernetes.io/kube-apiserver-client-kubelet: Rotating certificates Nov 16 11:54:59.923751 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:54:59.923639 1563 controller.go:144] failed to ensure lease exists, will retry in 7s, error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:55:00.021714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:00.021585 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:00.122368 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:00.122248 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:00.222641 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:00.222455 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:00.323394 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:00.323301 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:00.423966 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:00.423883 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:00.493801 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:00.493609 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:00.512258 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:00.512203 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:55:00.524664 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:00.524515 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:00.625928 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:00.625699 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:00.726580 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:00.726520 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:00.827019 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:00.826835 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:00.927755 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:00.927652 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:01.028820 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:01.028764 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:01.129386 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:01.129264 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:01.230442 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:01.230354 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:01.330996 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:01.330883 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:01.431857 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:01.431650 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:01.531835 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:01.531773 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:01.632619 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:01.632499 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:01.733627 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:01.733443 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:01.834739 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:01.834599 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:01.935756 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:01.935661 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:02.036810 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:02.036605 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:02.137280 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:02.137157 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:02.238416 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:02.238288 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:02.339499 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:02.339417 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:02.440731 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:02.440609 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:02.541293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:02.541197 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:02.641743 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:02.641549 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:02.742621 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:02.742557 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:02.843310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:02.843215 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:02.943890 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:02.943624 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:03.044681 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:03.044581 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:03.145645 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:03.145574 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:03.246704 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:03.246510 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:03.347102 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:03.346997 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:03.447714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:03.447620 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:03.548744 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:03.548569 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:03.649270 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:03.649206 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:03.750010 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:03.749936 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:03.850556 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:03.850466 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:03.951550 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:03.951482 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:04.051994 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:04.051897 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:04.153228 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:04.153023 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:04.254275 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:04.254114 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:04.354958 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:04.354851 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:04.456073 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:04.455926 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:04.556112 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:04.556013 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:04.657161 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:04.657081 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:04.757541 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:04.757375 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:04.858659 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:04.858521 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:04.959676 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:04.959579 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:05.060849 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:05.060653 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:05.161028 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:05.160936 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:05.262110 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:05.262051 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:05.362683 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:05.362577 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:05.463502 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:05.463392 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:05.514365 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:05.514324 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:55:05.563937 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:05.563823 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:05.665242 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:05.665026 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:05.765786 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:05.765685 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:05.866144 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:05.866016 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:05.967115 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:05.966918 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:06.067541 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:06.067464 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:06.168150 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:06.168031 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:06.268990 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:06.268831 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:06.369940 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:06.369878 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:06.470775 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:06.470676 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:06.571413 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:06.571222 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:06.671589 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:06.671487 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:06.772118 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:06.772050 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:06.873332 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:06.873224 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:06.974506 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:06.974388 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:07.075492 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:07.075382 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:07.176575 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:07.176329 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:07.276678 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:07.276569 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:07.377225 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:07.377117 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:07.478349 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:07.478185 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:07.578760 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:07.578663 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:07.678958 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:07.678890 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:07.779851 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:07.779713 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:07.880829 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:07.880724 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:07.981082 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:07.980948 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:08.081934 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:08.081732 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:08.182863 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:08.182766 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:08.283045 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:08.282932 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:08.383288 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:08.383193 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:08.483828 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:08.483762 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:08.584115 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:08.584018 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:08.684661 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:08.684474 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:08.784840 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:08.784777 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:08.885903 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:08.885781 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:08.987014 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:08.986876 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:08.994397 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:08.994230 1563 event.go:276] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef0b7d27e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 308015742, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 308015742, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'Post "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp 10.0.0.4:6443: i/o timeout'(may retry after sleeping) Nov 16 11:55:09.087411 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:09.087316 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:09.188395 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:09.188324 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:09.288901 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:09.288736 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:09.389401 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:09.389311 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:09.490358 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:09.490288 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:09.591354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:09.591308 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:09.692407 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:09.692341 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:09.793161 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:09.793040 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:09.893354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:09.893208 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:09.994149 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:09.994060 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:10.094630 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:10.094502 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:10.195522 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:10.195346 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:10.296091 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:10.295996 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:10.396709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:10.396653 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:10.494768 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:10.494597 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:10.496950 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:10.496882 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:10.515424 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:10.515385 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:55:10.598082 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:10.597973 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:10.699034 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:10.698939 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:10.799634 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:10.799500 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:10.900553 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:10.900493 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:11.001696 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:11.001597 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:11.102121 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:11.102071 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:11.202938 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:11.202820 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:11.303046 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:11.302945 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:11.403781 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:11.403643 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:11.504662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:11.504567 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:11.605054 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:11.605007 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:11.706022 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:11.705861 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:11.806587 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:11.806497 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:11.907466 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:11.907383 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:12.008735 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:12.008531 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:12.109674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:12.109585 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:12.210521 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:12.210442 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:12.311007 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:12.310838 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:12.411362 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:12.411312 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:12.512473 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:12.512390 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:12.613281 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:12.613205 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:12.714332 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:12.714241 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:12.814766 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:12.814684 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:12.915689 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:12.915542 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:13.016815 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:13.016734 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:13.117663 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:13.117556 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:13.218578 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:13.218409 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:13.319044 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:13.318946 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:13.420306 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:13.420151 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:13.521418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:13.521258 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:13.621878 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:13.621796 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:13.722979 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:13.722898 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:13.823577 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:13.823405 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:13.924407 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:13.924368 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:14.025485 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:14.025388 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:14.126566 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:14.126472 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:14.227062 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:14.226967 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:14.327583 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:14.327447 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:14.428477 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:14.428257 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:14.528559 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:14.528466 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:14.628714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:14.628617 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:14.729827 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:14.729608 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:14.830149 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:14.830079 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:14.931335 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:14.931277 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:15.032322 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:15.031999 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:15.132633 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:15.132521 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:15.232955 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:15.232910 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:15.333674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:15.333493 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:15.434759 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:15.434673 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:15.517228 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:15.517188 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:55:15.535575 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:15.535494 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:15.636065 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:15.635966 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:15.737114 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:15.737054 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:15.837554 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:15.837422 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:15.938697 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:15.938495 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:16.039527 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:16.039428 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:16.139893 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:16.139805 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:16.241153 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:16.240986 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:16.341627 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:16.341539 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:16.442509 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:16.442409 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:16.543720 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:16.543498 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:16.643745 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:16.643650 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:16.744303 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:16.744209 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:16.844491 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:16.844403 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:16.924611 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:16.924529 1563 controller.go:144] failed to ensure lease exists, will retry in 7s, error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:55:16.944929 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:16.944836 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:17.045989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:17.045886 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:17.147237 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:17.147066 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:17.247993 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:17.247926 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:17.348606 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:17.348500 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:17.449294 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:17.449096 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:17.549378 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:17.549233 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:17.649879 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:17.649813 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:17.751034 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:17.750854 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:17.851320 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:17.851265 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:17.952560 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:17.952441 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:18.053607 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:18.053417 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:18.154486 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:18.154332 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:18.254659 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:18.254553 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:18.354963 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:18.354901 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:18.456065 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:18.455969 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:18.557068 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:18.556968 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:18.657829 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:18.657649 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:18.758227 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:18.758101 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:18.859154 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:18.859052 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:18.959752 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:18.959558 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:19.060318 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:19.060254 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:19.161528 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:19.161429 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:19.262170 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:19.261996 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:19.362774 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:19.362680 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:19.463396 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:19.463318 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:19.563942 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:19.563722 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:19.664100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:19.663967 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:19.765240 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:19.765145 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:19.865768 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:19.865698 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:19.966103 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:19.966023 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:20.067182 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:20.067075 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:20.168327 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:20.168171 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:20.268406 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:20.268305 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:20.368965 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:20.368848 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:20.469236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:20.469088 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:20.495255 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:20.495215 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:20.518047 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:20.518019 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:55:20.569743 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:20.569652 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:20.670418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:20.670282 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:20.771495 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:20.771315 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:20.872676 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:20.872571 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:20.973706 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:20.973653 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:21.074840 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:21.074646 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:21.175363 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:21.175271 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:21.276445 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:21.276331 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:21.376581 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:21.376490 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:21.476709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:21.476598 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:21.577429 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:21.577306 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:21.678186 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:21.677962 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:21.778192 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:21.778095 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:21.821887 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:21.821814 1563 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp 10.0.0.4:6443: i/o timeout" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:55:21.879069 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:21.878991 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:21.979880 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:21.979736 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:22.080640 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:22.080546 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:22.181005 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:22.180910 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:22.282253 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:22.282030 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:22.383156 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:22.383065 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:22.483384 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:22.483312 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:22.584549 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:22.584392 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:22.685302 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:22.685238 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:22.786346 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:22.786261 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:22.886874 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:22.886676 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:22.987879 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:22.987785 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:23.088844 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:23.088744 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:23.190031 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:23.189789 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:23.289984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:23.289919 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:23.310418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:23.310361 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/ropatil1611az-gs6zj-worker-westus-j4ngg": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:55:23.390166 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:23.390061 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:23.422334 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:23.422235 1563 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach" Nov 16 11:55:23.422334 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:23.422297 1563 kubelet_node_status.go:424] "Adding label from cloud provider" labelKey="beta.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:55:23.422334 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:23.422310 1563 kubelet_node_status.go:426] "Adding node label from cloud provider" labelKey="node.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:55:23.422334 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:23.422324 1563 kubelet_node_status.go:437] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/zone" labelValue="0" Nov 16 11:55:23.422334 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:23.422336 1563 kubelet_node_status.go:439] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/zone" labelValue="0" Nov 16 11:55:23.422334 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:23.422347 1563 kubelet_node_status.go:443] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/region" labelValue="westus" Nov 16 11:55:23.422334 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:23.422357 1563 kubelet_node_status.go:445] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/region" labelValue="westus" Nov 16 11:55:23.423239 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:23.423212 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientMemory" Nov 16 11:55:23.423322 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:23.423262 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasNoDiskPressure" Nov 16 11:55:23.423322 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:23.423289 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientPID" Nov 16 11:55:23.423322 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:23.423317 1563 kubelet_node_status.go:72] "Attempting to register node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:55:23.490893 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:23.490702 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:23.591278 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:23.591147 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:23.692033 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:23.691968 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:23.793382 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:23.793251 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:23.894439 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:23.894341 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:23.995339 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:23.995264 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:24.096151 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:24.096057 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:24.196969 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:24.196862 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:24.298158 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:24.298024 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:24.399057 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:24.398846 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:24.499852 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:24.499789 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:24.600287 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:24.600221 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:24.701380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:24.701172 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:24.802042 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:24.801994 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:24.902903 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:24.902830 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:25.004123 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:25.003941 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:25.105098 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:25.105003 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:25.206160 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:25.206048 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:25.307089 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:25.306926 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:25.407700 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:25.407643 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:25.508824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:25.508687 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:25.518883 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:25.518852 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:55:25.609421 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:25.609335 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:25.709667 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:25.709613 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:25.810621 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:25.810524 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:25.911882 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:25.911637 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:26.012559 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:26.012493 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:26.113466 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:26.113366 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:26.214681 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:26.214569 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:26.315324 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:26.315271 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:26.416259 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:26.416195 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:26.425680 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:55:26.425590 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.Service: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:55:26.425680 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:26.425671 1563 trace.go:205] Trace[1851907851]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:54:56.424) (total time: 30001ms): Nov 16 11:55:26.425680 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1851907851]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30001ms (11:55:26.425) Nov 16 11:55:26.425680 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1851907851]: [30.001236413s] [30.001236413s] END Nov 16 11:55:26.425680 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:26.425690 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:55:26.516783 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:26.516629 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:26.617184 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:26.617070 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:26.718270 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:26.718160 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:26.819046 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:26.818887 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:26.920084 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:26.919992 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:27.021141 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:27.021024 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:27.122348 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:27.122258 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:27.222750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:27.222645 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:27.323581 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:27.323433 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:27.424109 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:27.423891 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:27.524903 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:27.524773 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:27.610538 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:55:27.610451 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.CSIDriver: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:55:27.610538 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:27.610560 1563 trace.go:205] Trace[1889149993]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:54:57.609) (total time: 30001ms): Nov 16 11:55:27.610538 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1889149993]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30001ms (11:55:27.610) Nov 16 11:55:27.610538 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1889149993]: [30.001363051s] [30.001363051s] END Nov 16 11:55:27.610861 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:27.610619 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:55:27.625779 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:27.625740 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:27.727009 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:27.726815 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:27.828058 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:27.827953 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:27.929248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:27.929095 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:28.030334 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:28.030174 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:28.130393 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:28.130315 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:28.231342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:28.231251 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:28.331622 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:28.331430 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:28.432193 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:28.432075 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:28.533038 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:28.532957 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:28.633546 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:28.633494 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:28.734574 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:28.734518 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:28.835510 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:28.835394 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:28.936315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:28.936208 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:29.036590 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:29.036496 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:29.056425 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:55:29.056366 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.RuntimeClass: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:55:29.056553 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:29.056444 1563 trace.go:205] Trace[1446774685]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:54:59.055) (total time: 30001ms): Nov 16 11:55:29.056553 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1446774685]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30001ms (11:55:29.056) Nov 16 11:55:29.056553 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1446774685]: [30.001408196s] [30.001408196s] END Nov 16 11:55:29.056553 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:29.056475 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:55:29.136713 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:29.136624 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:29.237809 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:29.237636 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:29.338540 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:29.338437 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:29.438956 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:29.438891 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:29.447477 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:55:29.447418 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.Node: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dropatil1611az-gs6zj-worker-westus-j4ngg&limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:55:29.447587 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:29.447540 1563 trace.go:205] Trace[1504267679]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (16-Nov-2022 11:54:59.446) (total time: 30001ms): Nov 16 11:55:29.447587 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1504267679]: ---"Objects listed" error:Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dropatil1611az-gs6zj-worker-westus-j4ngg&limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout 30001ms (11:55:29.447) Nov 16 11:55:29.447587 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Trace[1504267679]: [30.001446281s] [30.001446281s] END Nov 16 11:55:29.447587 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:29.447567 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dropatil1611az-gs6zj-worker-westus-j4ngg&limit=500&resourceVersion=0": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:55:29.539653 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:29.539500 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:29.640665 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:29.640601 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:29.741011 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:29.740927 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:29.841234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:29.841134 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:29.924239 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:29.924163 1563 certificate_manager.go:471] kubernetes.io/kube-apiserver-client-kubelet: Failed while requesting a signed certificate from the control plane: cannot create certificate signing request: Post "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/certificates.k8s.io/v1/certificatesigningrequests": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:55:29.942361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:29.942291 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:30.043242 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:30.043188 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:30.144527 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:30.144341 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:30.245494 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:30.245375 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:30.346604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:30.346502 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:30.447713 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:30.447522 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:30.496292 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:30.496179 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:30.520382 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:30.520355 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:55:30.548759 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:30.548681 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:30.649915 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:30.649845 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:30.750977 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:30.750827 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:30.851629 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:30.851529 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:30.952148 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:30.952044 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:31.053265 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:31.053012 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:31.153733 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:31.153671 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:31.254750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:31.254615 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:31.355145 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:31.355047 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:31.455366 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:31.455270 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:31.556486 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:31.556399 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:31.657117 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:31.656953 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:31.757199 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:31.757165 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:31.857945 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:31.857871 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:31.959030 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:31.958896 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:32.060069 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:32.059973 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:32.161062 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:32.160969 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:32.262170 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:32.261987 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:32.362443 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:32.362379 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:32.462808 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:32.462681 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:32.563012 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:32.562844 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:32.663705 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:32.663551 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:32.763737 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:32.763653 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:32.864050 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:32.863977 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:32.965041 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:32.964969 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:33.066123 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:33.066028 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:33.167314 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:33.167155 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:33.268274 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:33.268176 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:33.368753 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:33.368613 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:33.469322 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:33.469175 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:33.569591 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:33.569470 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:33.669625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:33.669560 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:33.770723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:33.770519 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:33.871698 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:33.871514 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:33.926522 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:33.926466 1563 controller.go:144] failed to ensure lease exists, will retry in 7s, error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:55:33.971743 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:33.971650 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:34.072723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.072589 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:34.172812 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.172735 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:34.273636 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.273599 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:34.374333 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.374290 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:34.441389 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:34.441145 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:34.441389 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.441163 1563 kubelet_node_status.go:94] "Unable to register node with API server" err="nodes is forbidden: User \"system:anonymous\" cannot create resource \"nodes\" in API group \"\" at the cluster scope" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:55:34.441389 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.441146 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef0b7d27e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 308015742, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 308015742, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:34.443418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.443335 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412696013, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412696013, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:34.445005 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.444841 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412722014, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412722014, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:34.446257 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.446189 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5b6ee", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412735214, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412735214, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:34.448636 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.448552 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412696013, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 431502752, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:34.450234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.450089 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412722014, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 432171157, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:34.451599 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.451523 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5b6ee", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412735214, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 432199957, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5b6ee" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:34.452881 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.452739 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7efb7f9b4f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated Node Allocatable limit across pods", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 488880975, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 488880975, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:34.454399 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.454310 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412696013, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 50, 616020414, time.Local), Count:3, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:34.455775 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.455702 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412722014, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 50, 616061714, time.Local), Count:3, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:34.457090 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.456978 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5b6ee", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412735214, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 53, 50, 616092215, time.Local), Count:3, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5b6ee" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:34.475260 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.475141 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:34.575774 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.575605 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:34.644324 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.644081 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412696013, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 54, 21, 18847610, time.Local), Count:4, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:34.676336 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.676258 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:34.776456 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.776342 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:34.843576 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.843343 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412722014, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 54, 21, 18871411, time.Local), Count:4, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:34.877427 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.877307 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:34.978442 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:34.978309 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:35.044602 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.044412 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5b6ee", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412735214, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 54, 21, 18888211, time.Local), Count:4, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5b6ee" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:35.079359 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.079286 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:35.179561 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.179496 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:35.231254 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:55:35.231091 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:35.231254 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.231161 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:35.244350 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.244169 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412696013, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 54, 51, 820382809, time.Local), Count:5, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:35.280140 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.280062 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:35.311904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:35.311822 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:35.381155 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.381074 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:35.444186 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.443977 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412722014, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 54, 51, 820400009, time.Local), Count:5, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:35.482233 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.482095 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:35.521613 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.521546 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:55:35.583121 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.583049 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:35.643585 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.643430 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5b6ee", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412735214, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 54, 51, 820413009, time.Local), Count:5, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5b6ee" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:35.683763 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.683681 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:35.784475 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.784276 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:35.843159 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.843001 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412696013, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 55, 23, 423236840, time.Local), Count:6, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:35.885285 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.885210 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:35.985832 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:35.985731 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:36.045054 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:36.044834 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412722014, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 55, 23, 423269740, time.Local), Count:6, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:36.086036 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:36.085995 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:36.187082 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:36.187038 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:36.244295 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:36.244045 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5b6ee", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412735214, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 55, 23, 423294340, time.Local), Count:6, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5b6ee" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:36.287414 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:36.287302 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:36.311908 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:36.311760 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:36.388445 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:36.388338 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:36.489322 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:36.489240 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:36.589883 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:36.589826 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:36.690776 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:36.690698 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:36.791186 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:36.791099 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:36.892250 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:36.892095 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:36.993152 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:36.993064 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:37.094066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.093999 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:37.194863 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.194733 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:37.294938 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.294811 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:37.313039 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:37.312965 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:37.368197 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:55:37.368097 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope Nov 16 11:55:37.368197 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.368151 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope Nov 16 11:55:37.395512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.395426 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:37.496021 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.495867 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:37.596394 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.596299 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:37.641291 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:37.641260 1563 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach" Nov 16 11:55:37.641291 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:37.641301 1563 kubelet_node_status.go:424] "Adding label from cloud provider" labelKey="beta.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:55:37.641418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:37.641313 1563 kubelet_node_status.go:426] "Adding node label from cloud provider" labelKey="node.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:55:37.641418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:37.641326 1563 kubelet_node_status.go:437] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/zone" labelValue="0" Nov 16 11:55:37.641418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:37.641336 1563 kubelet_node_status.go:439] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/zone" labelValue="0" Nov 16 11:55:37.641418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:37.641346 1563 kubelet_node_status.go:443] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/region" labelValue="westus" Nov 16 11:55:37.641418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:37.641356 1563 kubelet_node_status.go:445] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/region" labelValue="westus" Nov 16 11:55:37.642104 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:37.642045 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientMemory" Nov 16 11:55:37.642104 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:37.642083 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasNoDiskPressure" Nov 16 11:55:37.642104 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:37.642101 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientPID" Nov 16 11:55:37.642284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:37.642148 1563 kubelet_node_status.go:72] "Attempting to register node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:55:37.644590 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.644516 1563 kubelet_node_status.go:94] "Unable to register node with API server" err="nodes is forbidden: User \"system:anonymous\" cannot create resource \"nodes\" in API group \"\" at the cluster scope" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:55:37.644663 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.644524 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412696013, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 55, 37, 642067329, time.Local), Count:7, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:37.646343 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.646260 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412722014, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 55, 37, 642091929, time.Local), Count:7, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:37.647957 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.647811 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5b6ee", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412735214, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 55, 37, 642107229, time.Local), Count:7, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5b6ee" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:37.697108 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.697061 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:37.797670 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.797483 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:37.898476 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.898385 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:37.999465 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:37.999387 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:38.100480 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:38.100396 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:38.200685 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:38.200584 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:38.300983 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:38.300924 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:38.313062 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:38.312966 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:38.401458 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:38.401346 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:38.501562 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:38.501457 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:38.602286 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:38.602106 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:38.703325 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:38.703181 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:38.803430 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:38.803353 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:38.903532 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:38.903429 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:39.004635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:39.004458 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:39.104984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:39.104909 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:39.206070 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:39.205938 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:39.225257 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:55:39.225070 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.Node: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope Nov 16 11:55:39.225382 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:39.225198 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope Nov 16 11:55:39.306827 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:39.306667 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:39.311629 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:39.311526 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:39.407485 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:39.407392 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:39.508647 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:39.508546 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:39.609287 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:39.609209 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:39.710044 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:39.709971 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:39.810540 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:39.810472 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:39.910748 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:39.910611 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:40.011496 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.011353 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:40.112449 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.112323 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:40.152609 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:55:40.152513 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope Nov 16 11:55:40.152609 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.152565 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope Nov 16 11:55:40.212996 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.212857 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:40.312696 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:40.312534 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:40.313699 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.313637 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:40.414632 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.414556 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:40.497407 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.497221 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:40.515060 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.515022 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:40.522731 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.522701 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:55:40.615282 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.615175 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:40.716439 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.716298 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:40.816579 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.816410 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:40.917719 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.917629 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:40.929516 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:40.929443 1563 controller.go:144] failed to ensure lease exists, will retry in 7s, error: leases.coordination.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease" Nov 16 11:55:41.018524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:41.018375 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:41.119452 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:41.119405 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:41.220442 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:41.220385 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:41.312857 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:41.312774 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:41.321050 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:41.320977 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:41.421864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:41.421667 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:41.522791 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:41.522691 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:41.623564 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:41.623468 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:41.724457 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:41.724310 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:41.824982 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:41.824889 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:41.925436 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:41.925348 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:42.026609 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:42.026419 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:42.127084 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:42.126981 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:42.228216 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:42.228107 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:42.312123 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:42.311926 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:42.329271 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:42.329200 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:42.429742 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:42.429632 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:42.529963 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:42.529866 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:42.630784 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:42.630707 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:42.731716 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:42.731625 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:42.832715 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:42.832647 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:42.933913 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:42.933733 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:43.034940 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:43.034800 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:43.136022 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:43.135942 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:43.237120 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:43.236901 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:43.312077 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:43.312024 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:43.337598 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:43.337531 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:43.438315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:43.438216 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:43.539503 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:43.539321 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:43.640072 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:43.639981 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:43.740396 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:43.740273 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:43.841514 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:43.841392 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:43.942371 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:43.942298 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:44.043537 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:44.043459 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:44.044762 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:44.044680 1563 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach" Nov 16 11:55:44.044762 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:44.044735 1563 kubelet_node_status.go:424] "Adding label from cloud provider" labelKey="beta.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:55:44.044762 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:44.044763 1563 kubelet_node_status.go:426] "Adding node label from cloud provider" labelKey="node.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:55:44.044949 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:44.044797 1563 kubelet_node_status.go:437] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/zone" labelValue="0" Nov 16 11:55:44.044949 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:44.044822 1563 kubelet_node_status.go:439] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/zone" labelValue="0" Nov 16 11:55:44.044949 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:44.044848 1563 kubelet_node_status.go:443] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/region" labelValue="westus" Nov 16 11:55:44.044949 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:44.044857 1563 kubelet_node_status.go:445] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/region" labelValue="westus" Nov 16 11:55:44.045743 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:44.045715 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientMemory" Nov 16 11:55:44.045805 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:44.045756 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasNoDiskPressure" Nov 16 11:55:44.045805 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:44.045771 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientPID" Nov 16 11:55:44.045805 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:44.045801 1563 kubelet_node_status.go:72] "Attempting to register node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:55:44.048400 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:44.048350 1563 kubelet_node_status.go:94] "Unable to register node with API server" err="nodes is forbidden: User \"system:anonymous\" cannot create resource \"nodes\" in API group \"\" at the cluster scope" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:55:44.048400 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:44.048283 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412696013, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 55, 44, 45739418, time.Local), Count:8, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f51dcd" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:44.050198 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:44.050009 1563 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ropatil1611az-gs6zj-worker-westus-j4ngg", UID:"ropatil1611az-gs6zj-worker-westus-j4ngg", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node ropatil1611az-gs6zj-worker-westus-j4ngg status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"ropatil1611az-gs6zj-worker-westus-j4ngg"}, FirstTimestamp:time.Date(2022, time.November, 16, 11, 53, 20, 412722014, time.Local), LastTimestamp:time.Date(2022, time.November, 16, 11, 55, 44, 45762318, time.Local), Count:8, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "ropatil1611az-gs6zj-worker-westus-j4ngg.17280e7ef6f5835e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!) Nov 16 11:55:44.144313 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:44.144107 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:44.245271 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:44.245196 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:44.312500 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:44.312424 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:44.345734 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:44.345666 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:44.446668 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:44.446525 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:44.546797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:44.546709 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:44.647898 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:44.647770 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:44.748933 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:44.748782 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:44.849797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:44.849688 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:44.950845 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:44.950722 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:45.051543 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:45.051370 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:45.152055 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:45.151918 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:45.252585 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:45.252476 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:45.312425 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:45.312036 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:45.353496 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:45.353386 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:45.454499 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:45.454417 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:45.524188 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:45.524106 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:55:45.555487 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:45.555421 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:45.656464 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:45.656387 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:45.757526 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:45.757460 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:45.857735 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:45.857631 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:45.958150 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:45.958022 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:46.059152 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:46.059002 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:46.159362 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:46.159259 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:46.260472 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:46.260273 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:46.312627 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:46.312519 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:46.360931 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:46.360828 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:46.461473 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:46.461346 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:46.562019 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:46.561792 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:46.662582 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:46.662499 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:46.763094 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:46.762997 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:46.780611 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:46.780533 1563 certificate_manager.go:270] kubernetes.io/kube-apiserver-client-kubelet: Rotating certificates Nov 16 11:55:46.837741 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:46.837707 1563 csr.go:262] certificate signing request csr-jw5bd is approved, waiting to be issued Nov 16 11:55:46.846867 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:46.846830 1563 csr.go:258] certificate signing request csr-jw5bd is issued Nov 16 11:55:46.864261 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:46.864109 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:46.965252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:46.965100 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:47.066263 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:47.066111 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:47.167363 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:47.167187 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:47.267903 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:47.267814 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:47.312599 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:47.312461 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:47.368663 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:47.368583 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:47.468988 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:47.468765 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:47.569631 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:47.569557 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:47.670113 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:47.670035 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:47.771443 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:47.771256 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:47.848264 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:47.848186 1563 certificate_manager.go:270] kubernetes.io/kube-apiserver-client-kubelet: Certificate expiration is 2022-11-17 11:23:22 +0000 UTC, rotation deadline is 2022-11-17 08:37:47.925452937 +0000 UTC Nov 16 11:55:47.848264 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:47.848220 1563 certificate_manager.go:270] kubernetes.io/kube-apiserver-client-kubelet: Waiting 20h42m0.077236007s for next certificate rotation Nov 16 11:55:47.871583 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:47.871516 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:47.932624 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:47.932535 1563 controller.go:144] failed to ensure lease exists, will retry in 7s, error: leases.coordination.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease" Nov 16 11:55:47.971864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:47.971706 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:48.072805 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:48.072603 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:48.173649 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:48.173590 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:48.274583 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:48.274470 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:48.314700 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:48.314615 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:48.374941 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:48.374835 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:48.476236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:48.476082 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:48.576756 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:48.576649 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:48.677477 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:48.677302 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:48.777810 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:48.777727 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:48.877940 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:48.877862 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:48.979001 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:48.978831 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:49.080157 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:49.079945 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:49.181188 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:49.181074 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:49.281444 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:49.281268 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:49.317040 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:49.316997 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:49.382286 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:49.382209 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:49.482714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:49.482619 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:49.583407 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:49.583203 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:49.684460 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:49.684333 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:49.784664 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:49.784543 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:49.884976 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:49.884891 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:49.985623 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:49.985532 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:50.085867 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:50.085769 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:50.186106 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:50.185912 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:50.286550 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:50.286433 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:50.314247 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:50.314123 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:50.386683 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:50.386539 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:50.487565 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:50.487359 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:50.497662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:50.497614 1563 eviction_manager.go:254] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:50.525712 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:50.525678 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:55:50.588656 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:50.588558 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:50.689821 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:50.689635 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:50.790021 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:50.789870 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:50.891068 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:50.890987 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:50.992287 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:50.992199 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:51.049342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:51.049168 1563 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach" Nov 16 11:55:51.049342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:51.049224 1563 kubelet_node_status.go:424] "Adding label from cloud provider" labelKey="beta.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:55:51.049342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:51.049235 1563 kubelet_node_status.go:426] "Adding node label from cloud provider" labelKey="node.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:55:51.049342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:51.049248 1563 kubelet_node_status.go:437] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/zone" labelValue="0" Nov 16 11:55:51.049342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:51.049260 1563 kubelet_node_status.go:439] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/zone" labelValue="0" Nov 16 11:55:51.049342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:51.049270 1563 kubelet_node_status.go:443] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/region" labelValue="westus" Nov 16 11:55:51.049342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:51.049313 1563 kubelet_node_status.go:445] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/region" labelValue="westus" Nov 16 11:55:51.050308 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:51.050275 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientMemory" Nov 16 11:55:51.050383 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:51.050318 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasNoDiskPressure" Nov 16 11:55:51.050383 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:51.050333 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientPID" Nov 16 11:55:51.050383 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:51.050362 1563 kubelet_node_status.go:72] "Attempting to register node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:55:51.053434 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:51.053381 1563 kubelet_node_status.go:94] "Unable to register node with API server" err="nodes is forbidden: User \"system:anonymous\" cannot create resource \"nodes\" in API group \"\" at the cluster scope" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:55:51.092542 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:51.092477 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:51.193433 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:51.193331 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:51.294286 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:51.294180 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:51.312796 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:51.312621 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:51.378855 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:55:51.378760 1563 reflector.go:324] k8s.io/client-go/informers/factory.go:134: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:51.378855 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:51.378811 1563 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:51.395211 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:51.395110 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:51.496065 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:51.496009 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:51.596667 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:51.596591 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:51.696884 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:51.696755 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:51.798010 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:51.797923 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:51.898845 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:51.898633 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:51.999653 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:51.999571 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:52.100687 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:52.100581 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:52.201229 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:52.201005 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:52.302203 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:52.302074 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:52.314293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:52.314113 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:52.402725 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:52.402677 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:52.503746 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:52.503576 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:52.604192 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:52.604100 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:52.705231 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:52.705121 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:52.805498 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:52.805304 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:52.906169 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:52.906047 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:53.006844 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:53.006752 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:53.107747 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:53.107645 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:53.208661 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:53.208575 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:53.309972 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:53.309879 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:53.312166 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:53.312082 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:53.410324 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:53.410119 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:53.511281 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:53.511174 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:53.612119 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:53.612010 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:53.713242 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:53.713074 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:53.814087 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:53.814009 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:53.914451 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:53.914386 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:54.015592 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:54.015434 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:54.116094 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:54.115974 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:54.216990 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:54.216897 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:54.312435 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:54.312290 1563 csi_plugin.go:1063] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 16 11:55:54.317612 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:54.317516 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:54.418340 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:54.418228 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:54.519410 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:54.519276 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:54.620080 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:54.619944 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:54.721214 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:54.721107 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:54.822448 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:54.822325 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:54.922648 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:54.922479 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:54.936165 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:54.936034 1563 controller.go:144] failed to ensure lease exists, will retry in 7s, error: leases.coordination.k8s.io "ropatil1611az-gs6zj-worker-westus-j4ngg" is forbidden: User "system:anonymous" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease" Nov 16 11:55:55.023092 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:55.023000 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:55.124331 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:55.124237 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:55.224604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:55.224405 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:55.283751 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:55.283650 1563 transport.go:135] "Certificate rotation detected, shutting down client connections to start using new credentials" Nov 16 11:55:55.324629 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:55.324510 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:55.332997 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:55.332916 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:55.353795 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:55.353660 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:55.415315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:55.415230 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:55.425517 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:55.425447 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:55.526768 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:55.526559 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:55.527290 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:55.527261 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:55:55.627142 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:55.627081 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:55.679528 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:55.679459 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:55.679528 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:55.679489 1563 csi_plugin.go:297] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:55.705157 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:55.705090 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:55.725799 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:55.725703 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:55.727993 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:55.727960 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:55.787925 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:55.787751 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:55.828487 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:55.828390 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:55.929584 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:55.929490 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:56.030171 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:56.030083 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:56.064283 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:56.064079 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:56.064283 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:56.064152 1563 csi_plugin.go:297] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:56.130660 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:56.130562 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:56.164119 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:56.164049 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:56.184019 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:56.183887 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:56.230709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:56.230667 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:56.248281 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:56.248169 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:56.331493 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:56.331301 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:56.396088 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:56.395977 1563 apiserver.go:52] "Watching apiserver" Nov 16 11:55:56.398590 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:56.398526 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[] Nov 16 11:55:56.432189 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:56.432084 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:56.440534 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:56.440440 1563 reconciler.go:157] "Reconciler: start to sync state" Nov 16 11:55:56.532651 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:56.532576 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:56.633528 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:56.633453 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:56.734336 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:56.734238 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:56.740914 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:56.740819 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:56.740914 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:56.740894 1563 csi_plugin.go:297] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:56.834905 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:56.834749 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:56.935339 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:56.935170 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:57.036275 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:57.036205 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:57.137468 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:57.137366 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:57.238682 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:57.238494 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:57.305632 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:57.305497 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:57.325800 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:57.325724 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:57.339042 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:57.338968 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:57.387937 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:57.387854 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:57.439808 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:57.439717 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:57.540501 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:57.540374 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:57.641035 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:57.640994 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:57.648993 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:57.648962 1563 nodeinfomanager.go:401] Failed to publish CSINode: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:57.648993 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:57.648994 1563 csi_plugin.go:297] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ropatil1611az-gs6zj-worker-westus-j4ngg" not found Nov 16 11:55:57.741824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:57.741695 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:57.842748 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:57.842683 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:57.943862 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:57.943794 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:58.044789 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:55:58.044730 1563 kubelet.go:2489] "Error getting node" err="node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" not found" Nov 16 11:55:58.054430 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.054339 1563 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach" Nov 16 11:55:58.054430 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.054402 1563 kubelet_node_status.go:424] "Adding label from cloud provider" labelKey="beta.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:55:58.054430 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.054430 1563 kubelet_node_status.go:426] "Adding node label from cloud provider" labelKey="node.kubernetes.io/instance-type" labelValue="Standard_F8s" Nov 16 11:55:58.054657 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.054459 1563 kubelet_node_status.go:437] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/zone" labelValue="0" Nov 16 11:55:58.054657 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.054469 1563 kubelet_node_status.go:439] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/zone" labelValue="0" Nov 16 11:55:58.054657 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.054496 1563 kubelet_node_status.go:443] "Adding node label from cloud provider" labelKey="failure-domain.beta.kubernetes.io/region" labelValue="westus" Nov 16 11:55:58.054657 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.054530 1563 kubelet_node_status.go:445] "Adding node label from cloud provider" labelKey="topology.kubernetes.io/region" labelValue="westus" Nov 16 11:55:58.055249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.055222 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientMemory" Nov 16 11:55:58.055331 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.055268 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasNoDiskPressure" Nov 16 11:55:58.055331 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.055285 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeHasSufficientPID" Nov 16 11:55:58.055331 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.055325 1563 kubelet_node_status.go:72] "Attempting to register node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:55:58.068683 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.068610 1563 kubelet_node_status.go:75] "Successfully registered node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 11:55:58.417105 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.417007 1563 certificate_manager.go:270] kubernetes.io/kubelet-serving: Rotating certificates Nov 16 11:55:58.469495 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.469457 1563 log.go:184] http: TLS handshake error from 10.0.0.8:53486: no serving certificate available for the kubelet Nov 16 11:55:58.477140 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.477039 1563 csr.go:262] certificate signing request csr-h44d5 is approved, waiting to be issued Nov 16 11:55:58.488120 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:58.488025 1563 csr.go:258] certificate signing request csr-h44d5 is issued Nov 16 11:55:59.489619 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:59.489558 1563 certificate_manager.go:270] kubernetes.io/kubelet-serving: Certificate expiration is 2022-11-17 11:23:22 +0000 UTC, rotation deadline is 2022-11-17 05:06:47.723023144 +0000 UTC Nov 16 11:55:59.489619 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:55:59.489600 1563 certificate_manager.go:270] kubernetes.io/kubelet-serving: Waiting 17h10m48.2334265s for next certificate rotation Nov 16 11:56:00.490097 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:56:00.490043 1563 certificate_manager.go:270] kubernetes.io/kubelet-serving: Certificate expiration is 2022-11-17 11:23:22 +0000 UTC, rotation deadline is 2022-11-17 06:10:59.692140935 +0000 UTC Nov 16 11:56:00.490097 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:56:00.490090 1563 certificate_manager.go:270] kubernetes.io/kubelet-serving: Waiting 18h14m59.202053268s for next certificate rotation Nov 16 11:56:00.528498 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:56:00.528467 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:56:05.529724 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:56:05.529669 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:56:10.530336 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:56:10.530221 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:56:15.532062 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:56:15.531972 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:56:20.533518 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:56:20.533465 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:56:25.535051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:56:25.535006 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:56:30.536559 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:56:30.536512 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:56:35.537911 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:56:35.537849 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:56:40.539209 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:56:40.539117 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:56:45.540253 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:56:45.540205 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:56:50.540982 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:56:50.540938 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:56:55.542456 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:56:55.542399 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:57:00.543962 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:57:00.543921 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:57:05.545051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:57:05.545004 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:57:10.545970 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:57:10.545908 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:57:15.547532 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:57:15.547490 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:57:20.549255 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:57:20.549214 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:57:25.550835 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:57:25.550786 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:57:30.552021 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:57:30.551944 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:57:35.553463 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:57:35.553420 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:57:40.555228 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:57:40.555032 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:57:45.557055 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:57:45.557010 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:57:50.558588 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:57:50.558509 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:57:55.560206 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:57:55.560117 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:58:00.561483 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:00.561438 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:58:00.952707 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:00.952591 1563 kubelet_node_status.go:487] "Error updating node status, will retry" err="error getting node \"ropatil1611az-gs6zj-worker-westus-j4ngg\": Get \"https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes/ropatil1611az-gs6zj-worker-westus-j4ngg?resourceVersion=0&timeout=10s\": dial tcp 10.0.0.4:6443: i/o timeout (Client.Timeout exceeded while awaiting headers)" Nov 16 11:58:04.689312 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:04.689196 1563 controller.go:187] failed to update lease, error: Put "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:58:05.565379 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:05.563380 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:58:10.566530 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:10.566483 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:58:10.953853 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:10.953754 1563 kubelet_node_status.go:487] "Error updating node status, will retry" err="error getting node \"ropatil1611az-gs6zj-worker-westus-j4ngg\": Get \"https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 16 11:58:14.689814 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:14.689695 1563 controller.go:187] failed to update lease, error: Put "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:58:15.567987 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:15.567942 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:58:20.310248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:58:20.310199 1563 kubelet.go:1377] "Image garbage collection succeeded" Nov 16 11:58:20.323375 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:58:20.323281655Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=f725863a-86c3-47d8-9f18-12e29ee64936 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:58:20.323715 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:58:20.323548357Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40 not found" id=f725863a-86c3-47d8-9f18-12e29ee64936 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:58:20.569096 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:20.568978 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:58:20.954462 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:20.954388 1563 kubelet_node_status.go:487] "Error updating node status, will retry" err="error getting node \"ropatil1611az-gs6zj-worker-westus-j4ngg\": Get \"https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 16 11:58:24.690914 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:24.690776 1563 controller.go:187] failed to update lease, error: Put "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:58:25.569765 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:25.569703 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:58:30.571462 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:30.571405 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:58:30.955636 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:30.955582 1563 kubelet_node_status.go:487] "Error updating node status, will retry" err="error getting node \"ropatil1611az-gs6zj-worker-westus-j4ngg\": Get \"https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 16 11:58:34.691315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:34.691216 1563 controller.go:187] failed to update lease, error: Put "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": dial tcp 10.0.0.4:6443: i/o timeout Nov 16 11:58:35.572506 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:35.572463 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:58:40.574079 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:40.574038 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:58:40.956761 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:40.956662 1563 kubelet_node_status.go:487] "Error updating node status, will retry" err="error getting node \"ropatil1611az-gs6zj-worker-westus-j4ngg\": Get \"https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 16 11:58:40.956761 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:40.956699 1563 kubelet_node_status.go:474] "Unable to update node status" err="update node status exceeds retry count" Nov 16 11:58:44.691714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:44.691609 1563 controller.go:187] failed to update lease, error: Put "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:58:44.691714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:58:44.691667 1563 controller.go:114] failed to update lease using latest lease, fallback to ensure lease, err: failed 5 attempts to update lease Nov 16 11:58:45.575571 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:45.575526 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:58:50.576908 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:50.576863 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:58:54.692293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:54.692154 1563 controller.go:144] failed to ensure lease exists, will retry in 200ms, error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:58:55.578665 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:58:55.578619 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:59:00.579921 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:00.579883 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:59:01.319699 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:01.319589 1563 kubelet_node_status.go:487] "Error updating node status, will retry" err="error getting node \"ropatil1611az-gs6zj-worker-westus-j4ngg\": Get \"https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/api/v1/nodes/ropatil1611az-gs6zj-worker-westus-j4ngg?resourceVersion=0&timeout=10s\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 16 11:59:04.893408 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:04.893325 1563 controller.go:144] failed to ensure lease exists, will retry in 400ms, error: Get "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 16 11:59:05.581075 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:05.581018 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:59:09.888279 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:09.888059 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-machine-config-operator/machine-config-daemon-kxqnl openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh openshift-multus/multus-5wch2 openshift-multus/multus-additional-cni-plugins-q722j openshift-multus/network-metrics-daemon-hr6jx openshift-monitoring/node-exporter-dz75k openshift-network-diagnostics/network-check-target-299n8 openshift-sdn/sdn-pgwdx openshift-dns/node-resolver-l5trv openshift-image-registry/node-ca-dn4b9 openshift-cluster-node-tuning-operator/tuned-kt88r] Nov 16 11:59:09.888279 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:09.888107 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:09.888809 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:09.888395 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:09.888809 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:09.888536 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:09.888999 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:09.888961 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:09.889364 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:09.889336 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:09.889481 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:09.889473 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:09.889805 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:09.889776 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:09.889887 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:09.889808 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:09.889932 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:09.889888 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:09.890040 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:09.890018 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:09.890258 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:09.890229 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:09.890367 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:09.890344 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:09.890804 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:09.890776 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:09.898934 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod2e56696b_4c5c_4c9f_8681_c75ba7546700.slice. Nov 16 11:59:09.911224 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:09.911175 1563 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e56696b_4c5c_4c9f_8681_c75ba7546700.slice": inotify_add_watch /sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e56696b_4c5c_4c9f_8681_c75ba7546700.slice: no such file or directory Nov 16 11:59:09.921665 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod89deb745_c538_46ae_9d77_26643bf01a15.slice. Nov 16 11:59:09.922406 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-podd38704ca_571d_40ee_84b3_71c1d45d56f4.slice. Nov 16 11:59:09.927922 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod13c954d2_615d_4b55_be8a_e712b8fa30a8.slice. Nov 16 11:59:09.938669 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod5026bf33_68a7_4497_8c41_20a85e407ef9.slice. Nov 16 11:59:09.956643 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod45cd7283_fcc9_4e61_b2b1_517445b7bd5a.slice. Nov 16 11:59:09.957309 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod40251af6_202f_415a_aa28_c362b501dbd3.slice. Nov 16 11:59:09.962694 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod39a966fc_6573_4f04_9a9d_8cb766b277ba.slice. Nov 16 11:59:09.972323 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod37496343_5356_4736_accc_adc8f05b43c5.slice. Nov 16 11:59:10.056362 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056297 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-tls\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.056362 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056365 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-wtmp\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.056558 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056396 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5b5s\" (UniqueName: \"kubernetes.io/projected/5026bf33-68a7-4497-8c41-20a85e407ef9-kube-api-access-w5b5s\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.056558 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056464 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39a966fc-6573-4f04-9a9d-8cb766b277ba-host\") pod \"node-ca-dn4b9\" (UID: \"39a966fc-6573-4f04-9a9d-8cb766b277ba\") " pod="openshift-image-registry/node-ca-dn4b9" Nov 16 11:59:10.056625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056564 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-var-run-dbus\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.056688 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056656 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-os-release\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.056727 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056709 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srjjp\" (UniqueName: \"kubernetes.io/projected/d38704ca-571d-40ee-84b3-71c1d45d56f4-kube-api-access-srjjp\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.056763 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056744 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/39a966fc-6573-4f04-9a9d-8cb766b277ba-serviceca\") pod \"node-ca-dn4b9\" (UID: \"39a966fc-6573-4f04-9a9d-8cb766b277ba\") " pod="openshift-image-registry/node-ca-dn4b9" Nov 16 11:59:10.056817 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056774 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-run-netns\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-netns\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.056817 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056801 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/2e56696b-4c5c-4c9f-8681-c75ba7546700-rootfs\") pod \"machine-config-daemon-kxqnl\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" Nov 16 11:59:10.056891 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056830 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scsi-host-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-scsi-host-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.056891 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056858 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnggp\" (UniqueName: \"kubernetes.io/projected/89deb745-c538-46ae-9d77-26643bf01a15-kube-api-access-mnggp\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.056891 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056885 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-run-ovs\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-ovs\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.056977 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056910 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-run-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-kubernetes\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.056977 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056942 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"merged-cloud-config\" (UniqueName: \"kubernetes.io/empty-dir/89deb745-c538-46ae-9d77-26643bf01a15-merged-cloud-config\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.057040 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.056986 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd-system\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-run-systemd-system\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.057077 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057044 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-lib-modules\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.057112 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057088 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-device-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.057231 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057200 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-run-netns\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.057318 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057291 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pw2z\" (UniqueName: \"kubernetes.io/projected/37496343-5356-4736-accc-adc8f05b43c5-kube-api-access-8pw2z\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.057395 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057369 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-textfile\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.057440 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057427 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-cni-bin\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.057476 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057463 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-cnibin\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.057511 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057503 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cloud-config\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-host-cloud-config\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.057546 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057532 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-devices-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-sys-devices-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.057617 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057565 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89deb745-c538-46ae-9d77-26643bf01a15-non-standard-root-system-trust-ca-bundle\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.057661 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057634 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-cnibin\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.057696 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057669 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-root\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.057757 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057730 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-multus-cni-dir\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.057825 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057806 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.057871 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057847 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-run\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.057907 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057875 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9cd9\" (UniqueName: \"kubernetes.io/projected/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-kube-api-access-l9cd9\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.057907 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057900 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-etc-sysconfig\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.057970 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.057933 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-host\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.058054 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058010 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57s25\" (UniqueName: \"kubernetes.io/projected/2e56696b-4c5c-4c9f-8681-c75ba7546700-kube-api-access-57s25\") pod \"machine-config-daemon-kxqnl\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" Nov 16 11:59:10.058173 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058081 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs\") pod \"network-metrics-daemon-hr6jx\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " pod="openshift-multus/network-metrics-daemon-hr6jx" Nov 16 11:59:10.058173 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058116 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-system-cni-dir\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.058241 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058186 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-os-release\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.058278 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058263 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-sys\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.058313 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058302 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.058350 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058329 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9nsk\" (UniqueName: \"kubernetes.io/projected/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-kube-api-access-r9nsk\") pod \"network-metrics-daemon-hr6jx\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " pod="openshift-multus/network-metrics-daemon-hr6jx" Nov 16 11:59:10.058387 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058354 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.058387 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058385 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm8zx\" (UniqueName: \"kubernetes.io/projected/13c954d2-615d-4b55-be8a-e712b8fa30a8-kube-api-access-jm8zx\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.058448 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058415 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdfrr\" (UniqueName: \"kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr\") pod \"network-check-target-299n8\" (UID: \"1bd11256-096f-4190-9ee6-f23c6eaabf65\") " pod="openshift-network-diagnostics/network-check-target-299n8" Nov 16 11:59:10.058448 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058439 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-modules\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-modules\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.058511 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058467 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-lib-cni-networks-openshift-sdn\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.058511 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058491 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-socket-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.058570 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058549 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-registration-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.058604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058596 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-config\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.058644 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058628 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-conf\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-cni-conf\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.058680 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058655 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cookie-secret\" (UniqueName: \"kubernetes.io/secret/2e56696b-4c5c-4c9f-8681-c75ba7546700-cookie-secret\") pod \"machine-config-daemon-kxqnl\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" Nov 16 11:59:10.058715 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058706 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-slash\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.058750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058741 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sdn-metrics-certs\" (UniqueName: \"kubernetes.io/secret/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-sdn-metrics-certs\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.058785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058769 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e56696b-4c5c-4c9f-8681-c75ba7546700-proxy-tls\") pod \"machine-config-daemon-kxqnl\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" Nov 16 11:59:10.058824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058795 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-system-cni-dir\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.058874 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058850 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"msi\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-msi\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.058917 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058904 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5026bf33-68a7-4497-8c41-20a85e407ef9-metrics-client-ca\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.058954 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058942 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-env-overrides\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.058989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.058976 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-run-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-openshift-sdn\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.059027 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.059010 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/40251af6-202f-415a-aa28-c362b501dbd3-hosts-file\") pod \"node-resolver-l5trv\" (UID: \"40251af6-202f-415a-aa28-c362b501dbd3\") " pod="openshift-dns/node-resolver-l5trv" Nov 16 11:59:10.059088 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.059058 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwbs4\" (UniqueName: \"kubernetes.io/projected/40251af6-202f-415a-aa28-c362b501dbd3-kube-api-access-xwbs4\") pod \"node-resolver-l5trv\" (UID: \"40251af6-202f-415a-aa28-c362b501dbd3\") " pod="openshift-dns/node-resolver-l5trv" Nov 16 11:59:10.059170 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.059141 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-tuned-profiles-data\" (UniqueName: \"kubernetes.io/configmap/37496343-5356-4736-accc-adc8f05b43c5-var-lib-tuned-profiles-data\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.059218 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.059187 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-mountpoint-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.059255 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.059226 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-dbus\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.059293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.059255 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc6n8\" (UniqueName: \"kubernetes.io/projected/39a966fc-6573-4f04-9a9d-8cb766b277ba-kube-api-access-pc6n8\") pod \"node-ca-dn4b9\" (UID: \"39a966fc-6573-4f04-9a9d-8cb766b277ba\") " pod="openshift-image-registry/node-ca-dn4b9" Nov 16 11:59:10.059330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.059299 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-etc\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.059367 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.059331 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-sys\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.160196 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160027 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-mnggp\" (UniqueName: \"kubernetes.io/projected/89deb745-c538-46ae-9d77-26643bf01a15-kube-api-access-mnggp\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.160196 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160104 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-run-ovs\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-ovs\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.160196 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160155 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-run-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-kubernetes\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.160196 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160189 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"merged-cloud-config\" (UniqueName: \"kubernetes.io/empty-dir/89deb745-c538-46ae-9d77-26643bf01a15-merged-cloud-config\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160221 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"run-systemd-system\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-run-systemd-system\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160237 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-run-ovs\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-ovs\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160261 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-lib-modules\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160313 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-device-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160353 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-run-netns\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160384 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-8pw2z\" (UniqueName: \"kubernetes.io/projected/37496343-5356-4736-accc-adc8f05b43c5-kube-api-access-8pw2z\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160386 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-device-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160314 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-run-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-kubernetes\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160421 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-cnibin\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160459 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-root\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160423 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"run-systemd-system\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-run-systemd-system\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160491 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-textfile\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160464 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-run-netns\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.160504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160467 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-lib-modules\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160513 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-root\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160475 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-cnibin\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160522 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-cni-bin\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160562 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-cni-bin\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160593 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-cnibin\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160631 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-cloud-config\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-host-cloud-config\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160652 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-cnibin\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160662 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"sys-devices-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-sys-devices-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160698 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89deb745-c538-46ae-9d77-26643bf01a15-non-standard-root-system-trust-ca-bundle\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160720 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-cloud-config\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-host-cloud-config\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160727 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-multus-cni-dir\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160759 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-multus-cni-dir\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160747 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"sys-devices-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-sys-devices-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160789 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160825 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-run\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160869 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-l9cd9\" (UniqueName: \"kubernetes.io/projected/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-kube-api-access-l9cd9\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160873 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-run\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.160889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160900 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-sys\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160965 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-sys\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.160934 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161020 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-etc-sysconfig\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161048 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-host\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161078 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-57s25\" (UniqueName: \"kubernetes.io/projected/2e56696b-4c5c-4c9f-8681-c75ba7546700-kube-api-access-57s25\") pod \"machine-config-daemon-kxqnl\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161108 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs\") pod \"network-metrics-daemon-hr6jx\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " pod="openshift-multus/network-metrics-daemon-hr6jx" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161143 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-etc-sysconfig\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161155 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-system-cni-dir\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161190 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-os-release\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161195 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"merged-cloud-config\" (UniqueName: \"kubernetes.io/empty-dir/89deb745-c538-46ae-9d77-26643bf01a15-merged-cloud-config\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161218 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-modules\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-modules\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161202 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-host\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161244 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-system-cni-dir\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161249 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-lib-cni-networks-openshift-sdn\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161284 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-lib-cni-networks-openshift-sdn\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161309 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-r9nsk\" (UniqueName: \"kubernetes.io/projected/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-kube-api-access-r9nsk\") pod \"network-metrics-daemon-hr6jx\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " pod="openshift-multus/network-metrics-daemon-hr6jx" Nov 16 11:59:10.161403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161307 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-os-release\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161323 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-modules\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-modules\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161307 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-textfile\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161341 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161395 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-jm8zx\" (UniqueName: \"kubernetes.io/projected/13c954d2-615d-4b55-be8a-e712b8fa30a8-kube-api-access-jm8zx\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161437 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-zdfrr\" (UniqueName: \"kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr\") pod \"network-check-target-299n8\" (UID: \"1bd11256-096f-4190-9ee6-f23c6eaabf65\") " pod="openshift-network-diagnostics/network-check-target-299n8" Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161473 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-socket-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161504 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-registration-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161542 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-config\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161573 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-cni-conf\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-cni-conf\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161602 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"cookie-secret\" (UniqueName: \"kubernetes.io/secret/2e56696b-4c5c-4c9f-8681-c75ba7546700-cookie-secret\") pod \"machine-config-daemon-kxqnl\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:10.161626 1563 secret.go:195] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:10.161693 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs podName:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 nodeName:}" failed. No retries permitted until 2022-11-16 11:59:10.661671141 +0000 UTC m=+355.598446863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs") pod "network-metrics-daemon-hr6jx" (UID: "6754c3dd-53dd-4e4f-8d5f-ce077669c3d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161631 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-env-overrides\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161830 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-socket-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.161875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161848 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-run-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-openshift-sdn\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161883 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-slash\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161914 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"sdn-metrics-certs\" (UniqueName: \"kubernetes.io/secret/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-sdn-metrics-certs\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161942 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e56696b-4c5c-4c9f-8681-c75ba7546700-proxy-tls\") pod \"machine-config-daemon-kxqnl\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161972 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-system-cni-dir\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.161987 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-slash\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162001 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"msi\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-msi\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162030 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5026bf33-68a7-4497-8c41-20a85e407ef9-metrics-client-ca\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162057 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/40251af6-202f-415a-aa28-c362b501dbd3-hosts-file\") pod \"node-resolver-l5trv\" (UID: \"40251af6-202f-415a-aa28-c362b501dbd3\") " pod="openshift-dns/node-resolver-l5trv" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162069 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-run-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-openshift-sdn\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162068 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89deb745-c538-46ae-9d77-26643bf01a15-non-standard-root-system-trust-ca-bundle\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162090 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-xwbs4\" (UniqueName: \"kubernetes.io/projected/40251af6-202f-415a-aa28-c362b501dbd3-kube-api-access-xwbs4\") pod \"node-resolver-l5trv\" (UID: \"40251af6-202f-415a-aa28-c362b501dbd3\") " pod="openshift-dns/node-resolver-l5trv" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162098 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162114 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-system-cni-dir\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162124 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"var-lib-tuned-profiles-data\" (UniqueName: \"kubernetes.io/configmap/37496343-5356-4736-accc-adc8f05b43c5-var-lib-tuned-profiles-data\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162171 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-registration-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162178 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-sys\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162209 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-mountpoint-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.162310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162218 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-cni-conf\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-cni-conf\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162240 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-dbus\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162269 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-pc6n8\" (UniqueName: \"kubernetes.io/projected/39a966fc-6573-4f04-9a9d-8cb766b277ba-kube-api-access-pc6n8\") pod \"node-ca-dn4b9\" (UID: \"39a966fc-6573-4f04-9a9d-8cb766b277ba\") " pod="openshift-image-registry/node-ca-dn4b9" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162297 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"etc\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-etc\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162339 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-var-run-dbus\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162370 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-tls\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162401 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-wtmp\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162432 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-w5b5s\" (UniqueName: \"kubernetes.io/projected/5026bf33-68a7-4497-8c41-20a85e407ef9-kube-api-access-w5b5s\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162459 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39a966fc-6573-4f04-9a9d-8cb766b277ba-host\") pod \"node-ca-dn4b9\" (UID: \"39a966fc-6573-4f04-9a9d-8cb766b277ba\") " pod="openshift-image-registry/node-ca-dn4b9" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162487 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-os-release\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162515 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-srjjp\" (UniqueName: \"kubernetes.io/projected/d38704ca-571d-40ee-84b3-71c1d45d56f4-kube-api-access-srjjp\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162557 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/39a966fc-6573-4f04-9a9d-8cb766b277ba-serviceca\") pod \"node-ca-dn4b9\" (UID: \"39a966fc-6573-4f04-9a9d-8cb766b277ba\") " pod="openshift-image-registry/node-ca-dn4b9" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162584 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-run-netns\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-netns\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162593 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"msi\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-msi\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162622 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/2e56696b-4c5c-4c9f-8681-c75ba7546700-rootfs\") pod \"machine-config-daemon-kxqnl\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162652 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"scsi-host-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-scsi-host-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.162785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162725 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"scsi-host-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-scsi-host-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.163236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162805 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.163236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162808 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/40251af6-202f-415a-aa28-c362b501dbd3-hosts-file\") pod \"node-resolver-l5trv\" (UID: \"40251af6-202f-415a-aa28-c362b501dbd3\") " pod="openshift-dns/node-resolver-l5trv" Nov 16 11:59:10.163236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162938 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-wtmp\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.163236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.162987 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-sys\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.163236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.163017 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-config\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.163236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.163039 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-mountpoint-dir\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.163236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.163071 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-dbus\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.163427 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.163246 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-var-run-dbus\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.163427 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.163305 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"etc\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-etc\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.163661 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.163643 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-env-overrides\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.163707 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.163677 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5026bf33-68a7-4497-8c41-20a85e407ef9-metrics-client-ca\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.163747 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.163712 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-run-netns\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-netns\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.163747 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.163738 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39a966fc-6573-4f04-9a9d-8cb766b277ba-host\") pod \"node-ca-dn4b9\" (UID: \"39a966fc-6573-4f04-9a9d-8cb766b277ba\") " pod="openshift-image-registry/node-ca-dn4b9" Nov 16 11:59:10.163986 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.163834 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-os-release\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.163986 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.163902 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/2e56696b-4c5c-4c9f-8681-c75ba7546700-rootfs\") pod \"machine-config-daemon-kxqnl\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" Nov 16 11:59:10.164086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.164019 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"var-lib-tuned-profiles-data\" (UniqueName: \"kubernetes.io/configmap/37496343-5356-4736-accc-adc8f05b43c5-var-lib-tuned-profiles-data\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.165505 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.165464 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/39a966fc-6573-4f04-9a9d-8cb766b277ba-serviceca\") pod \"node-ca-dn4b9\" (UID: \"39a966fc-6573-4f04-9a9d-8cb766b277ba\") " pod="openshift-image-registry/node-ca-dn4b9" Nov 16 11:59:10.166163 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.166119 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"sdn-metrics-certs\" (UniqueName: \"kubernetes.io/secret/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-sdn-metrics-certs\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.166163 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.166168 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"cookie-secret\" (UniqueName: \"kubernetes.io/secret/2e56696b-4c5c-4c9f-8681-c75ba7546700-cookie-secret\") pod \"machine-config-daemon-kxqnl\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" Nov 16 11:59:10.166653 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.166590 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-tls\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.166653 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.166621 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.166653 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.166645 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e56696b-4c5c-4c9f-8681-c75ba7546700-proxy-tls\") pod \"machine-config-daemon-kxqnl\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" Nov 16 11:59:10.188380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.188349 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnggp\" (UniqueName: \"kubernetes.io/projected/89deb745-c538-46ae-9d77-26643bf01a15-kube-api-access-mnggp\") pod \"azure-disk-csi-driver-node-f55gh\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.198942 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.198877 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc6n8\" (UniqueName: \"kubernetes.io/projected/39a966fc-6573-4f04-9a9d-8cb766b277ba-kube-api-access-pc6n8\") pod \"node-ca-dn4b9\" (UID: \"39a966fc-6573-4f04-9a9d-8cb766b277ba\") " pod="openshift-image-registry/node-ca-dn4b9" Nov 16 11:59:10.202437 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.202393 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm8zx\" (UniqueName: \"kubernetes.io/projected/13c954d2-615d-4b55-be8a-e712b8fa30a8-kube-api-access-jm8zx\") pod \"multus-additional-cni-plugins-q722j\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.202600 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.202480 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pw2z\" (UniqueName: \"kubernetes.io/projected/37496343-5356-4736-accc-adc8f05b43c5-kube-api-access-8pw2z\") pod \"tuned-kt88r\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.203935 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.203880 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9nsk\" (UniqueName: \"kubernetes.io/projected/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-kube-api-access-r9nsk\") pod \"network-metrics-daemon-hr6jx\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " pod="openshift-multus/network-metrics-daemon-hr6jx" Nov 16 11:59:10.206058 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.205993 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9cd9\" (UniqueName: \"kubernetes.io/projected/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-kube-api-access-l9cd9\") pod \"sdn-pgwdx\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.206212 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.206180 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-srjjp\" (UniqueName: \"kubernetes.io/projected/d38704ca-571d-40ee-84b3-71c1d45d56f4-kube-api-access-srjjp\") pod \"multus-5wch2\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.207788 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.207729 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwbs4\" (UniqueName: \"kubernetes.io/projected/40251af6-202f-415a-aa28-c362b501dbd3-kube-api-access-xwbs4\") pod \"node-resolver-l5trv\" (UID: \"40251af6-202f-415a-aa28-c362b501dbd3\") " pod="openshift-dns/node-resolver-l5trv" Nov 16 11:59:10.208980 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.208948 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5b5s\" (UniqueName: \"kubernetes.io/projected/5026bf33-68a7-4497-8c41-20a85e407ef9-kube-api-access-w5b5s\") pod \"node-exporter-dz75k\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.209081 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.209056 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-57s25\" (UniqueName: \"kubernetes.io/projected/2e56696b-4c5c-4c9f-8681-c75ba7546700-kube-api-access-57s25\") pod \"machine-config-daemon-kxqnl\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" Nov 16 11:59:10.214981 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:10.214950 1563 projected.go:293] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 16 11:59:10.214981 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:10.214981 1563 projected.go:293] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 16 11:59:10.215097 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:10.214994 1563 projected.go:199] Error preparing data for projected volume kube-api-access-zdfrr for pod openshift-network-diagnostics/network-check-target-299n8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 16 11:59:10.215097 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:10.215058 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr podName:1bd11256-096f-4190-9ee6-f23c6eaabf65 nodeName:}" failed. No retries permitted until 2022-11-16 11:59:10.715039934 +0000 UTC m=+355.651815656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zdfrr" (UniqueName: "kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr") pod "network-check-target-299n8" (UID: "1bd11256-096f-4190-9ee6-f23c6eaabf65") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 16 11:59:10.217617 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.217555 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" Nov 16 11:59:10.218659 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.218395259Z" level=info msg="Running pod sandbox: openshift-machine-config-operator/machine-config-daemon-kxqnl/POD" id=1b6bb318-6142-4115-9bde-f5072ba75878 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:10.218659 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.218645761Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:10.232876 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.232814 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5wch2" Nov 16 11:59:10.233591 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.233516 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" Nov 16 11:59:10.233694 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.233624071Z" level=info msg="Running pod sandbox: openshift-multus/multus-5wch2/POD" id=608dc93e-0c4b-4aaa-a827-7b69da5f4ecb name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:10.233694 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.233687572Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:10.234662 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.234568078Z" level=info msg="Running pod sandbox: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/POD" id=ecfc0311-79d8-4bc8-998e-34402bd972fa name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:10.234854 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.234697179Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:10.238311 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.238286 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q722j" Nov 16 11:59:10.239023 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.238897210Z" level=info msg="Running pod sandbox: openshift-multus/multus-additional-cni-plugins-q722j/POD" id=60ca09ff-c732-4faf-acac-87f84aa34d92 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:10.239023 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.239004811Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:10.252465 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.252395 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dz75k" Nov 16 11:59:10.253062 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.253010714Z" level=info msg="Running pod sandbox: openshift-monitoring/node-exporter-dz75k/POD" id=7cfb6c5a-cde7-428b-84d6-42f02f1e9830 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:10.253183 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.253080815Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:10.266207 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.266122 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l5trv" Nov 16 11:59:10.266778 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.266721815Z" level=info msg="Running pod sandbox: openshift-dns/node-resolver-l5trv/POD" id=2dfd0c7c-3c16-40e4-9748-8e91501e3cbc name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:10.266860 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.266785016Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:10.268112 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.268032 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:10.268783 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.268726930Z" level=info msg="Running pod sandbox: openshift-sdn/sdn-pgwdx/POD" id=9212be22-fcf7-4ed3-bf57-35ad13693a75 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:10.268864 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.268795131Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:10.270002 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.269969 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dn4b9" Nov 16 11:59:10.270507 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.270452743Z" level=info msg="Running pod sandbox: openshift-image-registry/node-ca-dn4b9/POD" id=802c3d61-e77c-4d51-9ad6-2f89d3328e33 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:10.270579 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.270518343Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:10.277012 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.276935 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kt88r" Nov 16 11:59:10.277560 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.277511295Z" level=info msg="Running pod sandbox: openshift-cluster-node-tuning-operator/tuned-kt88r/POD" id=5fd3d7c7-ebe4-45c9-9a95-d2cca4a5a998 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:10.277633 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:10.277576595Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:10.582156 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:10.582023 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:59:10.667631 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.667527 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs\") pod \"network-metrics-daemon-hr6jx\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " pod="openshift-multus/network-metrics-daemon-hr6jx" Nov 16 11:59:10.667823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:10.667688 1563 secret.go:195] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 16 11:59:10.667823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:10.667772 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs podName:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 nodeName:}" failed. No retries permitted until 2022-11-16 11:59:11.667754673 +0000 UTC m=+356.604530295 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs") pod "network-metrics-daemon-hr6jx" (UID: "6754c3dd-53dd-4e4f-8d5f-ce077669c3d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 16 11:59:10.768031 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:10.767915 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-zdfrr\" (UniqueName: \"kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr\") pod \"network-check-target-299n8\" (UID: \"1bd11256-096f-4190-9ee6-f23c6eaabf65\") " pod="openshift-network-diagnostics/network-check-target-299n8" Nov 16 11:59:10.768302 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:10.768163 1563 projected.go:293] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 16 11:59:10.768302 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:10.768224 1563 projected.go:293] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 16 11:59:10.768302 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:10.768237 1563 projected.go:199] Error preparing data for projected volume kube-api-access-zdfrr for pod openshift-network-diagnostics/network-check-target-299n8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 16 11:59:10.768433 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:10.768318 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr podName:1bd11256-096f-4190-9ee6-f23c6eaabf65 nodeName:}" failed. No retries permitted until 2022-11-16 11:59:11.768283415 +0000 UTC m=+356.705059037 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zdfrr" (UniqueName: "kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr") pod "network-check-target-299n8" (UID: "1bd11256-096f-4190-9ee6-f23c6eaabf65") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 16 11:59:11.586662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:11.586557 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:11.587336 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:11.586718 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:11.675617 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:11.675552 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs\") pod \"network-metrics-daemon-hr6jx\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " pod="openshift-multus/network-metrics-daemon-hr6jx" Nov 16 11:59:11.675805 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:11.675783 1563 secret.go:195] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 16 11:59:11.675892 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:11.675873 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs podName:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 nodeName:}" failed. No retries permitted until 2022-11-16 11:59:13.675856709 +0000 UTC m=+358.612632331 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs") pod "network-metrics-daemon-hr6jx" (UID: "6754c3dd-53dd-4e4f-8d5f-ce077669c3d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 16 11:59:11.776832 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:11.776766 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-zdfrr\" (UniqueName: \"kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr\") pod \"network-check-target-299n8\" (UID: \"1bd11256-096f-4190-9ee6-f23c6eaabf65\") " pod="openshift-network-diagnostics/network-check-target-299n8" Nov 16 11:59:11.776994 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:11.776940 1563 projected.go:293] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 16 11:59:11.776994 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:11.776961 1563 projected.go:293] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 16 11:59:11.776994 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:11.776972 1563 projected.go:199] Error preparing data for projected volume kube-api-access-zdfrr for pod openshift-network-diagnostics/network-check-target-299n8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 16 11:59:11.777100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:11.777087 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr podName:1bd11256-096f-4190-9ee6-f23c6eaabf65 nodeName:}" failed. No retries permitted until 2022-11-16 11:59:13.777049455 +0000 UTC m=+358.713825077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zdfrr" (UniqueName: "kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr") pod "network-check-target-299n8" (UID: "1bd11256-096f-4190-9ee6-f23c6eaabf65") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 16 11:59:13.393881 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.393827736Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=1b6bb318-6142-4115-9bde-f5072ba75878 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.452355 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.452254169Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=802c3d61-e77c-4d51-9ad6-2f89d3328e33 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.536065 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.535979790Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=7cfb6c5a-cde7-428b-84d6-42f02f1e9830 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.586891 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:13.586758 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:13.587378 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:13.586913 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:13.603897 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.603806892Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=ecfc0311-79d8-4bc8-998e-34402bd972fa name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.613300 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.613199462Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=9212be22-fcf7-4ed3-bf57-35ad13693a75 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.641726 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.641637573Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=5fd3d7c7-ebe4-45c9-9a95-d2cca4a5a998 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.650652 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.650497539Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=608dc93e-0c4b-4aaa-a827-7b69da5f4ecb name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.665190 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.665081847Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=2dfd0c7c-3c16-40e4-9748-8e91501e3cbc name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.673816 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.673706011Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=60ca09ff-c732-4faf-acac-87f84aa34d92 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.695323 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:13.695210 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs\") pod \"network-metrics-daemon-hr6jx\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " pod="openshift-multus/network-metrics-daemon-hr6jx" Nov 16 11:59:13.695512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:13.695469 1563 secret.go:195] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 16 11:59:13.695564 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:13.695557 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs podName:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 nodeName:}" failed. No retries permitted until 2022-11-16 11:59:17.695537773 +0000 UTC m=+362.632313395 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs") pod "network-metrics-daemon-hr6jx" (UID: "6754c3dd-53dd-4e4f-8d5f-ce077669c3d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 16 11:59:13.766210 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: overlayfs: unrecognized mount option "volatile" or missing value Nov 16 11:59:13.768582 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:59:13.768479 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5026bf33_68a7_4497_8c41_20a85e407ef9.slice/crio-502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b.scope WatchSource:0}: Error finding container 502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b: Status 404 returned error &{%!s(*http.body=&{0xc000d7ba70 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 11:59:13.776196 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: overlayfs: unrecognized mount option "volatile" or missing value Nov 16 11:59:13.779525 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:59:13.779492 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89deb745_c538_46ae_9d77_26643bf01a15.slice/crio-8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883.scope WatchSource:0}: Error finding container 8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883: Status 404 returned error &{%!s(*http.body=&{0xc0018a3878 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 11:59:13.779898 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:59:13.779864 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37496343_5356_4736_accc_adc8f05b43c5.slice/crio-dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669.scope WatchSource:0}: Error finding container dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669: Status 404 returned error &{%!s(*http.body=&{0xc000a8fa70 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 11:59:13.784470 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:59:13.784440 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45cd7283_fcc9_4e61_b2b1_517445b7bd5a.slice/crio-6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75.scope WatchSource:0}: Error finding container 6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75: Status 404 returned error &{%!s(*http.body=&{0xc0018a3a10 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 11:59:13.785491 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.785446939Z" level=info msg="Ran pod sandbox 502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b with infra container: openshift-monitoring/node-exporter-dz75k/POD" id=7cfb6c5a-cde7-428b-84d6-42f02f1e9830 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.785707 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.785478939Z" level=info msg="Ran pod sandbox 57035fc21822c56b1677df50b84a40801ec6643b61f910c3ed96ed7ba49c71b8 with infra container: openshift-dns/node-resolver-l5trv/POD" id=2dfd0c7c-3c16-40e4-9748-8e91501e3cbc name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.785897 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.785857642Z" level=info msg="Ran pod sandbox dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669 with infra container: openshift-cluster-node-tuning-operator/tuned-kt88r/POD" id=5fd3d7c7-ebe4-45c9-9a95-d2cca4a5a998 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.786007 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.785976443Z" level=info msg="Ran pod sandbox 6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75 with infra container: openshift-sdn/sdn-pgwdx/POD" id=9212be22-fcf7-4ed3-bf57-35ad13693a75 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.786320 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.786273545Z" level=info msg="Ran pod sandbox 8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883 with infra container: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/POD" id=ecfc0311-79d8-4bc8-998e-34402bd972fa name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.790553 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.790475576Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:14c88d159fb9a50aa6e7c65727eb024281451ea06ad3f48bf934331dfb48f300" id=37ef905e-cd04-4472-a016-8e3e181cea88 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.790801 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.790763479Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:14c88d159fb9a50aa6e7c65727eb024281451ea06ad3f48bf934331dfb48f300 not found" id=37ef905e-cd04-4472-a016-8e3e181cea88 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.790900 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.790871879Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bee02fcb1fc5f7fd8127772a514ea08c04eeb1241e41a4ec5fe9fa7307417794" id=b90cbcc7-a474-40af-92ec-3658a239f379 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.791118 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.791090981Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bee02fcb1fc5f7fd8127772a514ea08c04eeb1241e41a4ec5fe9fa7307417794 not found" id=b90cbcc7-a474-40af-92ec-3658a239f379 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.791236 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.791204382Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48432695b88d749dcba50d0e9a1f5072ddf9067355a01b59fe3b0cc917bf8b23" id=a815bf41-dcc2-40f8-8e84-da3e349452d6 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.791450 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.791423483Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48432695b88d749dcba50d0e9a1f5072ddf9067355a01b59fe3b0cc917bf8b23 not found" id=a815bf41-dcc2-40f8-8e84-da3e349452d6 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.791939 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.791553984Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3909f49bf7a6d86ee5d24039d79506226fa6d0a80ac11956ac1dd770be4630d0" id=7d3aa0f2-e337-4f1d-b9a6-7aad02cfe7d4 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.792306 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.792270990Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3909f49bf7a6d86ee5d24039d79506226fa6d0a80ac11956ac1dd770be4630d0 not found" id=7d3aa0f2-e337-4f1d-b9a6-7aad02cfe7d4 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.792389 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:13.792369 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 11:59:13.792582 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:59:13.792549 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38704ca_571d_40ee_84b3_71c1d45d56f4.slice/crio-31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427.scope WatchSource:0}: Error finding container 31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427: Status 404 returned error &{%!s(*http.body=&{0xc0018a3ad0 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 11:59:13.793044 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:59:13.793006 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13c954d2_615d_4b55_be8a_e712b8fa30a8.slice/crio-8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3.scope WatchSource:0}: Error finding container 8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3: Status 404 returned error &{%!s(*http.body=&{0xc000d7be60 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 11:59:13.793528 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:59:13.793495 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e56696b_4c5c_4c9f_8681_c75ba7546700.slice/crio-1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c.scope WatchSource:0}: Error finding container 1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c: Status 404 returned error &{%!s(*http.body=&{0xc000d53a58 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 11:59:13.793798 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.793750601Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:14c88d159fb9a50aa6e7c65727eb024281451ea06ad3f48bf934331dfb48f300" id=3529c2cc-989b-4536-a320-28b4656600be name=/runtime.v1.ImageService/PullImage Nov 16 11:59:13.794028 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.793994302Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bee02fcb1fc5f7fd8127772a514ea08c04eeb1241e41a4ec5fe9fa7307417794" id=1798497c-7e47-4e2f-81a4-06cec42bbf37 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:13.794196 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.794052003Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48432695b88d749dcba50d0e9a1f5072ddf9067355a01b59fe3b0cc917bf8b23" id=bbfa526a-cee7-43f1-b920-14fe43851687 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:13.794489 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.794459906Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4578b08e8c013651a18f58957f777d7fb635959165e9353e6d20f192ae8386dc" id=76bfadb2-083a-413b-ab2e-14c6729da9e7 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.794633 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.794489306Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3909f49bf7a6d86ee5d24039d79506226fa6d0a80ac11956ac1dd770be4630d0" id=986936dd-1e93-49a8-b9cb-67131cd6a483 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:13.794803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.794772808Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4578b08e8c013651a18f58957f777d7fb635959165e9353e6d20f192ae8386dc not found" id=76bfadb2-083a-413b-ab2e-14c6729da9e7 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.795662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:13.795639 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-zdfrr\" (UniqueName: \"kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr\") pod \"network-check-target-299n8\" (UID: \"1bd11256-096f-4190-9ee6-f23c6eaabf65\") " pod="openshift-network-diagnostics/network-check-target-299n8" Nov 16 11:59:13.795783 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.795675615Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4578b08e8c013651a18f58957f777d7fb635959165e9353e6d20f192ae8386dc" id=92eec47f-77ba-4b7e-aa5c-d6f18156fd77 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:13.795978 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:59:13.795956 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a966fc_6573_4f04_9a9d_8cb766b277ba.slice/crio-e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e.scope WatchSource:0}: Error finding container e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e: Status 404 returned error &{%!s(*http.body=&{0xc00102e150 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 11:59:13.796073 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.795995217Z" level=info msg="Ran pod sandbox 8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3 with infra container: openshift-multus/multus-additional-cni-plugins-q722j/POD" id=60ca09ff-c732-4faf-acac-87f84aa34d92 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.796121 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.796067818Z" level=info msg="Ran pod sandbox 31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427 with infra container: openshift-multus/multus-5wch2/POD" id=608dc93e-0c4b-4aaa-a827-7b69da5f4ecb name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.796248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:13.795981 1563 projected.go:293] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 16 11:59:13.796370 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.796334920Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4578b08e8c013651a18f58957f777d7fb635959165e9353e6d20f192ae8386dc\"" Nov 16 11:59:13.796435 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.796370220Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48432695b88d749dcba50d0e9a1f5072ddf9067355a01b59fe3b0cc917bf8b23\"" Nov 16 11:59:13.796435 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.796420420Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:14c88d159fb9a50aa6e7c65727eb024281451ea06ad3f48bf934331dfb48f300\"" Nov 16 11:59:13.796535 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.796502621Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3909f49bf7a6d86ee5d24039d79506226fa6d0a80ac11956ac1dd770be4630d0\"" Nov 16 11:59:13.796735 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.796378820Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bee02fcb1fc5f7fd8127772a514ea08c04eeb1241e41a4ec5fe9fa7307417794\"" Nov 16 11:59:13.796977 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.796940924Z" level=info msg="Ran pod sandbox e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e with infra container: openshift-image-registry/node-ca-dn4b9/POD" id=802c3d61-e77c-4d51-9ad6-2f89d3328e33 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.797119 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.797080625Z" level=info msg="Ran pod sandbox 1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c with infra container: openshift-machine-config-operator/machine-config-daemon-kxqnl/POD" id=1b6bb318-6142-4115-9bde-f5072ba75878 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:13.797275 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:13.796360 1563 projected.go:293] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 16 11:59:13.797275 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:13.797280 1563 projected.go:199] Error preparing data for projected volume kube-api-access-zdfrr for pod openshift-network-diagnostics/network-check-target-299n8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 16 11:59:13.797403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:13.797330 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr podName:1bd11256-096f-4190-9ee6-f23c6eaabf65 nodeName:}" failed. No retries permitted until 2022-11-16 11:59:17.797312027 +0000 UTC m=+362.734087749 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zdfrr" (UniqueName: "kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr") pod "network-check-target-299n8" (UID: "1bd11256-096f-4190-9ee6-f23c6eaabf65") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 16 11:59:13.797452 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.797269527Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1288cd9c16ae683a98a48a1dc79f451f0514648b85101f53d94fc0d0c9ca73b5" id=5e2ad6a5-02c9-4b3a-af24-932c44514a84 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.797452 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.797317827Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd6de4896805b834eb7a374038b47ad6db8958bde085753ef1d0e36af4da45fc" id=ca3f1aa2-2fa4-46a9-903d-d7ac2a95b389 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.797579 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.797548529Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd6de4896805b834eb7a374038b47ad6db8958bde085753ef1d0e36af4da45fc not found" id=ca3f1aa2-2fa4-46a9-903d-d7ac2a95b389 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.797634 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.797589629Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1288cd9c16ae683a98a48a1dc79f451f0514648b85101f53d94fc0d0c9ca73b5 not found" id=5e2ad6a5-02c9-4b3a-af24-932c44514a84 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.798049 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.798013032Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fda2548f8fb1f7f1f59f9251f151f4d0b60f44799b69edfd2a8387df0298eb0f" id=7318ee79-997b-48fe-8098-a90c40f28b32 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.798116 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.798018532Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5ce45f37b20d2dd4b22bf117caf0a2234a9fdd640803269b101c6f1b4e90e60e" id=85723573-50bc-42b9-881f-f0ba3466d5ee name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.798210 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.798174333Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1288cd9c16ae683a98a48a1dc79f451f0514648b85101f53d94fc0d0c9ca73b5" id=878e0443-5c25-4081-bcc9-0958af678fce name=/runtime.v1.ImageService/PullImage Nov 16 11:59:13.798254 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.798212734Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:733c1343f71786d137b938b4071bba9cc7fb64b301e1464e63f775ec8d1531c7,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fda2548f8fb1f7f1f59f9251f151f4d0b60f44799b69edfd2a8387df0298eb0f],Size_:465842987,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=7318ee79-997b-48fe-8098-a90c40f28b32 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.798290 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.798249334Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd6de4896805b834eb7a374038b47ad6db8958bde085753ef1d0e36af4da45fc" id=37244ca1-a987-48aa-b9d9-edf3c5ec2aef name=/runtime.v1.ImageService/PullImage Nov 16 11:59:13.798385 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.798350335Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5ce45f37b20d2dd4b22bf117caf0a2234a9fdd640803269b101c6f1b4e90e60e not found" id=85723573-50bc-42b9-881f-f0ba3466d5ee name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.799055 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.798933339Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5ce45f37b20d2dd4b22bf117caf0a2234a9fdd640803269b101c6f1b4e90e60e" id=d48b7d19-70d4-4a72-accc-4f2fa741de76 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:13.799191 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.799155041Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fda2548f8fb1f7f1f59f9251f151f4d0b60f44799b69edfd2a8387df0298eb0f" id=665cc7e2-fa22-4814-b95d-4ace02ab39ad name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.799377 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.799337342Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:733c1343f71786d137b938b4071bba9cc7fb64b301e1464e63f775ec8d1531c7,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fda2548f8fb1f7f1f59f9251f151f4d0b60f44799b69edfd2a8387df0298eb0f],Size_:465842987,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=665cc7e2-fa22-4814-b95d-4ace02ab39ad name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:13.800205 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.800169448Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5ce45f37b20d2dd4b22bf117caf0a2234a9fdd640803269b101c6f1b4e90e60e\"" Nov 16 11:59:13.800285 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.800217049Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1288cd9c16ae683a98a48a1dc79f451f0514648b85101f53d94fc0d0c9ca73b5\"" Nov 16 11:59:13.800285 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.800240949Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd6de4896805b834eb7a374038b47ad6db8958bde085753ef1d0e36af4da45fc\"" Nov 16 11:59:13.800468 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.800432850Z" level=info msg="Creating container: openshift-machine-config-operator/machine-config-daemon-kxqnl/machine-config-daemon" id=6b542114-fb2f-4be2-9c6d-328cfc3f3065 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:13.800550 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.800526951Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:13.860603 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope. Nov 16 11:59:13.893385 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588. Nov 16 11:59:13.919212 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: cgroup: cgroup: disabling cgroup2 socket matching due to net_prio or net_cls activation Nov 16 11:59:13.980626 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.980563786Z" level=info msg="Created container ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588: openshift-machine-config-operator/machine-config-daemon-kxqnl/machine-config-daemon" id=6b542114-fb2f-4be2-9c6d-328cfc3f3065 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:13.981535 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.981461392Z" level=info msg="Starting container: ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588" id=3f00e8de-61c8-4da5-beeb-c68bbf2e5081 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:13.994296 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:13.994241587Z" level=info msg="Started container" PID=1735 containerID=ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588 description=openshift-machine-config-operator/machine-config-daemon-kxqnl/machine-config-daemon id=3f00e8de-61c8-4da5-beeb-c68bbf2e5081 name=/runtime.v1.RuntimeService/StartContainer sandboxID=1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c Nov 16 11:59:14.020429 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:14.020368581Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9" id=3e548e86-cfc0-4bad-bc37-f6bd196b31f5 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:14.020682 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:14.020646783Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9 not found" id=3e548e86-cfc0-4bad-bc37-f6bd196b31f5 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:14.021564 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:14.021512889Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9" id=083c4537-34af-4a5e-b086-fee6bcb6099c name=/runtime.v1.ImageService/PullImage Nov 16 11:59:14.024794 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:14.024760913Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9\"" Nov 16 11:59:14.103179 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting rpm-ostree System Management Daemon... Nov 16 11:59:14.142825 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[1771]: Reading config file '/etc/rpm-ostreed.conf' Nov 16 11:59:14.143756 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1176]: [system] Activating via systemd: service name='org.freedesktop.PolicyKit1' unit='polkit.service' requested by ':1.33' (uid=0 pid=1771 comm="/usr/bin/rpm-ostree start-daemon " label="system_u:system_r:install_t:s0") Nov 16 11:59:14.146608 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Authorization Manager... Nov 16 11:59:14.151823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:14.151611 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q722j" event=&{ID:13c954d2-615d-4b55-be8a-e712b8fa30a8 Type:ContainerStarted Data:8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3} Nov 16 11:59:14.152471 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:14.152407 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dn4b9" event=&{ID:39a966fc-6573-4f04-9a9d-8cb766b277ba Type:ContainerStarted Data:e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e} Nov 16 11:59:14.153778 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:14.153751 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" event=&{ID:2e56696b-4c5c-4c9f-8681-c75ba7546700 Type:ContainerStarted Data:ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588} Nov 16 11:59:14.153859 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:14.153785 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" event=&{ID:2e56696b-4c5c-4c9f-8681-c75ba7546700 Type:ContainerStarted Data:1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c} Nov 16 11:59:14.154632 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:14.154605 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5wch2" event=&{ID:d38704ca-571d-40ee-84b3-71c1d45d56f4 Type:ContainerStarted Data:31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427} Nov 16 11:59:14.155493 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:14.155462 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dz75k" event=&{ID:5026bf33-68a7-4497-8c41-20a85e407ef9 Type:ContainerStarted Data:502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b} Nov 16 11:59:14.156477 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:14.156442 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" event=&{ID:89deb745-c538-46ae-9d77-26643bf01a15 Type:ContainerStarted Data:8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883} Nov 16 11:59:14.157226 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:14.157193 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-sdn/sdn-pgwdx" event=&{ID:45cd7283-fcc9-4e61-b2b1-517445b7bd5a Type:ContainerStarted Data:6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75} Nov 16 11:59:14.158004 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:14.157945 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l5trv" event=&{ID:40251af6-202f-415a-aa28-c362b501dbd3 Type:ContainerStarted Data:57035fc21822c56b1677df50b84a40801ec6643b61f910c3ed96ed7ba49c71b8} Nov 16 11:59:14.158695 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:14.158637 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kt88r" event=&{ID:37496343-5356-4736-accc-adc8f05b43c5 Type:ContainerStarted Data:dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669} Nov 16 11:59:14.164368 ropatil1611az-gs6zj-worker-westus-j4ngg polkitd[1774]: Started polkitd version 0.115 Nov 16 11:59:14.201634 ropatil1611az-gs6zj-worker-westus-j4ngg polkitd[1774]: Loading rules from directory /etc/polkit-1/rules.d Nov 16 11:59:14.201723 ropatil1611az-gs6zj-worker-westus-j4ngg polkitd[1774]: Loading rules from directory /usr/share/polkit-1/rules.d Nov 16 11:59:14.202642 ropatil1611az-gs6zj-worker-westus-j4ngg polkitd[1774]: Finished loading, compiling and executing 3 rules Nov 16 11:59:14.203515 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1176]: [system] Successfully activated service 'org.freedesktop.PolicyKit1' Nov 16 11:59:14.203862 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Authorization Manager. Nov 16 11:59:14.203880 ropatil1611az-gs6zj-worker-westus-j4ngg polkitd[1774]: Acquired the name org.freedesktop.PolicyKit1 on the system bus Nov 16 11:59:14.218374 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[1771]: In idle state; will auto-exit in 64 seconds Nov 16 11:59:14.219067 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started rpm-ostree System Management Daemon. Nov 16 11:59:14.268969 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[1771]: client(id:machine-config-operator dbus:1.36 unit:crio-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope uid:0) added; new total=1 Nov 16 11:59:14.285711 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[1771]: client(id:machine-config-operator dbus:1.36 unit:crio-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope uid:0) vanished; remaining=0 Nov 16 11:59:14.285733 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[1771]: In idle state; will auto-exit in 60 seconds Nov 16 11:59:14.324318 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[1771]: client(id:machine-config-operator dbus:1.37 unit:crio-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope uid:0) added; new total=1 Nov 16 11:59:14.342804 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[1771]: client(id:machine-config-operator dbus:1.37 unit:crio-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope uid:0) vanished; remaining=0 Nov 16 11:59:14.342835 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[1771]: In idle state; will auto-exit in 64 seconds Nov 16 11:59:14.397918 ropatil1611az-gs6zj-worker-westus-j4ngg root[1795]: machine-config-daemon[1735]: Starting to manage node: ropatil1611az-gs6zj-worker-westus-j4ngg Nov 16 11:59:14.433830 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[1771]: client(id:machine-config-operator dbus:1.38 unit:crio-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope uid:0) added; new total=1 Nov 16 11:59:14.462190 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[1771]: client(id:machine-config-operator dbus:1.38 unit:crio-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope uid:0) vanished; remaining=0 Nov 16 11:59:14.462215 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[1771]: In idle state; will auto-exit in 63 seconds Nov 16 11:59:14.988240 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:14.988156256Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1288cd9c16ae683a98a48a1dc79f451f0514648b85101f53d94fc0d0c9ca73b5\"" Nov 16 11:59:15.000261 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:15.000114944Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3909f49bf7a6d86ee5d24039d79506226fa6d0a80ac11956ac1dd770be4630d0\"" Nov 16 11:59:15.005018 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:15.004949780Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd6de4896805b834eb7a374038b47ad6db8958bde085753ef1d0e36af4da45fc\"" Nov 16 11:59:15.048810 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:15.048719505Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5ce45f37b20d2dd4b22bf117caf0a2234a9fdd640803269b101c6f1b4e90e60e\"" Nov 16 11:59:15.054347 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:15.054306446Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:14c88d159fb9a50aa6e7c65727eb024281451ea06ad3f48bf934331dfb48f300\"" Nov 16 11:59:15.082169 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:15.082056652Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48432695b88d749dcba50d0e9a1f5072ddf9067355a01b59fe3b0cc917bf8b23\"" Nov 16 11:59:15.082368 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:15.082330954Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bee02fcb1fc5f7fd8127772a514ea08c04eeb1241e41a4ec5fe9fa7307417794\"" Nov 16 11:59:15.123047 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:15.122942855Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4578b08e8c013651a18f58957f777d7fb635959165e9353e6d20f192ae8386dc\"" Nov 16 11:59:15.266323 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:15.266102716Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9\"" Nov 16 11:59:15.583755 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:15.583618 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:59:15.587116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:15.587080 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:15.587218 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:15.587186 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:17.586651 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:17.586539 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:17.586651 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:17.586574 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:17.730215 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:17.730167 1563 secret.go:195] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 16 11:59:17.730215 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:17.730269 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs podName:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 nodeName:}" failed. No retries permitted until 2022-11-16 11:59:25.730249385 +0000 UTC m=+370.667025107 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs") pod "network-metrics-daemon-hr6jx" (UID: "6754c3dd-53dd-4e4f-8d5f-ce077669c3d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 16 11:59:17.730655 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:17.730013 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs\") pod \"network-metrics-daemon-hr6jx\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " pod="openshift-multus/network-metrics-daemon-hr6jx" Nov 16 11:59:17.831604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:17.831550 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-zdfrr\" (UniqueName: \"kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr\") pod \"network-check-target-299n8\" (UID: \"1bd11256-096f-4190-9ee6-f23c6eaabf65\") " pod="openshift-network-diagnostics/network-check-target-299n8" Nov 16 11:59:17.831758 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:17.831741 1563 projected.go:293] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 16 11:59:17.831813 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:17.831765 1563 projected.go:293] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 16 11:59:17.831813 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:17.831781 1563 projected.go:199] Error preparing data for projected volume kube-api-access-zdfrr for pod openshift-network-diagnostics/network-check-target-299n8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 16 11:59:17.831889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:17.831837 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr podName:1bd11256-096f-4190-9ee6-f23c6eaabf65 nodeName:}" failed. No retries permitted until 2022-11-16 11:59:25.831818438 +0000 UTC m=+370.768594060 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zdfrr" (UniqueName: "kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr") pod "network-check-target-299n8" (UID: "1bd11256-096f-4190-9ee6-f23c6eaabf65") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 16 11:59:19.586378 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:19.586312 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:19.586905 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:19.586469 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:20.584763 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:20.584723 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:59:21.587329 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:21.586879 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:21.587329 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:21.586930 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:23.587022 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:23.586931 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:23.587546 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:23.587055 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:25.586298 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:25.586249 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:25.586742 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:25.586452 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:25.586742 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:25.586571 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:59:25.803573 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:25.803452 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs\") pod \"network-metrics-daemon-hr6jx\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " pod="openshift-multus/network-metrics-daemon-hr6jx" Nov 16 11:59:25.803753 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:25.803658 1563 secret.go:195] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 16 11:59:25.803753 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:25.803751 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs podName:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 nodeName:}" failed. No retries permitted until 2022-11-16 11:59:41.803722859 +0000 UTC m=+386.740498481 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs") pod "network-metrics-daemon-hr6jx" (UID: "6754c3dd-53dd-4e4f-8d5f-ce077669c3d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 16 11:59:25.904305 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:25.904164 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-zdfrr\" (UniqueName: \"kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr\") pod \"network-check-target-299n8\" (UID: \"1bd11256-096f-4190-9ee6-f23c6eaabf65\") " pod="openshift-network-diagnostics/network-check-target-299n8" Nov 16 11:59:25.904453 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:25.904339 1563 projected.go:293] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 16 11:59:25.904453 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:25.904361 1563 projected.go:293] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 16 11:59:25.904453 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:25.904374 1563 projected.go:199] Error preparing data for projected volume kube-api-access-zdfrr for pod openshift-network-diagnostics/network-check-target-299n8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 16 11:59:25.904453 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:25.904440 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr podName:1bd11256-096f-4190-9ee6-f23c6eaabf65 nodeName:}" failed. No retries permitted until 2022-11-16 11:59:41.904421708 +0000 UTC m=+386.841197430 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zdfrr" (UniqueName: "kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr") pod "network-check-target-299n8" (UID: "1bd11256-096f-4190-9ee6-f23c6eaabf65") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 16 11:59:26.379859 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.379719641Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4578b08e8c013651a18f58957f777d7fb635959165e9353e6d20f192ae8386dc" id=92eec47f-77ba-4b7e-aa5c-d6f18156fd77 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:26.380899 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.380828049Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4578b08e8c013651a18f58957f777d7fb635959165e9353e6d20f192ae8386dc" id=d27baac3-7f80-4d3e-b82e-a8985942bb38 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.406658 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.406602941Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:815eae0229119ff532d7b1469225cc2e5e4b75914bbbd34b2fb7d66f7e7c8721,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4578b08e8c013651a18f58957f777d7fb635959165e9353e6d20f192ae8386dc],Size_:270148006,Uid:nil,Username:nobody,Spec:nil,},Info:map[string]string{},}" id=d27baac3-7f80-4d3e-b82e-a8985942bb38 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.407772 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.407687849Z" level=info msg="Creating container: openshift-monitoring/node-exporter-dz75k/init-textfile" id=ab299bdd-86a5-4a07-b20f-36ad250b45b4 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:26.407854 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.407816250Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:26.782517 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.782316434Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9" id=083c4537-34af-4a5e-b086-fee6bcb6099c name=/runtime.v1.ImageService/PullImage Nov 16 11:59:26.787715 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.787670974Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9" id=a9fec37b-a54d-47f6-8624-856e81bbd53c name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.838119 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.838063448Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5ce45f37b20d2dd4b22bf117caf0a2234a9fdd640803269b101c6f1b4e90e60e" id=d48b7d19-70d4-4a72-accc-4f2fa741de76 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:26.838977 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.838928755Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5ce45f37b20d2dd4b22bf117caf0a2234a9fdd640803269b101c6f1b4e90e60e" id=3f65bba3-7f87-451d-ab85-07f2a10be441 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.847955 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.847875821Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48432695b88d749dcba50d0e9a1f5072ddf9067355a01b59fe3b0cc917bf8b23" id=bbfa526a-cee7-43f1-b920-14fe43851687 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:26.859560 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.859518808Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48432695b88d749dcba50d0e9a1f5072ddf9067355a01b59fe3b0cc917bf8b23" id=a115cd04-fa2f-4103-9839-606bee1387a0 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.863754 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.863650139Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:cd35189fa8b001e06ada495b7c204aa0d2784a2ea23746527811e83e56f376b9,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9],Size_:296071171,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=a9fec37b-a54d-47f6-8624-856e81bbd53c name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.864150 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.864085742Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:14c88d159fb9a50aa6e7c65727eb024281451ea06ad3f48bf934331dfb48f300" id=3529c2cc-989b-4536-a320-28b4656600be name=/runtime.v1.ImageService/PullImage Nov 16 11:59:26.864624 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.864574745Z" level=info msg="Creating container: openshift-machine-config-operator/machine-config-daemon-kxqnl/oauth-proxy" id=a01313c9-ddab-485c-96f7-6bf7ce1a16d5 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:26.864713 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.864678746Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:26.864713 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.864698246Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:14c88d159fb9a50aa6e7c65727eb024281451ea06ad3f48bf934331dfb48f300" id=ee138dde-bb58-46d9-96d6-66497d9ea69c name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.865881 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.865781554Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1288cd9c16ae683a98a48a1dc79f451f0514648b85101f53d94fc0d0c9ca73b5" id=878e0443-5c25-4081-bcc9-0958af678fce name=/runtime.v1.ImageService/PullImage Nov 16 11:59:26.866263 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.866218158Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:1e8bdf325668974610233452a772cfe2c95bbc12561f6eea20b8afc49ad0f009,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48432695b88d749dcba50d0e9a1f5072ddf9067355a01b59fe3b0cc917bf8b23],Size_:389840947,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=a115cd04-fa2f-4103-9839-606bee1387a0 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.866349 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.866251258Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3909f49bf7a6d86ee5d24039d79506226fa6d0a80ac11956ac1dd770be4630d0" id=986936dd-1e93-49a8-b9cb-67131cd6a483 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:26.866513 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.866338158Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:84a917cb7120bc9a0cba54383a41fdfdd2ca4376de179fcbcde627dc5d33c149,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:14c88d159fb9a50aa6e7c65727eb024281451ea06ad3f48bf934331dfb48f300],Size_:412712206,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=ee138dde-bb58-46d9-96d6-66497d9ea69c name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.866601 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.866388459Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1288cd9c16ae683a98a48a1dc79f451f0514648b85101f53d94fc0d0c9ca73b5" id=8eddbc9f-dbbf-462f-aeb7-b570e43ce242 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.867642 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.867595268Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd6de4896805b834eb7a374038b47ad6db8958bde085753ef1d0e36af4da45fc" id=37244ca1-a987-48aa-b9d9-edf3c5ec2aef name=/runtime.v1.ImageService/PullImage Nov 16 11:59:26.867829 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.867696869Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bee02fcb1fc5f7fd8127772a514ea08c04eeb1241e41a4ec5fe9fa7307417794" id=1798497c-7e47-4e2f-81a4-06cec42bbf37 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:26.867985 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.867827470Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5cb85929a28208909166fee4735af16120e23811dffb0022274663118b07e45f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5ce45f37b20d2dd4b22bf117caf0a2234a9fdd640803269b101c6f1b4e90e60e],Size_:360269060,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=3f65bba3-7f87-451d-ab85-07f2a10be441 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.868159 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.867982671Z" level=info msg="Creating container: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/azure-inject-credentials" id=b5a77a11-8ffd-4309-b459-27a6b2b75374 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:26.868361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.868318073Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd6de4896805b834eb7a374038b47ad6db8958bde085753ef1d0e36af4da45fc" id=d1c60d93-c88e-4942-ac96-0061cfc3639a name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.868447 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.868246073Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:46d6874c0fbc2d49165549a5a3bb17887206e64404f99ce6ee1456a174ac3db2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1288cd9c16ae683a98a48a1dc79f451f0514648b85101f53d94fc0d0c9ca73b5],Size_:425617347,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=8eddbc9f-dbbf-462f-aeb7-b570e43ce242 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.868447 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.868347873Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:26.868511 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.867840270Z" level=info msg="Creating container: openshift-dns/node-resolver-l5trv/dns-node-resolver" id=63c4e7b6-5b01-4d52-9352-e62b33cbc4af name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:26.868542 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.868523675Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:26.868581 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.867995271Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3909f49bf7a6d86ee5d24039d79506226fa6d0a80ac11956ac1dd770be4630d0" id=1c4302f1-ed5e-49fe-b54f-da8d3b593a10 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.869964 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.869930785Z" level=info msg="Creating container: openshift-multus/multus-5wch2/kube-multus" id=7d8057c3-af02-45f5-97fc-b37df8970e2a name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:26.870035 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.869955285Z" level=info msg="Creating container: openshift-image-registry/node-ca-dn4b9/node-ca" id=14f4648a-83c5-4be7-a56e-1ec6f748a6ff name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:26.870138 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.870012086Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:26.870138 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.870096486Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:26.889799 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.889594431Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bee02fcb1fc5f7fd8127772a514ea08c04eeb1241e41a4ec5fe9fa7307417794" id=130f8af6-4cdd-446e-a161-a9b61bb1cd07 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.905745 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb.scope. Nov 16 11:59:26.914403 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.914337515Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:95b3d88543a2c70ed1816d81b829973d2734c522865bede8e041a56f39f7d139,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3909f49bf7a6d86ee5d24039d79506226fa6d0a80ac11956ac1dd770be4630d0],Size_:572880412,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=1c4302f1-ed5e-49fe-b54f-da8d3b593a10 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.914719 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.914673918Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:bedfad2ea8d906de2596641fe6ee379f34c8c227af7b574cdd22aad8304678e8,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bee02fcb1fc5f7fd8127772a514ea08c04eeb1241e41a4ec5fe9fa7307417794],Size_:478691316,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=130f8af6-4cdd-446e-a161-a9b61bb1cd07 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.916061 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.916029228Z" level=info msg="Creating container: openshift-sdn/sdn-pgwdx/sdn" id=4069e078-32ae-4078-8399-69419ecf3034 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:26.916273 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.916248930Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:26.916399 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.916041928Z" level=info msg="Creating container: openshift-cluster-node-tuning-operator/tuned-kt88r/tuned" id=9512e213-1fe4-4afa-84bf-84d4832aa3b9 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:26.916479 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.916438631Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:26.916524 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.916468531Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:3720f81855ccc45979722cc6f910cdeab63288179019d2a270cafd9ee9cea73f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd6de4896805b834eb7a374038b47ad6db8958bde085753ef1d0e36af4da45fc],Size_:377976258,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=d1c60d93-c88e-4942-ac96-0061cfc3639a name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:26.917867 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.917810041Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-q722j/egress-router-binary-copy" id=56a26b0e-5ccd-4f1b-a1e1-79318d46785f name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:26.917949 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.917897942Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:26.920988 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403.scope. Nov 16 11:59:26.925795 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb. Nov 16 11:59:26.942409 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403. Nov 16 11:59:26.963348 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82.scope. Nov 16 11:59:26.969035 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:26.983570 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82. Nov 16 11:59:26.991101 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.991043686Z" level=info msg="Created container 9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb: openshift-monitoring/node-exporter-dz75k/init-textfile" id=ab299bdd-86a5-4a07-b20f-36ad250b45b4 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:26.991620 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:26.991589690Z" level=info msg="Starting container: 9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb" id=71349bb8-944f-4133-b064-1575fcf8ec04 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:27.003785 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:27.003686580Z" level=info msg="Started container" PID=1936 containerID=9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb description=openshift-monitoring/node-exporter-dz75k/init-textfile id=71349bb8-944f-4133-b064-1575fcf8ec04 name=/runtime.v1.RuntimeService/StartContainer sandboxID=502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b Nov 16 11:59:27.018649 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:27.026340 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:27.026267347Z" level=info msg="Created container f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403: openshift-machine-config-operator/machine-config-daemon-kxqnl/oauth-proxy" id=a01313c9-ddab-485c-96f7-6bf7ce1a16d5 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:27.026887 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:27.026849852Z" level=info msg="Starting container: f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403" id=b4b5d633-3224-4b99-a01e-0e59374c7884 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:27.038195 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:27.037875134Z" level=info msg="Started container" PID=1945 containerID=f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403 description=openshift-machine-config-operator/machine-config-daemon-kxqnl/oauth-proxy id=b4b5d633-3224-4b99-a01e-0e59374c7884 name=/runtime.v1.RuntimeService/StartContainer sandboxID=1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c Nov 16 11:59:27.209271 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:27.209213 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dz75k" event=&{ID:5026bf33-68a7-4497-8c41-20a85e407ef9 Type:ContainerStarted Data:9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb} Nov 16 11:59:27.210671 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:27.210646 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" event=&{ID:2e56696b-4c5c-4c9f-8681-c75ba7546700 Type:ContainerStarted Data:f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403} Nov 16 11:59:27.586871 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:27.586828 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:27.587095 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:27.586980 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:27.786591 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:27.786541699Z" level=info msg="Created container 6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/azure-inject-credentials" id=b5a77a11-8ffd-4309-b459-27a6b2b75374 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:27.787143 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:27.787071203Z" level=info msg="Starting container: 6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82" id=b22c19db-06f1-4f2b-b1cc-47ff237cd99a name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:27.798805 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:27.798763490Z" level=info msg="Started container" PID=1977 containerID=6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82 description=openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/azure-inject-credentials id=b22c19db-06f1-4f2b-b1cc-47ff237cd99a name=/runtime.v1.RuntimeService/StartContainer sandboxID=8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883 Nov 16 11:59:27.803483 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb.scope: Succeeded. Nov 16 11:59:27.804167 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb.scope: Consumed 172ms CPU time Nov 16 11:59:27.804444 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb.scope: Succeeded. Nov 16 11:59:27.805063 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb.scope: Consumed 31ms CPU time Nov 16 11:59:27.808018 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82.scope: Succeeded. Nov 16 11:59:27.808663 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82.scope: Consumed 34ms CPU time Nov 16 11:59:27.812160 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82.scope: Succeeded. Nov 16 11:59:27.812724 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82.scope: Consumed 27ms CPU time Nov 16 11:59:28.215262 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:28.215168 1563 generic.go:296] "Generic (PLEG): container finished" podID=89deb745-c538-46ae-9d77-26643bf01a15 containerID="6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82" exitCode=0 Nov 16 11:59:28.215262 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:28.215237 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" event=&{ID:89deb745-c538-46ae-9d77-26643bf01a15 Type:ContainerDied Data:6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82} Nov 16 11:59:28.216056 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.216008696Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:94efd22fba0607fd214d5e5aa65cc90e5e6f4dd0b41dde42c6a5c9ed80e5b8c3" id=dce4f082-b7c3-4803-b59b-504abf01e4be name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:28.216334 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.216298798Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:94efd22fba0607fd214d5e5aa65cc90e5e6f4dd0b41dde42c6a5c9ed80e5b8c3 not found" id=dce4f082-b7c3-4803-b59b-504abf01e4be name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:28.216558 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:28.216533 1563 generic.go:296] "Generic (PLEG): container finished" podID=5026bf33-68a7-4497-8c41-20a85e407ef9 containerID="9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb" exitCode=0 Nov 16 11:59:28.216642 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:28.216613 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dz75k" event=&{ID:5026bf33-68a7-4497-8c41-20a85e407ef9 Type:ContainerDied Data:9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb} Nov 16 11:59:28.217062 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.216996204Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:94efd22fba0607fd214d5e5aa65cc90e5e6f4dd0b41dde42c6a5c9ed80e5b8c3" id=3cdad600-07c6-432c-8ec1-81dc31bc9761 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:28.217192 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.217118305Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4578b08e8c013651a18f58957f777d7fb635959165e9353e6d20f192ae8386dc" id=6ff226b3-dd19-4020-9a1a-23d97178a680 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:28.217565 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.217512208Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:815eae0229119ff532d7b1469225cc2e5e4b75914bbbd34b2fb7d66f7e7c8721,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4578b08e8c013651a18f58957f777d7fb635959165e9353e6d20f192ae8386dc],Size_:270148006,Uid:nil,Username:nobody,Spec:nil,},Info:map[string]string{},}" id=6ff226b3-dd19-4020-9a1a-23d97178a680 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:28.230398 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.230360903Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:94efd22fba0607fd214d5e5aa65cc90e5e6f4dd0b41dde42c6a5c9ed80e5b8c3\"" Nov 16 11:59:28.241794 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.241740788Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4578b08e8c013651a18f58957f777d7fb635959165e9353e6d20f192ae8386dc" id=1704c216-7805-4b92-b793-f406cd13e762 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:28.241992 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.241942990Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:815eae0229119ff532d7b1469225cc2e5e4b75914bbbd34b2fb7d66f7e7c8721,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4578b08e8c013651a18f58957f777d7fb635959165e9353e6d20f192ae8386dc],Size_:270148006,Uid:nil,Username:nobody,Spec:nil,},Info:map[string]string{},}" id=1704c216-7805-4b92-b793-f406cd13e762 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:28.243031 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.242960497Z" level=info msg="Creating container: openshift-monitoring/node-exporter-dz75k/node-exporter" id=8caeddd6-5201-4a5f-9e4e-78e608180b01 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:28.243120 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.243068798Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:28.286604 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228.scope. Nov 16 11:59:28.323059 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2.scope. Nov 16 11:59:28.324606 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228. Nov 16 11:59:28.334271 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf.scope. Nov 16 11:59:28.338152 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6.scope. Nov 16 11:59:28.341762 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a.scope. Nov 16 11:59:28.374705 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf. Nov 16 11:59:28.381425 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2. Nov 16 11:59:28.383060 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a. Nov 16 11:59:28.386103 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6. Nov 16 11:59:28.435648 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.435602832Z" level=info msg="Created container 91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228: openshift-dns/node-resolver-l5trv/dns-node-resolver" id=63c4e7b6-5b01-4d52-9352-e62b33cbc4af name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:28.436254 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.436216237Z" level=info msg="Starting container: 91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228" id=5d00ff1d-affa-4f17-94f6-e7dd2f4f0d7b name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:28.447270 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.447222619Z" level=info msg="Started container" PID=2160 containerID=91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228 description=openshift-dns/node-resolver-l5trv/dns-node-resolver id=5d00ff1d-affa-4f17-94f6-e7dd2f4f0d7b name=/runtime.v1.RuntimeService/StartContainer sandboxID=57035fc21822c56b1677df50b84a40801ec6643b61f910c3ed96ed7ba49c71b8 Nov 16 11:59:28.450365 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.450325542Z" level=info msg="Created container 9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2: openshift-multus/multus-5wch2/kube-multus" id=7d8057c3-af02-45f5-97fc-b37df8970e2a name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:28.450900 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.450858246Z" level=info msg="Starting container: 9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2" id=b1a198b2-8ab7-4fac-9ce4-4102bfcb07f3 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:28.455012 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.454962277Z" level=info msg="Created container 9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf: openshift-cluster-node-tuning-operator/tuned-kt88r/tuned" id=9512e213-1fe4-4afa-84bf-84d4832aa3b9 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:28.455152 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.454951777Z" level=info msg="Created container 962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a: openshift-sdn/sdn-pgwdx/sdn" id=4069e078-32ae-4078-8399-69419ecf3034 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:28.455429 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.455390380Z" level=info msg="Starting container: 962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a" id=03ba6761-caec-4d14-b99f-2101048217f3 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:28.455514 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.455483581Z" level=info msg="Starting container: 9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf" id=bba80ad8-537d-4364-ba84-78a697410a80 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:28.460600 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.460551718Z" level=info msg="Created container 131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6: openshift-image-registry/node-ca-dn4b9/node-ca" id=14f4648a-83c5-4be7-a56e-1ec6f748a6ff name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:28.465796 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.463907243Z" level=info msg="Starting container: 131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6" id=86d86caa-4fc2-4063-b823-dc753d4ee419 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:28.464373 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470.scope. Nov 16 11:59:28.466061 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.465932158Z" level=info msg="Started container" PID=2223 containerID=9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2 description=openshift-multus/multus-5wch2/kube-multus id=b1a198b2-8ab7-4fac-9ce4-4102bfcb07f3 name=/runtime.v1.RuntimeService/StartContainer sandboxID=31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427 Nov 16 11:59:28.467823 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.467766872Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": CREATE" Nov 16 11:59:28.467963 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.467943373Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.468067 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.468050074Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.468218 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.468201075Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.468335 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.468318576Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.468468 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.468451577Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.468583 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.468567078Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.468768 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.468749179Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.468880 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.468863680Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.469012 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.468996281Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.469108 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.469092282Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.469247 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.469230983Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.469353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.469337184Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.469532 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.469513785Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.469643 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.469625986Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.469752 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.469737787Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.469879 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.469862288Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.470004 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.469988289Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.470098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.470083289Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.470251 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.470224990Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.470340 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.470323991Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.470471 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.470454592Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.470582 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.470565193Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.470699 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.470683294Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.470832 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.470814495Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.470971 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.470954396Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.471085 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.471068397Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.471242 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.471226298Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.471399 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.471372099Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.471526 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.471510800Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.471633 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.471617301Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.471763 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.471747702Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.471866 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.471850202Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.472005 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.471988803Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.472107 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.472091304Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.472237 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.472221605Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.472345 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.472328406Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.472493 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.472474407Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.472836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.472795709Z" level=info msg="Started container" PID=2236 containerID=962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a description=openshift-sdn/sdn-pgwdx/sdn id=03ba6761-caec-4d14-b99f-2101048217f3 name=/runtime.v1.RuntimeService/StartContainer sandboxID=6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75 Nov 16 11:59:28.473389 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.473361414Z" level=info msg="Started container" PID=2220 containerID=9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf description=openshift-cluster-node-tuning-operator/tuned-kt88r/tuned id=bba80ad8-537d-4364-ba84-78a697410a80 name=/runtime.v1.RuntimeService/StartContainer sandboxID=dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669 Nov 16 11:59:28.475238 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.475215928Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.475350 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.475333728Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.475447 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.475429829Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.475527 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.475513130Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.475664 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.475645031Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.475780 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.475765532Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.475879 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.475863532Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.475985 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.475969633Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.476108 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.476091434Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.476344 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.476326336Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.476453 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.476436537Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.476593 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.476577038Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.476696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.476680038Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.476847 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.476820239Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.477423 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.476943940Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.477552 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.477532645Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.477694 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.477673646Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.477818 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.477801947Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.477990 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.477972448Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.478121 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.478105949Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.478386 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.478367751Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.478510 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.478494952Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.478609 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.478591853Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.478730 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.478714154Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.478844 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.478827554Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.478967 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.478949655Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.479074 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.479057756Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.479260 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.479240458Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.479364 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.479347758Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.479472 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.479457759Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.479594 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.479577560Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": CREATE" Nov 16 11:59:28.479694 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.479679661Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.479806 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.479789762Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.479914 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.479898562Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.480039 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.480021363Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.480201 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.480185265Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.480297 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.480281165Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.480397 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.480382866Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.480499 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.480483067Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.480600 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.480585868Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.480706 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.480691068Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.480804 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.480789569Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.480933 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.480915770Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.481363 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.481029171Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.481493 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.481472574Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.481603 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.481587475Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.481710 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.481693676Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.481832 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.481817177Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.481943 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.481925478Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.482074 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.482058179Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.482196 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.482178479Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.482260 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28.scope. Nov 16 11:59:28.482474 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.482457081Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.482825 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.482802084Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.482958 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.482942885Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.483088 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.482211480Z" level=info msg="Started container" PID=2237 containerID=131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6 description=openshift-image-registry/node-ca-dn4b9/node-ca id=86d86caa-4fc2-4063-b823-dc753d4ee419 name=/runtime.v1.RuntimeService/StartContainer sandboxID=e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e Nov 16 11:59:28.483353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.483060486Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.483478 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.483460289Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.483581 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.483565590Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.483670 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.483653990Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.483783 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.483766791Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.483874 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.483860292Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.483937 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.483923592Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.484014 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.484001293Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.484103 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.484089194Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.484238 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.484223895Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.484320 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.484305995Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.484408 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.484394496Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.484495 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.484466796Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.484601 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.484586497Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.484673 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.484658298Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.484847 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.484828699Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.485045 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.485024601Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.485929 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.485903907Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.486087 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.486069108Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.486235 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.486217809Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.486367 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.486338310Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.486469 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.486453511Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.486594 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.486577912Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.486706 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.486691113Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.486802 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.486786914Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.486937 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.486922615Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.487043 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.487027316Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.487202 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.487185617Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.487363 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.487335118Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.487496 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.487480619Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.487607 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.487591620Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.490339 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.490317840Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.490500 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.490464341Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.490653 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.490635442Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.490771 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.490753543Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.490913 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.490898044Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.491313 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.491290347Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.491466 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.491450948Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.491569 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.491553149Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.491730 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.491712950Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.491844 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.491827451Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.492016 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.491976152Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.492250 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.492230654Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.492410 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.492394056Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.492519 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.492502256Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.492635 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.492619457Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.492733 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.492717558Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.493254 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.492827359Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.493474 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.493451763Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.493603 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.493586664Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.493705 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.493688765Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.493830 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.493814366Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.493926 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.493910067Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.494096 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.494066968Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.494197 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.494181269Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.494318 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.494302370Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.494433 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.494415871Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.494552 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.494537371Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.494645 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.494629272Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.495036 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.495014375Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.495233 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.495213277Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.495402 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.495384578Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.495523 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.495504579Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.495634 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.495618980Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.495744 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.495727180Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.495881 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.495865481Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.495985 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.495969582Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.496201 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.496098383Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.496388 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.496368885Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.496556 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.496536386Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.496678 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.496662387Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.496816 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.496799788Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.496915 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.496898989Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.497040 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.497024090Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.497175 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.497138591Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.497291 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.497275892Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.497403 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.497386093Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.497583 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.497548394Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.497699 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.497684095Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.497833 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.497817996Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.497931 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.497915897Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.498068 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.498050398Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.498259 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.498241599Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.498426 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.498409000Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.498529 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.498513301Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.498668 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.498651402Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.498768 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.498751903Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.498893 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.498877504Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.499002 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.498986105Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.499144 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.499114906Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.499232 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.499216706Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.499353 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470. Nov 16 11:59:28.499510 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.499352707Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.499601 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.499584509Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.499747 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.499730210Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.500238 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.500180414Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.502681 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.502659532Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.502832 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.502813233Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.502993 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.502975834Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.503093 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503077035Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.503246 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503227836Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.503379 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503363037Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.503537 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503520238Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.503637 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503621939Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.503783 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503754940Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.503783 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503783640Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.503904 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503813741Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.503904 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503837141Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.503904 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503854641Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.503904 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503865341Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.503904 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503882141Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.503904 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503899541Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.504063 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503916141Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504063 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503928841Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.504063 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503944942Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504063 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503962042Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.504063 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503978442Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504063 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.503987642Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.504063 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504003942Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504063 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504019942Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.504063 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504037542Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504063 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504047142Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.504063 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504063342Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504079843Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504096543Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504105743Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504121343Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504154543Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504173743Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504183143Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504198343Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504214544Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504230544Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504239544Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504254844Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504270344Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504291244Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504303244Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504319744Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504335544Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504352245Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504361645Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504379445Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504395845Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504412245Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504422145Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504438045Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504454045Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504470645Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504480346Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504496046Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504512346Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504529846Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504539246Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504555046Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504571046Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504587846Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504597246Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504613547Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504629147Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.504803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504645347Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504656447Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504672847Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504689247Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504705747Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504715147Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504730847Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504746748Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504766148Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504775748Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504791948Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504808348Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504826148Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.504904549Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505085950Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505110850Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505185751Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505209551Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505261451Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505279252Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505327852Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505346952Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505365952Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505401352Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505422953Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505443653Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505481553Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505500453Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505520153Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505562354Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505584054Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505593954Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505623454Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505663554Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505707755Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505720755Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505740955Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505762055Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.505822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505780855Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505791455Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505808155Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505824956Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505842556Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505852656Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505870156Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505887256Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505904556Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505902056Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=571d273a-8055-44b8-ac7d-23b0a554dcb6 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505914656Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505973857Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.505986257Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506002957Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506020457Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506037357Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506047357Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506063357Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506079257Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506095558Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506106158Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506123758Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506174258Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506208758Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506219559Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506237659Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506253459Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506269659Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506296259Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506313359Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506323359Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506339159Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506355660Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506373460Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506382960Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506399360Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506415460Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.506732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506434060Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506444560Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506300459Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f not found" id=571d273a-8055-44b8-ac7d-23b0a554dcb6 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506471760Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506532261Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506563761Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506575561Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506593361Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506626862Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506646962Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506662362Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506679762Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506692162Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506710662Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506730062Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506747362Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506757063Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506809063Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506834463Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506854363Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506864763Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506882363Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506898764Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506915564Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506925164Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506942664Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506958964Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506975564Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.506985064Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507001164Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507031565Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507071665Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507086065Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507104265Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507123565Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507168766Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.507762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507145265Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=d66ff06c-5be5-49ec-a5f9-4b43a6044579 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507179666Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507285166Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507298167Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507315067Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507334367Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507351467Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507361967Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507379067Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507397067Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507415567Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507425167Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507441168Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507461968Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507494368Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507516168Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507536168Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507554368Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507574469Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507595869Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507613169Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507623069Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507638769Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507656469Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507673269Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507690969Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507709170Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507726870Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507743470Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507753470Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507768970Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507778170Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507793870Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507823370Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507864571Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507887371Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507910571Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507930571Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.508679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507949071Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507959271Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.507975872Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508018672Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508040172Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508058572Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508076672Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508099873Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508118373Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508147973Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508169373Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508187873Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508205073Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508214873Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508230673Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508248774Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508265474Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508276474Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508292574Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508308774Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508325174Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508335874Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508352174Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508481375Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508533776Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508583476Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508603476Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508621776Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508639477Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508649577Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508664977Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508675577Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508691177Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508701477Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508717777Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508734977Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508760877Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508789778Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.509836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508811178Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508822678Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508838878Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508869378Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508912079Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508948179Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.508968179Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509008179Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509113880Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509190581Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509217781Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509241581Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509259581Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509276481Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509294981Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509313082Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509330482Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509340582Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509357082Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509374782Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509392682Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509402382Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509419582Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509435682Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509452383Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509463083Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509480183Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509496983Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509516183Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509526183Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509542683Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509579284Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509617684Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509631384Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509638084Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f\"" Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509649184Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509708385Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.510696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509746685Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509768585Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509786685Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509803385Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509821385Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509838985Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509856486Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509867486Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509883786Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509900186Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509919986Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509937386Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509954186Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509963486Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509980487Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.509991087Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510006687Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510024587Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510068387Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510084087Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510102987Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510121288Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510731492Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510749792Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510774192Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510795993Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510820293Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510829693Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510845093Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510889193Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510908993Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510918894Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510937194Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510955694Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510973694Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.510994394Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511013594Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511035994Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.511655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511078595Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511108595Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511142895Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511167195Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511186296Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511196096Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511211796Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511243996Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511260496Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511269696Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511286696Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511319197Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511351597Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511364497Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511380797Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511397797Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511415997Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511426097Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": WRITE" Nov 16 11:59:28.512098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.511441497Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.523759 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28. Nov 16 11:59:28.536372 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:28.550613 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:28.594614 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.594495516Z" level=info msg="CNI monitoring event \"/etc/kubernetes/cni/net.d/multus.d\": CREATE" Nov 16 11:59:28.594614 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.594617317Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.596798 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.596743733Z" level=info msg="Created container 4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470: openshift-multus/multus-additional-cni-plugins-q722j/egress-router-binary-copy" id=56a26b0e-5ccd-4f1b-a1e1-79318d46785f name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:28.597195 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.597158736Z" level=info msg="Starting container: 4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470" id=450677fb-78a5-4450-8ff4-ed369d565e62 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:28.608075 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.608032817Z" level=info msg="Started container" PID=2406 containerID=4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470 description=openshift-multus/multus-additional-cni-plugins-q722j/egress-router-binary-copy id=450677fb-78a5-4450-8ff4-ed369d565e62 name=/runtime.v1.RuntimeService/StartContainer sandboxID=8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3 Nov 16 11:59:28.611383 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611348242Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": CREATE" Nov 16 11:59:28.611444 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611424142Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.611478 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611445642Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.611478 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611471043Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.611589 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611556143Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.611629 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611613144Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.611695 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611637244Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.611726 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611694644Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.611771 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611753045Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.611824 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611802545Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.611878 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611846745Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.611878 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611871546Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.611989 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611930346Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.611989 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.611962746Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.612043 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612008547Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.612043 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612036947Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.612105 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612087847Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.612154 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612119548Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.612222 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612201348Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.612261 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612235448Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.612291 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612270649Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.612336 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612297449Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.612431 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612395550Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.612431 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612428450Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.612509 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612477050Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.612509 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612478650Z" level=info msg="Created container aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28: openshift-monitoring/node-exporter-dz75k/node-exporter" id=8caeddd6-5201-4a5f-9e4e-78e608180b01 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:28.612509 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612503650Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.612622 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612596451Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.612694 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612639551Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.613065 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612943354Z" level=info msg="Starting container: aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28" id=5f3d2105-b0c5-4aca-8f75-9df7be3bf308 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:28.613065 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.612996454Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.613065 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613044754Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.613065 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613060655Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.613229 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613082755Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.613229 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613157755Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.613229 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613206256Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.613229 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613224556Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.613330 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613247556Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.613330 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613259056Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.613330 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613278456Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.613420 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613339057Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.613420 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613379257Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.613491 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613423557Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.613491 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613451357Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.613587 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613559858Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.613631 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613604759Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.613631 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613617259Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.613691 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613636959Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.613691 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613677559Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.613745 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613711959Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.613776 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613761860Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.613806 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613789660Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.613856 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613836860Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.613900 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613867761Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.613931 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613920861Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.613963 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.613946861Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.614022 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614003362Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.614065 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614033362Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.614096 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614087862Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.614140 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614113462Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.614224 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614196163Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.614270 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614255763Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.614301 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614277664Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.614329 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614311664Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.614362 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614351264Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.614394 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614381864Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.614465 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614445165Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.614502 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614476465Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.614531 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614520665Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.614567 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614545966Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.614636 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614613866Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.614679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614651166Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.614743 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614708567Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.614781 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614758467Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.614816 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614780867Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.614816 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614803668Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.614929 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614903968Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.615025 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.614986669Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.615068 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615030769Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.615068 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615064969Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.615429 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615108470Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.615470 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615451672Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.615513 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615471672Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.615513 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615494173Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.615573 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615523573Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.615573 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615547173Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.615573 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615557073Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.615666 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615575673Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.615732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615702074Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.615865 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615831175Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.615903 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615864975Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.615903 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615899076Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.615961 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615944076Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.615993 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.615971476Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.616073 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616051177Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.616114 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616094577Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.616174 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616159478Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.616213 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616193978Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.616273 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616252378Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.616333 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616294379Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.616333 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616324279Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.616481 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616431880Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.616523 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616501380Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.616579 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616554881Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.616579 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616574081Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.616646 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616596581Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.616718 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616688382Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.616758 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616744182Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.616789 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616767982Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.616818 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616794282Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.616881 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616854783Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.616962 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616891083Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.616962 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616955384Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.617022 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.616982184Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.617114 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617085085Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.617183 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617145285Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.617183 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617166085Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.617235 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617191485Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.617264 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617248086Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.617306 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617292186Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.617342 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617313986Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.617377 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617340186Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.617407 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617395587Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.617435 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617426587Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.617464 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617445587Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.617493 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617467487Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.617573 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617553288Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.617610 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617584788Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.617714 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617694189Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.617752 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617729289Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.617752 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617747289Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.617812 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617768890Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.617843 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617825190Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.617872 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617855490Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.617906 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617874190Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.617906 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617896391Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.617976 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617954691Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.618018 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.617987691Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.618064 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618045192Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.618108 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618075992Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.618155 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618138492Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.618187 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618166893Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.618267 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618239493Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.618310 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618289193Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.618340 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618309594Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.618340 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618331894Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.618439 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618410194Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.618476 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618463495Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.618506 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618485795Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.618597 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618511695Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.618597 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618570496Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.618672 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618602096Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.618672 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618649396Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.618722 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618674896Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.618786 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618753797Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.618824 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618813297Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.618860 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618835798Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.618890 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618860698Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.618944 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618918098Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.618980 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618966699Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.619032 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.618987799Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.619032 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619026999Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.619155 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619108700Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.619223 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619166500Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.619223 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619188300Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.619223 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619212900Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.619296 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619276701Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.619325 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619303501Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.619380 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619358401Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.619424 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619392102Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.619470 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619452502Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.619510 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619482502Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.619559 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619539603Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.619595 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619570403Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.619646 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619627103Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.619688 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619657304Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.619721 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619704104Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.619756 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619732604Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.619808 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619788805Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.619849 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619818905Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.619881 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619866205Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.619914 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.619892205Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.620067 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620040607Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.620119 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620084907Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.620179 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620157507Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.620213 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620187508Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.620253 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620234408Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.620289 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620264708Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.620339 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620319609Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.620377 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620350909Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.620447 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620422909Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.620493 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620462010Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.620525 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620510410Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.620558 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620536610Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.620611 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620589511Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.620707 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620621211Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.620747 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620711712Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.620782 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620764312Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.620816 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620783412Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.620848 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620817512Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.620878 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620855813Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.620907 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620884413Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.620984 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.620959813Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.621041 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621000014Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.621079 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621050114Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.621114 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621080214Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.621157 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621136315Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.621188 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621166415Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.621243 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621217515Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.621408 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621252116Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.621408 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621373716Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.621476 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621424117Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.621476 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621446517Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.621476 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621471317Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.621559 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621537618Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.621588 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621568618Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.621588 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621579618Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.621637 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621599918Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.621779 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621748519Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.621850 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621802820Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.621850 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621827120Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.621934 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621851920Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.621934 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621911520Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.621987 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621948321Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.621987 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621966921Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.622040 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.621988221Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.622069 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622051222Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.622102 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622084522Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.622154 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622143022Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.622187 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622171622Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.622245 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622227323Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.622394 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622361224Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.622523 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622504925Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.622612 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622597126Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.622727 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622702426Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.622778 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622745127Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.622824 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622782827Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.622824 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622822127Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.622878 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622856628Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.622938 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622883728Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.622995 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.622973328Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.623041 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623008929Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.623200 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623178930Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.623300 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623284931Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.623380 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623364031Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.623490 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623459732Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.623490 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623491032Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.623591 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623518032Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.623639 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623580533Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.623639 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623630133Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.623695 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623653633Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.623695 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623680534Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.623755 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623738234Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.623790 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623771634Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.624064 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.623977436Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.624064 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624036336Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.624064 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624060036Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.624216 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624085037Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.624248 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624222238Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.624279 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624254838Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.624279 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624274838Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.624332 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624299038Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.624371 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624346339Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.624411 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624389139Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.624469 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624436939Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.624507 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624495840Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.624536 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624520040Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.624565 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624542940Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.624617 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624597140Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.624694 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624646041Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.624767 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624746742Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.624804 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624795442Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.624834 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624823442Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.624863 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624847442Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.624941 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624921943Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.624977 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.624954243Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.625140 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625100444Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.625179 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625165945Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.625208 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625181745Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.625246 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625205045Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.625313 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625291046Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.625355 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625326346Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.625355 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625348246Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.625415 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625372646Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.625449 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625432947Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.625494 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625475447Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.625531 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625497647Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.625531 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625523147Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.625604 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625579948Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.625641 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625616948Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.625641 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625636248Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.628009 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.625739849Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.628091 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628023466Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.628091 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628068566Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.628170 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628121167Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.628212 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628196267Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.628257 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628216067Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.628257 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628240768Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.628369 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628339868Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.628442 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628389769Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.628442 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628411169Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.628442 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628436869Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.628541 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628515470Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.628580 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628557370Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.628580 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628576870Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.628644 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628599670Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.628692 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628670171Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.628730 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628700871Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.628771 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628754571Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.628806 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628784672Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.628894 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628862172Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.628938 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628920273Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.628970 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628942973Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.629008 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.628969573Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.629037 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629009973Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.629073 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629054774Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.629150 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629104374Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.629193 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629153774Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.629238 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629205675Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.629238 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629234575Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.629302 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629283275Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.629400 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629313576Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.629400 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629392876Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.629469 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629457477Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.629501 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629479577Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.629530 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629517777Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.629578 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629557677Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.629633 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629589078Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.629671 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629644478Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.629706 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629671778Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.629737 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629724879Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.629782 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629765879Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.629897 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629876580Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.629941 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629918380Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.629941 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629932380Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.629993 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629952680Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.630022 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.629990481Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.630022 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630016381Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.630078 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630008581Z" level=info msg="Started container" PID=2440 containerID=aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28 description=openshift-monitoring/node-exporter-dz75k/node-exporter id=5f3d2105-b0c5-4aca-8f75-9df7be3bf308 name=/runtime.v1.RuntimeService/StartContainer sandboxID=502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b Nov 16 11:59:28.630205 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630105182Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.630205 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630169282Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.630205 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630185882Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.630294 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630208982Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.630294 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630281483Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.630345 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630311883Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.630381 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630355483Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.630416 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630385984Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.630477 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630448984Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.630518 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630500884Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.630548 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630522985Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.630577 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630547885Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.630656 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630627485Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.630693 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630680286Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.630730 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630703786Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.630769 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630728786Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.630823 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630799587Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.630861 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630837987Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.630891 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630860987Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.630891 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630883887Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.630983 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630962588Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.631027 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.630995088Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.631059 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631046789Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.631094 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631073189Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.631169 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631148889Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.631219 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631183690Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.631257 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631239190Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.631291 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631266490Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.631347 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631328391Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.631393 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631359091Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.631456 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631417491Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.631524 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631466892Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.631576 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631548592Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.631613 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631592393Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.631678 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631659193Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.631733 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631692093Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.631770 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631754094Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.631800 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631779894Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.631860 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631840794Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.631907 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631872095Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.631940 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631929795Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.631970 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.631955395Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.632036 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632016896Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.632076 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632047296Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.632233 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632136297Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.632233 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632173797Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.632320 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632238897Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.632320 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632267098Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.632370 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632347898Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.632416 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632385299Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.632450 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632427599Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.632489 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632467699Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.632551 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632524900Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.632653 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632573600Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.632653 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632639200Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.632709 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632669101Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.632746 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632730301Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.632782 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632757101Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.632840 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632818402Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.632880 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632852802Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.632933 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632912202Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.632986 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.632945903Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.633072 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.633052303Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": WRITE" Nov 16 11:59:28.633113 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.633088904Z" level=info msg="Updated default CNI network name to " Nov 16 11:59:28.649882 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.649832228Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=08e4238c-2ef6-4106-abbf-ae557540cc3e name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:28.650038 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470.scope: Succeeded. Nov 16 11:59:28.650165 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.650144431Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f not found" id=08e4238c-2ef6-4106-abbf-ae557540cc3e name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:28.650742 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470.scope: Consumed 52ms CPU time Nov 16 11:59:28.651555 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.651517941Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=37658c06-c4eb-4df1-8023-478395d39fd3 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:28.654563 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:28.654337562Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f\"" Nov 16 11:59:28.674939 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470.scope: Succeeded. Nov 16 11:59:28.675644 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470.scope: Consumed 31ms CPU time Nov 16 11:59:29.070573 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00008|memory|INFO|143764 kB peak resident set size after 376.8 seconds Nov 16 11:59:29.071382 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599969.0709] manager: (br0): new Open vSwitch Interface device (/org/freedesktop/NetworkManager/Devices/3) Nov 16 11:59:29.072105 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599969.0720] manager: (br0): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/4) Nov 16 11:59:29.072979 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[2654]: Using default interface naming scheme 'rhel-8.0'. Nov 16 11:59:29.072980 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599969.0729] manager: (br0): new Open vSwitch Bridge device (/org/freedesktop/NetworkManager/Devices/5) Nov 16 11:59:29.077828 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[2654]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:59:29.077925 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[2654]: Could not generate persistent MAC address for ovs-system: No such file or directory Nov 16 11:59:29.078145 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device ovs-system entered promiscuous mode Nov 16 11:59:29.078237 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00009|dpif_netlink|INFO|Datapath dispatch mode: per-vport Nov 16 11:59:29.078595 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00010|ofproto_dpif|INFO|system@ovs-system: Datapath supports recirculation Nov 16 11:59:29.078667 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00011|ofproto_dpif|INFO|system@ovs-system: VLAN header stack length probed as 2 Nov 16 11:59:29.078742 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00012|ofproto_dpif|INFO|system@ovs-system: MPLS label stack length probed as 3 Nov 16 11:59:29.078777 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00013|ofproto_dpif|INFO|system@ovs-system: Datapath supports truncate action Nov 16 11:59:29.078812 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00014|ofproto_dpif|INFO|system@ovs-system: Datapath supports unique flow ids Nov 16 11:59:29.078833 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00015|ofproto_dpif|INFO|system@ovs-system: Datapath supports clone action Nov 16 11:59:29.078987 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00016|ofproto_dpif|INFO|system@ovs-system: Max sample nesting level probed as 10 Nov 16 11:59:29.079047 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00017|ofproto_dpif|INFO|system@ovs-system: Datapath supports eventmask in conntrack action Nov 16 11:59:29.079082 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00018|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_clear action Nov 16 11:59:29.079103 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00019|ofproto_dpif|INFO|system@ovs-system: Max dp_hash algorithm probed to be 0 Nov 16 11:59:29.079185 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00020|ofproto_dpif|INFO|system@ovs-system: Datapath supports check_pkt_len action Nov 16 11:59:29.090073 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: Timeout policy base is empty Nov 16 11:59:29.090175 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: Failed to associated timeout policy `ovs_test_tp' Nov 16 11:59:29.090053 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00021|ofproto_dpif|INFO|system@ovs-system: Datapath supports timeout policy in conntrack action Nov 16 11:59:29.090073 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00022|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_zero_snat Nov 16 11:59:29.090113 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00023|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_state Nov 16 11:59:29.090166 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00024|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_zone Nov 16 11:59:29.090194 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00025|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_mark Nov 16 11:59:29.090228 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00026|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_label Nov 16 11:59:29.090278 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00027|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_state_nat Nov 16 11:59:29.090313 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00028|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_orig_tuple Nov 16 11:59:29.090351 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00029|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_orig_tuple6 Nov 16 11:59:29.090374 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00030|ofproto_dpif|INFO|system@ovs-system: Datapath does not support IPv6 ND Extensions Nov 16 11:59:29.090533 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00031|ofproto_dpif_upcall|INFO|Setting n-handler-threads to 5, setting n-revalidator-threads to 3 Nov 16 11:59:29.090550 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00032|ofproto_dpif_upcall|INFO|Starting 8 threads Nov 16 11:59:29.109631 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[2669]: Using default interface naming scheme 'rhel-8.0'. Nov 16 11:59:29.112740 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[2669]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:59:29.112747 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00033|bridge|INFO|bridge br0: added interface br0 on port 65534 Nov 16 11:59:29.112836 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[2669]: Could not generate persistent MAC address for br0: No such file or directory Nov 16 11:59:29.112913 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00034|bridge|INFO|bridge br0: using datapath ID 0000b639e53e0640 Nov 16 11:59:29.113109 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00035|connmgr|INFO|br0: added service controller "punix:/var/run/openvswitch/br0.mgmt" Nov 16 11:59:29.113189 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device br0 entered promiscuous mode Nov 16 11:59:29.136527 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599969.1365] manager: (vxlan0): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/6) Nov 16 11:59:29.148805 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599969.1487] manager: (vxlan_sys_4789): new Vxlan device (/org/freedesktop/NetworkManager/Devices/7) Nov 16 11:59:29.149123 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[2678]: Using default interface naming scheme 'rhel-8.0'. Nov 16 11:59:29.154286 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00036|bridge|INFO|bridge br0: added interface vxlan0 on port 1 Nov 16 11:59:29.154400 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[2678]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:59:29.154502 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[2678]: Could not generate persistent MAC address for vxlan_sys_4789: No such file or directory Nov 16 11:59:29.155204 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vxlan_sys_4789 entered promiscuous mode Nov 16 11:59:29.162910 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599969.1628] device (vxlan_sys_4789): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external') Nov 16 11:59:29.164142 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599969.1641] device (vxlan_sys_4789): state change: unavailable -> disconnected (reason 'none', sys-iface-state: 'external') Nov 16 11:59:29.181536 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599969.1815] manager: (tun0): new Open vSwitch Interface device (/org/freedesktop/NetworkManager/Devices/8) Nov 16 11:59:29.182199 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599969.1821] manager: (tun0): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/9) Nov 16 11:59:29.185090 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00037|netdev|WARN|failed to set MTU for network device tun0: No such device Nov 16 11:59:29.186902 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[2686]: Using default interface naming scheme 'rhel-8.0'. Nov 16 11:59:29.190947 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[2686]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:59:29.191042 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[2686]: Could not generate persistent MAC address for tun0: No such file or directory Nov 16 11:59:29.190976 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00038|bridge|INFO|bridge br0: added interface tun0 on port 2 Nov 16 11:59:29.191176 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device tun0 entered promiscuous mode Nov 16 11:59:29.219702 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:29.219629 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5wch2" event=&{ID:d38704ca-571d-40ee-84b3-71c1d45d56f4 Type:ContainerStarted Data:9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2} Nov 16 11:59:29.221235 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:29.221195 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-sdn/sdn-pgwdx" event=&{ID:45cd7283-fcc9-4e61-b2b1-517445b7bd5a Type:ContainerStarted Data:962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a} Nov 16 11:59:29.222454 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:29.222427 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l5trv" event=&{ID:40251af6-202f-415a-aa28-c362b501dbd3 Type:ContainerStarted Data:91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228} Nov 16 11:59:29.223622 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:29.223598 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dn4b9" event=&{ID:39a966fc-6573-4f04-9a9d-8cb766b277ba Type:ContainerStarted Data:131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6} Nov 16 11:59:29.224734 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:29.224708 1563 generic.go:296] "Generic (PLEG): container finished" podID=13c954d2-615d-4b55-be8a-e712b8fa30a8 containerID="4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470" exitCode=0 Nov 16 11:59:29.224734 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:29.224742 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q722j" event=&{ID:13c954d2-615d-4b55-be8a-e712b8fa30a8 Type:ContainerDied Data:4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470} Nov 16 11:59:29.225449 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:29.225401617Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:17d913a46cdddddc58f82d0429fba20dfdf73a53d57c176d10422e363086503f" id=8d957ecf-3fab-42ac-aed1-e006842b8ab4 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:29.225773 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:29.225683019Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:17d913a46cdddddc58f82d0429fba20dfdf73a53d57c176d10422e363086503f not found" id=8d957ecf-3fab-42ac-aed1-e006842b8ab4 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:29.225880 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:29.225849 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kt88r" event=&{ID:37496343-5356-4736-accc-adc8f05b43c5 Type:ContainerStarted Data:9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf} Nov 16 11:59:29.226332 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:29.226301623Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:17d913a46cdddddc58f82d0429fba20dfdf73a53d57c176d10422e363086503f" id=c8765966-bc2f-44ae-8f3d-494bb41f598c name=/runtime.v1.ImageService/PullImage Nov 16 11:59:29.227151 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:29.227102 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dz75k" event=&{ID:5026bf33-68a7-4497-8c41-20a85e407ef9 Type:ContainerStarted Data:aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28} Nov 16 11:59:29.227745 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00039|connmgr|INFO|br0<->unix#4: 49 flow_mods in the last 0 s (49 adds) Nov 16 11:59:29.228985 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:29.228949643Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:17d913a46cdddddc58f82d0429fba20dfdf73a53d57c176d10422e363086503f\"" Nov 16 11:59:29.229511 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599969.2294] device (tun0): carrier: link connected Nov 16 11:59:29.273496 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00040|connmgr|INFO|br0<->unix#7: 4 flow_mods in the last 0 s (4 adds) Nov 16 11:59:29.349323 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00041|connmgr|INFO|br0<->unix#11: 1 flow_mods in the last 0 s (1 adds) Nov 16 11:59:29.540940 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00042|connmgr|INFO|br0<->unix#16: 3 flow_mods in the last 0 s (3 adds) Nov 16 11:59:29.565694 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00043|connmgr|INFO|br0<->unix#19: 1 flow_mods in the last 0 s (1 adds) Nov 16 11:59:29.587050 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:29.587010 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:29.587050 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:29.587142 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:29.589581 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00044|connmgr|INFO|br0<->unix#22: 3 flow_mods in the last 0 s (3 adds) Nov 16 11:59:29.612668 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00045|connmgr|INFO|br0<->unix#25: 1 flow_mods in the last 0 s (1 adds) Nov 16 11:59:29.637037 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00046|connmgr|INFO|br0<->unix#28: 3 flow_mods in the last 0 s (3 adds) Nov 16 11:59:29.661028 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00047|connmgr|INFO|br0<->unix#31: 1 flow_mods in the last 0 s (1 adds) Nov 16 11:59:29.684765 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00048|connmgr|INFO|br0<->unix#34: 3 flow_mods in the last 0 s (3 adds) Nov 16 11:59:29.708709 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00049|connmgr|INFO|br0<->unix#37: 1 flow_mods in the last 0 s (1 adds) Nov 16 11:59:29.717831 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:29.717755885Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f\"" Nov 16 11:59:29.725437 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:29.725396542Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:94efd22fba0607fd214d5e5aa65cc90e5e6f4dd0b41dde42c6a5c9ed80e5b8c3\"" Nov 16 11:59:29.732561 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00050|connmgr|INFO|br0<->unix#40: 3 flow_mods in the last 0 s (3 adds) Nov 16 11:59:29.756598 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00051|connmgr|INFO|br0<->unix#43: 1 flow_mods in the last 0 s (1 adds) Nov 16 11:59:29.872385 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:29.872257736Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f\"" Nov 16 11:59:30.450885 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:30.450793847Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:17d913a46cdddddc58f82d0429fba20dfdf73a53d57c176d10422e363086503f\"" Nov 16 11:59:30.587325 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:30.587283 1563 kubelet.go:2414] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Nov 16 11:59:30.713312 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:30.713165301Z" level=info msg="CNI monitoring event \"/etc/kubernetes/cni/net.d/00-multus.conf\": CREATE" Nov 16 11:59:30.726717 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:30.726681602Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:30.726717 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:30.726712502Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:30.726813 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:30.726735702Z" level=info msg="CNI monitoring event \"/etc/kubernetes/cni/net.d/00-multus.conf\": WRITE" Nov 16 11:59:30.739057 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:30.739020294Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:30.739057 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:30.739052494Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:30.739181 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:30.739067394Z" level=info msg="CNI monitoring event \"/etc/kubernetes/cni/net.d/00-multus.conf\": CHMOD" Nov 16 11:59:30.739181 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:30.739155595Z" level=info msg="CNI monitoring event \"/etc/kubernetes/cni/net.d/00-multus.conf\": CHMOD" Nov 16 11:59:31.586838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:31.586777 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:31.587038 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:31.586930 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:33.586819 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:33.586727 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:33.587479 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:33.586878 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:33.886416 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:33.886277743Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=37658c06-c4eb-4df1-8023-478395d39fd3 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:33.886802 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:33.886436944Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=d66ff06c-5be5-49ec-a5f9-4b43a6044579 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:33.887396 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:33.887348850Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=99741f1a-0a69-48bd-9a28-ba505ed0f381 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:33.889308 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:33.889254465Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=99741f1a-0a69-48bd-9a28-ba505ed0f381 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:33.890253 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:33.890215972Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:94efd22fba0607fd214d5e5aa65cc90e5e6f4dd0b41dde42c6a5c9ed80e5b8c3" id=3cdad600-07c6-432c-8ec1-81dc31bc9761 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:33.890991 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:33.890937577Z" level=info msg="Creating container: openshift-monitoring/node-exporter-dz75k/kube-rbac-proxy" id=59f07917-7c6a-4867-8ac5-cd3f9f10faa8 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:33.891183 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:33.891109679Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:33.892041 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:33.891996385Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=7e664bc3-ee0b-4266-9b6e-bc24534daef5 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:33.894194 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:33.894071801Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=7e664bc3-ee0b-4266-9b6e-bc24534daef5 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:33.897105 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:33.896973822Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:94efd22fba0607fd214d5e5aa65cc90e5e6f4dd0b41dde42c6a5c9ed80e5b8c3" id=afb4a319-67ae-4b89-8f74-2888a3199fc0 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:33.897105 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:33.897062423Z" level=info msg="Creating container: openshift-sdn/sdn-pgwdx/kube-rbac-proxy" id=ddf3873f-73ea-42fb-bfe3-5841000afd0c name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:33.897335 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:33.897196124Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:34.102003 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.101943549Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5b790a1b5543c94964adf6e2c523e96f247c7dcca2e374ac2022ffd6c3f49054,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:94efd22fba0607fd214d5e5aa65cc90e5e6f4dd0b41dde42c6a5c9ed80e5b8c3],Size_:372296313,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=afb4a319-67ae-4b89-8f74-2888a3199fc0 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:34.103331 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.103291359Z" level=info msg="Creating container: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-driver" id=4d3217e1-9378-417a-a1c0-68d3c8b0e327 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:34.103455 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.103419360Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:34.113799 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697.scope. Nov 16 11:59:34.138489 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697. Nov 16 11:59:34.167915 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:34.234922 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.234866840Z" level=info msg="Created container cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697: openshift-monitoring/node-exporter-dz75k/kube-rbac-proxy" id=59f07917-7c6a-4867-8ac5-cd3f9f10faa8 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:34.235835 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.235804947Z" level=info msg="Starting container: cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697" id=d797b738-c060-43b2-b1ee-d6a0d184edb7 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:34.247355 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.247310232Z" level=info msg="Started container" PID=3021 containerID=cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697 description=openshift-monitoring/node-exporter-dz75k/kube-rbac-proxy id=d797b738-c060-43b2-b1ee-d6a0d184edb7 name=/runtime.v1.RuntimeService/StartContainer sandboxID=502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b Nov 16 11:59:34.305852 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8.scope. Nov 16 11:59:34.331933 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8. Nov 16 11:59:34.366974 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:34.461303 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1.scope. Nov 16 11:59:34.481377 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1. Nov 16 11:59:34.485514 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.485459307Z" level=info msg="Created container 516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8: openshift-sdn/sdn-pgwdx/kube-rbac-proxy" id=ddf3873f-73ea-42fb-bfe3-5841000afd0c name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:34.490373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.490332643Z" level=info msg="Starting container: 516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8" id=42122816-a025-4177-8244-2bb5227a71f2 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:34.506464 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.506355062Z" level=info msg="Started container" PID=3080 containerID=516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8 description=openshift-sdn/sdn-pgwdx/kube-rbac-proxy id=42122816-a025-4177-8244-2bb5227a71f2 name=/runtime.v1.RuntimeService/StartContainer sandboxID=6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75 Nov 16 11:59:34.531068 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.531012246Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3909f49bf7a6d86ee5d24039d79506226fa6d0a80ac11956ac1dd770be4630d0" id=899969a9-f059-47ab-8045-22d4c2d8e297 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:34.532948 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.532910660Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:95b3d88543a2c70ed1816d81b829973d2734c522865bede8e041a56f39f7d139,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3909f49bf7a6d86ee5d24039d79506226fa6d0a80ac11956ac1dd770be4630d0],Size_:572880412,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=899969a9-f059-47ab-8045-22d4c2d8e297 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:34.534578 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.533728966Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3909f49bf7a6d86ee5d24039d79506226fa6d0a80ac11956ac1dd770be4630d0" id=bcf6cb21-da0e-438b-8bb2-a6eaff69c048 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:34.535618 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.535441879Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:95b3d88543a2c70ed1816d81b829973d2734c522865bede8e041a56f39f7d139,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3909f49bf7a6d86ee5d24039d79506226fa6d0a80ac11956ac1dd770be4630d0],Size_:572880412,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=bcf6cb21-da0e-438b-8bb2-a6eaff69c048 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:34.536764 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.536703588Z" level=info msg="Creating container: openshift-sdn/sdn-pgwdx/drop-icmp" id=fa168bde-5b06-46be-a122-e812966504b7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:34.536834 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.536804989Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:34.659335 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 11:59:34.659791 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 11:59:34.687954 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 11:59:34.688306 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 11:59:34.688530 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.688110417Z" level=info msg="Created container 3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-driver" id=4d3217e1-9378-417a-a1c0-68d3c8b0e327 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:34.688806 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.688766821Z" level=info msg="Starting container: 3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1" id=5e471c64-39e1-496e-8fc6-53b4cb400ae2 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:34.702630 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.702583724Z" level=info msg="Started container" PID=3272 containerID=3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1 description=openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-driver id=5e471c64-39e1-496e-8fc6-53b4cb400ae2 name=/runtime.v1.RuntimeService/StartContainer sandboxID=8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883 Nov 16 11:59:34.818473 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19.scope. Nov 16 11:59:34.838781 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19. Nov 16 11:59:34.855726 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.855677065Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b90df0181fa84317e12429c6da4171b9d8680f8f7ffd29d54c0379153de063ca" id=54cf5531-e59b-4c21-8319-18ade0704fe3 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:34.856012 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.855984967Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b90df0181fa84317e12429c6da4171b9d8680f8f7ffd29d54c0379153de063ca not found" id=54cf5531-e59b-4c21-8319-18ade0704fe3 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:34.856813 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.856762373Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b90df0181fa84317e12429c6da4171b9d8680f8f7ffd29d54c0379153de063ca" id=75b5d8d0-5477-4f3d-a80f-c84c765efdd6 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:34.860195 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.860159998Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b90df0181fa84317e12429c6da4171b9d8680f8f7ffd29d54c0379153de063ca\"" Nov 16 11:59:34.935481 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.935429059Z" level=info msg="Created container adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19: openshift-sdn/sdn-pgwdx/drop-icmp" id=fa168bde-5b06-46be-a122-e812966504b7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:34.936163 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.936008463Z" level=info msg="Starting container: adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19" id=1291f012-e36e-4dda-b282-243d0a09ed6a name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:34.947227 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:34.946998245Z" level=info msg="Started container" PID=3511 containerID=adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19 description=openshift-sdn/sdn-pgwdx/drop-icmp id=1291f012-e36e-4dda-b282-243d0a09ed6a name=/runtime.v1.RuntimeService/StartContainer sandboxID=6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75 Nov 16 11:59:35.066014 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 11:59:35.066440 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 11:59:35.085441 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 11:59:35.085741 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 11:59:35.221104 ropatil1611az-gs6zj-worker-westus-j4ngg logger[3620]: rendered-worker-8b35090bacf1ad998c4e01e0fd1046d1 Nov 16 11:59:35.238332 ropatil1611az-gs6zj-worker-westus-j4ngg root[3621]: machine-config-daemon[1735]: Update completed for config rendered-worker-8b35090bacf1ad998c4e01e0fd1046d1 and node has been successfully uncordoned Nov 16 11:59:35.246470 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:35.246444 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" event=&{ID:89deb745-c538-46ae-9d77-26643bf01a15 Type:ContainerStarted Data:3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1} Nov 16 11:59:35.248647 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:35.248617 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dz75k" event=&{ID:5026bf33-68a7-4497-8c41-20a85e407ef9 Type:ContainerStarted Data:cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697} Nov 16 11:59:35.250427 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:35.250398 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-sdn/sdn-pgwdx" event=&{ID:45cd7283-fcc9-4e61-b2b1-517445b7bd5a Type:ContainerStarted Data:adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19} Nov 16 11:59:35.250594 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:35.250435 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-sdn/sdn-pgwdx" event=&{ID:45cd7283-fcc9-4e61-b2b1-517445b7bd5a Type:ContainerStarted Data:516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8} Nov 16 11:59:35.250825 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:35.250800 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:35.308887 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:35.308837 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-sdn/sdn-pgwdx" Nov 16 11:59:35.586894 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:35.586858 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Nov 16 11:59:35.587052 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 11:59:35.587024 1563 pod_workers.go:951] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 Nov 16 11:59:36.481548 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:36.481486781Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b90df0181fa84317e12429c6da4171b9d8680f8f7ffd29d54c0379153de063ca\"" Nov 16 11:59:37.600558 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod1bd11256_096f_4190_9ee6_f23c6eaabf65.slice. Nov 16 11:59:37.618454 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod6754c3dd_53dd_4e4f_8d5f_ce077669c3d4.slice. Nov 16 11:59:38.330105 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.330017222Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:17d913a46cdddddc58f82d0429fba20dfdf73a53d57c176d10422e363086503f" id=c8765966-bc2f-44ae-8f3d-494bb41f598c name=/runtime.v1.ImageService/PullImage Nov 16 11:59:38.335698 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.335621965Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:17d913a46cdddddc58f82d0429fba20dfdf73a53d57c176d10422e363086503f" id=b236158d-a741-4d03-9697-d07274ac48ed name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:38.338316 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.338142684Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:53779f5cf5b3e4225e2e9af9dd4cf1be05e79168193167d50a8d7e01a90372c5,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:17d913a46cdddddc58f82d0429fba20dfdf73a53d57c176d10422e363086503f],Size_:503048298,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=b236158d-a741-4d03-9697-d07274ac48ed name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:38.339030 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.338971590Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-q722j/cni-plugins" id=fce6cab4-0cc4-4079-95ec-60ae77bdd28e name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:38.339170 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.339092491Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:38.391270 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632.scope. Nov 16 11:59:38.414307 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632. Nov 16 11:59:38.445632 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:38.612184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.612105180Z" level=info msg="Created container 7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632: openshift-multus/multus-additional-cni-plugins-q722j/cni-plugins" id=fce6cab4-0cc4-4079-95ec-60ae77bdd28e name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:38.612696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.612607783Z" level=info msg="Starting container: 7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632" id=1deec5d9-640d-4609-9fb8-ab8dcf193db3 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:38.625340 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.625281880Z" level=info msg="Started container" PID=3770 containerID=7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632 description=openshift-multus/multus-additional-cni-plugins-q722j/cni-plugins id=1deec5d9-640d-4609-9fb8-ab8dcf193db3 name=/runtime.v1.RuntimeService/StartContainer sandboxID=8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3 Nov 16 11:59:38.630335 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.630254918Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/bandwidth\": CREATE" Nov 16 11:59:38.646660 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.646599643Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.646660 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.646645644Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.646660 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.646665744Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/bandwidth\": WRITE" Nov 16 11:59:38.660311 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.660248248Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.660311 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.660297348Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.660311 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.660316448Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/bandwidth\": WRITE" Nov 16 11:59:38.672417 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.672357540Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.672584 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.672411241Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.672584 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.672494642Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/bridge\": CREATE" Nov 16 11:59:38.684275 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.684219231Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.684275 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.684264132Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.684275 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.684290432Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/bridge\": WRITE" Nov 16 11:59:38.692304 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632.scope: Succeeded. Nov 16 11:59:38.692967 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632.scope: Consumed 78ms CPU time Nov 16 11:59:38.698437 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.698394340Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.698437 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.698433940Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.698569 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.698455940Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/dhcp\": CREATE" Nov 16 11:59:38.709370 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.709335623Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.709370 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.709369324Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.709370 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.709395924Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/dhcp\": WRITE" Nov 16 11:59:38.719277 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.719242099Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.719277 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.719276499Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.719397 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.719291600Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/firewall\": CREATE" Nov 16 11:59:38.730368 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.730332284Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.730368 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.730365084Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.730482 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.730381684Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/firewall\": WRITE" Nov 16 11:59:38.743675 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.743638786Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.743675 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.743670986Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.743828 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.743686086Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/host-device\": CREATE" Nov 16 11:59:38.755904 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.755856079Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.755904 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.755888879Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.755904 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.755905580Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/host-device\": WRITE" Nov 16 11:59:38.767901 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.767867571Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.767901 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.767897271Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.768005 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.767911471Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/host-device\": WRITE" Nov 16 11:59:38.778415 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.778348251Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.778415 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.778381752Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.778415 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.778414752Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/host-local\": CREATE" Nov 16 11:59:38.788640 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.788574129Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.788640 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.788602030Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.788640 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.788615930Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/host-local\": WRITE" Nov 16 11:59:38.799236 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.799200111Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.799236 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.799232511Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.799236 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.799249911Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/ipvlan\": CREATE" Nov 16 11:59:38.810331 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.810297796Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.810331 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.810328696Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.810331 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.810343196Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/ipvlan\": WRITE" Nov 16 11:59:38.820309 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.820267672Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.820309 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.820303272Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.820309 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.820318672Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/loopback\": CREATE" Nov 16 11:59:38.833735 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.833654874Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.833735 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.833686475Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.833735 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.833726175Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/loopback\": WRITE" Nov 16 11:59:38.843827 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.843747552Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.843827 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.843781052Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.843827 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.843814152Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/macvlan\": CREATE" Nov 16 11:59:38.854968 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.854934337Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.854968 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.854965937Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.855098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.854986638Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/macvlan\": WRITE" Nov 16 11:59:38.865416 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.865309116Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.865416 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.865340417Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.865416 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.865355817Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/portmap\": CREATE" Nov 16 11:59:38.879715 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.879618126Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.879715 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.879652926Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.879715 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.879677026Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/portmap\": WRITE" Nov 16 11:59:38.892099 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.892014321Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.892099 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.892053221Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.892099 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.892085021Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/ptp\": CREATE" Nov 16 11:59:38.902976 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.902937804Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.902976 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.902968005Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.902976 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.902982805Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/ptp\": WRITE" Nov 16 11:59:38.914221 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.914183390Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.914221 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.914217191Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.914320 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.914233191Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/sbr\": CREATE" Nov 16 11:59:38.924328 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.924290768Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.924328 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.924330868Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.924469 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.924346968Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/sbr\": WRITE" Nov 16 11:59:38.935348 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.935311552Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.935348 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.935342252Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.935348 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.935358052Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/static\": CREATE" Nov 16 11:59:38.945419 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.945383229Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.945419 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.945413829Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.945550 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.945429329Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/static\": WRITE" Nov 16 11:59:38.956877 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.956778916Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.956877 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.956822917Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.956877 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.956857117Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/tuning\": CREATE" Nov 16 11:59:38.966826 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:38.966794 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeReady" Nov 16 11:59:38.967219 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.967004994Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.967219 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.967028495Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.967219 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.967051395Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/tuning\": WRITE" Nov 16 11:59:38.980098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.980014094Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.980098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.980049194Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.980098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.980075194Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/vlan\": CREATE" Nov 16 11:59:38.991447 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.991411181Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:38.991447 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.991442381Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:38.991580 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:38.991463781Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/vlan\": WRITE" Nov 16 11:59:39.003785 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:39.003713975Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:39.003785 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:39.003746675Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:39.003785 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:39.003761376Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/vrf\": CREATE" Nov 16 11:59:39.015932 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:39.015826768Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:39.015932 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:39.015889668Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:39.015932 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:39.015911369Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/vrf\": WRITE" Nov 16 11:59:39.026392 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:39.026352948Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:39.026392 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:39.026384949Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:39.259928 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:39.259805 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q722j" event=&{ID:13c954d2-615d-4b55-be8a-e712b8fa30a8 Type:ContainerStarted Data:7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632} Nov 16 11:59:39.327056 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632.scope: Succeeded. Nov 16 11:59:39.327739 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632.scope: Consumed 35ms CPU time Nov 16 11:59:39.768727 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:39.768643 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-dns/dns-default-kg69s] Nov 16 11:59:39.768727 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:39.768701 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:39.779123 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod306e3397_e392_42af_b8d9_63e5f5dfae1b.slice. Nov 16 11:59:39.785169 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:39.785116 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-ingress-canary/ingress-canary-c77h2] Nov 16 11:59:39.785169 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:39.785167 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:39.799331 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-podb0793e26_e08f_46cd_bee3_f0ec4509d53a.slice. Nov 16 11:59:39.810933 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:39.810900 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306e3397-e392-42af-b8d9-63e5f5dfae1b-metrics-tls\") pod \"dns-default-kg69s\" (UID: \"306e3397-e392-42af-b8d9-63e5f5dfae1b\") " pod="openshift-dns/dns-default-kg69s" Nov 16 11:59:39.811032 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:39.810948 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/306e3397-e392-42af-b8d9-63e5f5dfae1b-config-volume\") pod \"dns-default-kg69s\" (UID: \"306e3397-e392-42af-b8d9-63e5f5dfae1b\") " pod="openshift-dns/dns-default-kg69s" Nov 16 11:59:39.811032 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:39.810984 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6wbv\" (UniqueName: \"kubernetes.io/projected/306e3397-e392-42af-b8d9-63e5f5dfae1b-kube-api-access-n6wbv\") pod \"dns-default-kg69s\" (UID: \"306e3397-e392-42af-b8d9-63e5f5dfae1b\") " pod="openshift-dns/dns-default-kg69s" Nov 16 11:59:39.911487 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:39.911439 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306e3397-e392-42af-b8d9-63e5f5dfae1b-metrics-tls\") pod \"dns-default-kg69s\" (UID: \"306e3397-e392-42af-b8d9-63e5f5dfae1b\") " pod="openshift-dns/dns-default-kg69s" Nov 16 11:59:39.911487 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:39.911502 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/306e3397-e392-42af-b8d9-63e5f5dfae1b-config-volume\") pod \"dns-default-kg69s\" (UID: \"306e3397-e392-42af-b8d9-63e5f5dfae1b\") " pod="openshift-dns/dns-default-kg69s" Nov 16 11:59:39.911705 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:39.911539 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-n6wbv\" (UniqueName: \"kubernetes.io/projected/306e3397-e392-42af-b8d9-63e5f5dfae1b-kube-api-access-n6wbv\") pod \"dns-default-kg69s\" (UID: \"306e3397-e392-42af-b8d9-63e5f5dfae1b\") " pod="openshift-dns/dns-default-kg69s" Nov 16 11:59:39.912520 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:39.912399 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/306e3397-e392-42af-b8d9-63e5f5dfae1b-config-volume\") pod \"dns-default-kg69s\" (UID: \"306e3397-e392-42af-b8d9-63e5f5dfae1b\") " pod="openshift-dns/dns-default-kg69s" Nov 16 11:59:39.916386 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:39.916354 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306e3397-e392-42af-b8d9-63e5f5dfae1b-metrics-tls\") pod \"dns-default-kg69s\" (UID: \"306e3397-e392-42af-b8d9-63e5f5dfae1b\") " pod="openshift-dns/dns-default-kg69s" Nov 16 11:59:40.113437 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:40.113397 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4dwq\" (UniqueName: \"kubernetes.io/projected/b0793e26-e08f-46cd-bee3-f0ec4509d53a-kube-api-access-k4dwq\") pod \"ingress-canary-c77h2\" (UID: \"b0793e26-e08f-46cd-bee3-f0ec4509d53a\") " pod="openshift-ingress-canary/ingress-canary-c77h2" Nov 16 11:59:40.214154 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:40.214097 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-k4dwq\" (UniqueName: \"kubernetes.io/projected/b0793e26-e08f-46cd-bee3-f0ec4509d53a-kube-api-access-k4dwq\") pod \"ingress-canary-c77h2\" (UID: \"b0793e26-e08f-46cd-bee3-f0ec4509d53a\") " pod="openshift-ingress-canary/ingress-canary-c77h2" Nov 16 11:59:40.263084 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:40.263052 1563 generic.go:296] "Generic (PLEG): container finished" podID=13c954d2-615d-4b55-be8a-e712b8fa30a8 containerID="7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632" exitCode=0 Nov 16 11:59:40.263084 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:40.263097 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q722j" event=&{ID:13c954d2-615d-4b55-be8a-e712b8fa30a8 Type:ContainerDied Data:7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632} Nov 16 11:59:40.263802 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:40.263753014Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a2070d322c3195865dc45d129bebc1ac75b0bdbc36265f4554bfe09785521cac" id=7925c935-024b-4441-ac9b-9f21445395e3 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:40.874544 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:40.874483786Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a2070d322c3195865dc45d129bebc1ac75b0bdbc36265f4554bfe09785521cac not found" id=7925c935-024b-4441-ac9b-9f21445395e3 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:40.875915 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:40.875818096Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a2070d322c3195865dc45d129bebc1ac75b0bdbc36265f4554bfe09785521cac" id=56afd114-3d10-402d-b3d3-991b70c54e58 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:40.888006 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:40.887961789Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a2070d322c3195865dc45d129bebc1ac75b0bdbc36265f4554bfe09785521cac\"" Nov 16 11:59:40.898783 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:40.898716271Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b90df0181fa84317e12429c6da4171b9d8680f8f7ffd29d54c0379153de063ca" id=75b5d8d0-5477-4f3d-a80f-c84c765efdd6 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:40.899761 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:40.899631278Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b90df0181fa84317e12429c6da4171b9d8680f8f7ffd29d54c0379153de063ca" id=e67f4fa9-4b07-4632-90cb-92bed519cfb3 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:40.901242 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:40.901201390Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:6d1ff5a2f4ad0bfa9ae09fbdbdb2c0103ab23bbbe073804e4c220cdababdc4c4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b90df0181fa84317e12429c6da4171b9d8680f8f7ffd29d54c0379153de063ca],Size_:301950747,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=e67f4fa9-4b07-4632-90cb-92bed519cfb3 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:40.902096 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:40.902014296Z" level=info msg="Creating container: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-node-driver-registrar" id=8be8935e-4fc9-47ea-93de-6e0eafac43fe name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:40.902193 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:40.902123697Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:40.946358 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0.scope. Nov 16 11:59:40.970441 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0. Nov 16 11:59:41.016202 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:41.016169 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6wbv\" (UniqueName: \"kubernetes.io/projected/306e3397-e392-42af-b8d9-63e5f5dfae1b-kube-api-access-n6wbv\") pod \"dns-default-kg69s\" (UID: \"306e3397-e392-42af-b8d9-63e5f5dfae1b\") " pod="openshift-dns/dns-default-kg69s" Nov 16 11:59:41.037417 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.037349332Z" level=info msg="Created container 83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-node-driver-registrar" id=8be8935e-4fc9-47ea-93de-6e0eafac43fe name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:41.038008 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.037973436Z" level=info msg="Starting container: 83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0" id=80b47a91-9e22-4ee7-99b5-dc3229b67881 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:41.038226 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:41.038193 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4dwq\" (UniqueName: \"kubernetes.io/projected/b0793e26-e08f-46cd-bee3-f0ec4509d53a-kube-api-access-k4dwq\") pod \"ingress-canary-c77h2\" (UID: \"b0793e26-e08f-46cd-bee3-f0ec4509d53a\") " pod="openshift-ingress-canary/ingress-canary-c77h2" Nov 16 11:59:41.047977 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.047895112Z" level=info msg="Started container" PID=4132 containerID=83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0 description=openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-node-driver-registrar id=80b47a91-9e22-4ee7-99b5-dc3229b67881 name=/runtime.v1.RuntimeService/StartContainer sandboxID=8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883 Nov 16 11:59:41.058909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:41.058860 1563 plugin_watcher.go:203] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/disk.csi.azure.com-reg.sock" Nov 16 11:59:41.067927 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.067870565Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6114d0ffa57dba48271a6d3cca48aada961cf8742ce3d499498343170f9b9c7" id=21bb33fd-0189-4b91-af05-4f8972a94f51 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:41.068203 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.068173667Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6114d0ffa57dba48271a6d3cca48aada961cf8742ce3d499498343170f9b9c7 not found" id=21bb33fd-0189-4b91-af05-4f8972a94f51 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:41.068994 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.068916373Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6114d0ffa57dba48271a6d3cca48aada961cf8742ce3d499498343170f9b9c7" id=b5518a48-da34-435a-8fc2-28ad0bcc4061 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:41.071702 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.071605794Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6114d0ffa57dba48271a6d3cca48aada961cf8742ce3d499498343170f9b9c7\"" Nov 16 11:59:41.267284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:41.267174 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" event=&{ID:89deb745-c538-46ae-9d77-26643bf01a15 Type:ContainerStarted Data:83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0} Nov 16 11:59:41.294303 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:41.294194 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kg69s" Nov 16 11:59:41.294888 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.294800801Z" level=info msg="Running pod sandbox: openshift-dns/dns-default-kg69s/POD" id=78b57bcb-4d25-4b0a-90b9-b61935f348be name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:41.295313 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.294929702Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:41.304206 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:41.304113 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c77h2" Nov 16 11:59:41.304863 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.304737877Z" level=info msg="Running pod sandbox: openshift-ingress-canary/ingress-canary-c77h2/POD" id=2ecd1a07-d86d-40ac-a1dc-9ddddd807a15 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:41.304863 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.304834378Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:41.347403 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.347360703Z" level=info msg="Got pod network &{Name:dns-default-kg69s Namespace:openshift-dns ID:9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f UID:306e3397-e392-42af-b8d9-63e5f5dfae1b NetNS:/var/run/netns/beaa79cf-4611-4079-83a4-5e17cfd681cc Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 11:59:41.347403 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.347403103Z" level=info msg="Adding pod openshift-dns_dns-default-kg69s to CNI network \"multus-cni-network\" (type=multus)" Nov 16 11:59:41.398812 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 11:59:41.422231 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): vethb019706b: link is not ready Nov 16 11:59:41.423821 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.423774788Z" level=info msg="Got pod network &{Name:ingress-canary-c77h2 Namespace:openshift-ingress-canary ID:4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171 UID:b0793e26-e08f-46cd-bee3-f0ec4509d53a NetNS:/var/run/netns/bf1e2de8-c57d-477e-97f9-032a09eaee73 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 11:59:41.423821 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:41.423818488Z" level=info msg="Adding pod openshift-ingress-canary_ingress-canary-c77h2 to CNI network \"multus-cni-network\" (type=multus)" Nov 16 11:59:41.431203 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethb019706b: link becomes ready Nov 16 11:59:41.431459 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 11:59:41.439551 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[4211]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:59:41.439723 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[4211]: Could not generate persistent MAC address for vethb019706b: No such file or directory Nov 16 11:59:41.439815 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599981.4396] manager: (vethb019706b): new Veth device (/org/freedesktop/NetworkManager/Devices/10) Nov 16 11:59:41.440731 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599981.4406] device (vethb019706b): carrier: link connected Nov 16 11:59:41.447818 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599981.4477] manager: (vethb019706b): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/11) Nov 16 11:59:41.452473 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00052|bridge|INFO|bridge br0: added interface vethb019706b on port 3 Nov 16 11:59:41.452889 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethb019706b entered promiscuous mode Nov 16 11:59:41.470184 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 11:59:41.479965 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599981.4799] manager: (vethb0219afb): new Veth device (/org/freedesktop/NetworkManager/Devices/12) Nov 16 11:59:41.480702 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599981.4806] device (vethb0219afb): carrier: link connected Nov 16 11:59:41.486593 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[4240]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:59:41.486732 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[4240]: Could not generate persistent MAC address for vethb0219afb: No such file or directory Nov 16 11:59:41.487157 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 11:59:41.491334 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00053|connmgr|INFO|br0<->unix#46: 5 flow_mods in the last 0 s (5 adds) Nov 16 11:59:41.540955 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00054|connmgr|INFO|br0<->unix#50: 6 flow_mods in the last 0 s (2 adds, 4 deletes) Nov 16 11:59:41.542266 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00055|connmgr|INFO|br0<->unix#52: 2 flow_mods in the last 0 s (2 deletes) Nov 16 11:59:41.564190 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599981.5641] manager: (vethb0219afb): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/13) Nov 16 11:59:41.569710 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00056|bridge|INFO|bridge br0: added interface vethb0219afb on port 4 Nov 16 11:59:41.570154 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethb0219afb entered promiscuous mode Nov 16 11:59:41.601217 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00057|connmgr|INFO|br0<->unix#55: 5 flow_mods in the last 0 s (5 adds) Nov 16 11:59:41.649118 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00058|connmgr|INFO|br0<->unix#59: 2 flow_mods in the last 0 s (2 deletes) Nov 16 11:59:41.650939 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00059|connmgr|INFO|br0<->unix#61: 6 flow_mods in the last 0 s (2 adds, 4 deletes) Nov 16 11:59:41.728771 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:41.728731 1563 reconciler.go:160] "OperationExecutor.RegisterPlugin started" plugin={SocketPath:/var/lib/kubelet/plugins_registry/disk.csi.azure.com-reg.sock Timestamp:2022-11-16 11:59:41.058903096 +0000 UTC m=+385.995678718 Handler: Name:} Nov 16 11:59:41.731423 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:41.731397 1563 csi_plugin.go:99] kubernetes.io/csi: Trying to validate a new CSI Driver with name: disk.csi.azure.com endpoint: /var/lib/kubelet/plugins/disk.csi.azure.com/csi.sock versions: 1.0.0 Nov 16 11:59:41.731697 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:41.731676 1563 csi_plugin.go:112] kubernetes.io/csi: Register new plugin with name: disk.csi.azure.com at endpoint: /var/lib/kubelet/plugins/disk.csi.azure.com/csi.sock Nov 16 11:59:41.828115 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:41.827936 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs\") pod \"network-metrics-daemon-hr6jx\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " pod="openshift-multus/network-metrics-daemon-hr6jx" Nov 16 11:59:41.830777 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:41.830715 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs\") pod \"network-metrics-daemon-hr6jx\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " pod="openshift-multus/network-metrics-daemon-hr6jx" Nov 16 11:59:41.929384 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:41.929308 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-zdfrr\" (UniqueName: \"kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr\") pod \"network-check-target-299n8\" (UID: \"1bd11256-096f-4190-9ee6-f23c6eaabf65\") " pod="openshift-network-diagnostics/network-check-target-299n8" Nov 16 11:59:41.932354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:41.932326 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdfrr\" (UniqueName: \"kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr\") pod \"network-check-target-299n8\" (UID: \"1bd11256-096f-4190-9ee6-f23c6eaabf65\") " pod="openshift-network-diagnostics/network-check-target-299n8" Nov 16 11:59:42.119185 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:42.119118 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-299n8" Nov 16 11:59:42.119861 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.119800912Z" level=info msg="Running pod sandbox: openshift-network-diagnostics/network-check-target-299n8/POD" id=a1d9f353-a1db-4344-bf1b-f9a45cb80160 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:42.119948 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.119878712Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:42.125279 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:42.125252 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr6jx" Nov 16 11:59:42.125755 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.125705157Z" level=info msg="Running pod sandbox: openshift-multus/network-metrics-daemon-hr6jx/POD" id=8ffde09b-415a-444f-8ae7-954db0cf1140 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:42.125838 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.125764657Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:42.152596 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.152486662Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a2070d322c3195865dc45d129bebc1ac75b0bdbc36265f4554bfe09785521cac\"" Nov 16 11:59:42.178412 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.178361160Z" level=info msg="Got pod network &{Name:network-check-target-299n8 Namespace:openshift-network-diagnostics ID:b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76 UID:1bd11256-096f-4190-9ee6-f23c6eaabf65 NetNS:/var/run/netns/dda9a386-e7c6-4a1c-a6c3-1e31c9168371 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 11:59:42.178412 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.178412460Z" level=info msg="Adding pod openshift-network-diagnostics_network-check-target-299n8 to CNI network \"multus-cni-network\" (type=multus)" Nov 16 11:59:42.221172 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 11:59:42.233097 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599982.2330] manager: (veth886b239b): new Veth device (/org/freedesktop/NetworkManager/Devices/14) Nov 16 11:59:42.234012 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599982.2339] device (veth886b239b): carrier: link connected Nov 16 11:59:42.238927 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[4309]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:59:42.239328 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 11:59:42.240104 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[4309]: Could not generate persistent MAC address for veth886b239b: No such file or directory Nov 16 11:59:42.241509 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.241445142Z" level=info msg="Got pod network &{Name:network-metrics-daemon-hr6jx Namespace:openshift-multus ID:2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a UID:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 NetNS:/var/run/netns/ebb09e20-5775-49e7-8cfc-a0ae9c03e9d4 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 11:59:42.241509 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.241511043Z" level=info msg="Adding pod openshift-multus_network-metrics-daemon-hr6jx to CNI network \"multus-cni-network\" (type=multus)" Nov 16 11:59:42.254979 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599982.2549] manager: (veth886b239b): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/15) Nov 16 11:59:42.260642 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00060|bridge|INFO|bridge br0: added interface veth886b239b on port 5 Nov 16 11:59:42.261159 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth886b239b entered promiscuous mode Nov 16 11:59:42.284151 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 11:59:42.293039 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599982.2930] manager: (vethe8e66c39): new Veth device (/org/freedesktop/NetworkManager/Devices/16) Nov 16 11:59:42.293998 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599982.2939] device (vethe8e66c39): carrier: link connected Nov 16 11:59:42.300353 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[4344]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:59:42.300485 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[4344]: Could not generate persistent MAC address for vethe8e66c39: No such file or directory Nov 16 11:59:42.300947 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.300890097Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6114d0ffa57dba48271a6d3cca48aada961cf8742ce3d499498343170f9b9c7\"" Nov 16 11:59:42.301348 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 11:59:42.304026 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00061|connmgr|INFO|br0<->unix#64: 5 flow_mods in the last 0 s (5 adds) Nov 16 11:59:42.349942 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00062|connmgr|INFO|br0<->unix#67: 2 flow_mods in the last 0 s (2 deletes) Nov 16 11:59:42.369371 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599982.3693] manager: (vethe8e66c39): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/17) Nov 16 11:59:42.375863 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00063|bridge|INFO|bridge br0: added interface vethe8e66c39 on port 6 Nov 16 11:59:42.376166 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethe8e66c39 entered promiscuous mode Nov 16 11:59:42.408620 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00064|connmgr|INFO|br0<->unix#70: 5 flow_mods in the last 0 s (5 adds) Nov 16 11:59:42.457313 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00065|connmgr|INFO|br0<->unix#73: 2 flow_mods in the last 0 s (2 deletes) Nov 16 11:59:42.676589 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00066|connmgr|INFO|br0<->unix#76: 10 flow_mods in the last 0 s (4 adds, 6 deletes) Nov 16 11:59:42.952676 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:42.952569 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-dns/dns-default-kg69s] Nov 16 11:59:42.953077 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T11:59:42Z [verbose] Add: openshift-dns:dns-default-kg69s:306e3397-e392-42af-b8d9-63e5f5dfae1b:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/beaa79cf-4611-4079-83a4-5e17cfd681cc"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.2/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 11:59:42.953077 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 11:59:42.931730 4189 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-dns", Name:"dns-default-kg69s", UID:"306e3397-e392-42af-b8d9-63e5f5dfae1b", APIVersion:"v1", ResourceVersion:"24928", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.2/23] from openshift-sdn Nov 16 11:59:42.953803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.953728691Z" level=info msg="Got pod network &{Name:dns-default-kg69s Namespace:openshift-dns ID:9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f UID:306e3397-e392-42af-b8d9-63e5f5dfae1b NetNS:/var/run/netns/beaa79cf-4611-4079-83a4-5e17cfd681cc Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 11:59:42.953883 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.953869892Z" level=info msg="Checking pod openshift-dns_dns-default-kg69s for CNI network multus-cni-network (type=multus)" Nov 16 11:59:42.956238 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:59:42.956208 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306e3397_e392_42af_b8d9_63e5f5dfae1b.slice/crio-9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f.scope WatchSource:0}: Error finding container 9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f: Status 404 returned error &{%!s(*http.body=&{0xc0022e71d0 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 11:59:42.967261 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.967227794Z" level=info msg="Ran pod sandbox 9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f with infra container: openshift-dns/dns-default-kg69s/POD" id=78b57bcb-4d25-4b0a-90b9-b61935f348be name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:42.968340 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.968301802Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:adb10bb2eed278eedcfa7aa9e65acce4467d8630ca37593a760788ceda16fc86" id=4ebbc20d-6c57-44aa-a4d2-5f065a20769b name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:42.968591 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.968562104Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:adb10bb2eed278eedcfa7aa9e65acce4467d8630ca37593a760788ceda16fc86 not found" id=4ebbc20d-6c57-44aa-a4d2-5f065a20769b name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:42.969364 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.969322410Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:adb10bb2eed278eedcfa7aa9e65acce4467d8630ca37593a760788ceda16fc86" id=08c3a308-05f1-454d-ac63-ad6aa1ae19f6 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:42.981618 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:42.981573904Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:adb10bb2eed278eedcfa7aa9e65acce4467d8630ca37593a760788ceda16fc86\"" Nov 16 11:59:43.272676 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:43.272529 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kg69s" event=&{ID:306e3397-e392-42af-b8d9-63e5f5dfae1b Type:ContainerStarted Data:9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f} Nov 16 11:59:43.335774 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:43.335698 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-ingress-canary/ingress-canary-c77h2] Nov 16 11:59:43.336869 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T11:59:43Z [verbose] Add: openshift-ingress-canary:ingress-canary-c77h2:b0793e26-e08f-46cd-bee3-f0ec4509d53a:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/bf1e2de8-c57d-477e-97f9-032a09eaee73"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.3/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 11:59:43.336869 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 11:59:43.298100 4213 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-ingress-canary", Name:"ingress-canary-c77h2", UID:"b0793e26-e08f-46cd-bee3-f0ec4509d53a", APIVersion:"v1", ResourceVersion:"24930", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.3/23] from openshift-sdn Nov 16 11:59:43.337790 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.337714628Z" level=info msg="Got pod network &{Name:ingress-canary-c77h2 Namespace:openshift-ingress-canary ID:4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171 UID:b0793e26-e08f-46cd-bee3-f0ec4509d53a NetNS:/var/run/netns/bf1e2de8-c57d-477e-97f9-032a09eaee73 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 11:59:43.337896 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.337860729Z" level=info msg="Checking pod openshift-ingress-canary_ingress-canary-c77h2 for CNI network multus-cni-network (type=multus)" Nov 16 11:59:43.340623 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:59:43.340558 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0793e26_e08f_46cd_bee3_f0ec4509d53a.slice/crio-4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171.scope WatchSource:0}: Error finding container 4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171: Status 404 returned error &{%!s(*http.body=&{0xc001be3890 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 11:59:43.348375 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.348341310Z" level=info msg="Ran pod sandbox 4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171 with infra container: openshift-ingress-canary/ingress-canary-c77h2/POD" id=2ecd1a07-d86d-40ac-a1dc-9ddddd807a15 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:43.349323 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.349287717Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:264660c55199b0faa7b19c04a437eb9cd9c6d6824d28fec7526f3656461e7cfe" id=d681fd16-e090-4ebc-9c92-55d859958d61 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:43.349653 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.349588619Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:264660c55199b0faa7b19c04a437eb9cd9c6d6824d28fec7526f3656461e7cfe not found" id=d681fd16-e090-4ebc-9c92-55d859958d61 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:43.350494 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.350458726Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:264660c55199b0faa7b19c04a437eb9cd9c6d6824d28fec7526f3656461e7cfe" id=394d5b0f-3cd9-408a-8026-f0abea1f1bf3 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:43.352943 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.352876044Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:264660c55199b0faa7b19c04a437eb9cd9c6d6824d28fec7526f3656461e7cfe\"" Nov 16 11:59:43.413931 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T11:59:43Z [verbose] Add: openshift-network-diagnostics:network-check-target-299n8:1bd11256-096f-4190-9ee6-f23c6eaabf65:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/dda9a386-e7c6-4a1c-a6c3-1e31c9168371"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.4/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 11:59:43.413931 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 11:59:43.386833 4290 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-network-diagnostics", Name:"network-check-target-299n8", UID:"1bd11256-096f-4190-9ee6-f23c6eaabf65", APIVersion:"v1", ResourceVersion:"23544", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.4/23] from openshift-sdn Nov 16 11:59:43.414605 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.414566416Z" level=info msg="Got pod network &{Name:network-check-target-299n8 Namespace:openshift-network-diagnostics ID:b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76 UID:1bd11256-096f-4190-9ee6-f23c6eaabf65 NetNS:/var/run/netns/dda9a386-e7c6-4a1c-a6c3-1e31c9168371 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 11:59:43.414767 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.414739417Z" level=info msg="Checking pod openshift-network-diagnostics_network-check-target-299n8 for CNI network multus-cni-network (type=multus)" Nov 16 11:59:43.415679 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:43.415643 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-network-diagnostics/network-check-target-299n8] Nov 16 11:59:43.417189 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:59:43.417147 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bd11256_096f_4190_9ee6_f23c6eaabf65.slice/crio-b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76.scope WatchSource:0}: Error finding container b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76: Status 404 returned error &{%!s(*http.body=&{0xc002340168 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 11:59:43.425229 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.425099497Z" level=info msg="Ran pod sandbox b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76 with infra container: openshift-network-diagnostics/network-check-target-299n8/POD" id=a1d9f353-a1db-4344-bf1b-f9a45cb80160 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:43.426216 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.426178105Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5973f89c60dd0ee15c6249c7913170a9097194ff6cde0a1d231512ad592d9559" id=1ab18115-ecdb-47ce-aef3-03e59823d05f name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:43.426490 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.426459207Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5973f89c60dd0ee15c6249c7913170a9097194ff6cde0a1d231512ad592d9559 not found" id=1ab18115-ecdb-47ce-aef3-03e59823d05f name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:43.427279 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.427237513Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5973f89c60dd0ee15c6249c7913170a9097194ff6cde0a1d231512ad592d9559" id=29812b9e-c903-4285-a61f-ef95de255e4c name=/runtime.v1.ImageService/PullImage Nov 16 11:59:43.429808 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:43.429735132Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5973f89c60dd0ee15c6249c7913170a9097194ff6cde0a1d231512ad592d9559\"" Nov 16 11:59:44.176458 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:44.176414 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-multus/network-metrics-daemon-hr6jx] Nov 16 11:59:44.176867 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T11:59:44Z [verbose] Add: openshift-multus:network-metrics-daemon-hr6jx:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/ebb09e20-5775-49e7-8cfc-a0ae9c03e9d4"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.5/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 11:59:44.176867 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 11:59:44.153430 4314 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-multus", Name:"network-metrics-daemon-hr6jx", UID:"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4", APIVersion:"v1", ResourceVersion:"23539", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.5/23] from openshift-sdn Nov 16 11:59:44.177405 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:44.177348559Z" level=info msg="Got pod network &{Name:network-metrics-daemon-hr6jx Namespace:openshift-multus ID:2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a UID:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 NetNS:/var/run/netns/ebb09e20-5775-49e7-8cfc-a0ae9c03e9d4 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 11:59:44.177549 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:44.177519360Z" level=info msg="Checking pod openshift-multus_network-metrics-daemon-hr6jx for CNI network multus-cni-network (type=multus)" Nov 16 11:59:44.179827 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 11:59:44.179786 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6754c3dd_53dd_4e4f_8d5f_ce077669c3d4.slice/crio-2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a.scope WatchSource:0}: Error finding container 2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a: Status 404 returned error &{%!s(*http.body=&{0xc00217e1f8 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 11:59:44.188314 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:44.188277648Z" level=info msg="Ran pod sandbox 2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a with infra container: openshift-multus/network-metrics-daemon-hr6jx/POD" id=8ffde09b-415a-444f-8ae7-954db0cf1140 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:44.189380 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:44.189317457Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9472b8291bcb75e928fe7853c8c53735fc34d909a79e0185396235205127b3b7" id=da3835a7-c98f-4cbe-82ac-8a43908bbe46 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:44.189628 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:44.189600159Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9472b8291bcb75e928fe7853c8c53735fc34d909a79e0185396235205127b3b7 not found" id=da3835a7-c98f-4cbe-82ac-8a43908bbe46 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:44.190564 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:44.190522767Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9472b8291bcb75e928fe7853c8c53735fc34d909a79e0185396235205127b3b7" id=6369522a-d986-49de-8303-ff9e21d11e1e name=/runtime.v1.ImageService/PullImage Nov 16 11:59:44.193512 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:44.193408690Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9472b8291bcb75e928fe7853c8c53735fc34d909a79e0185396235205127b3b7\"" Nov 16 11:59:44.218501 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:44.218429995Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:adb10bb2eed278eedcfa7aa9e65acce4467d8630ca37593a760788ceda16fc86\"" Nov 16 11:59:44.275939 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:44.275899 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-299n8" event=&{ID:1bd11256-096f-4190-9ee6-f23c6eaabf65 Type:ContainerStarted Data:b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76} Nov 16 11:59:44.276980 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:44.276910 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c77h2" event=&{ID:b0793e26-e08f-46cd-bee3-f0ec4509d53a Type:ContainerStarted Data:4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171} Nov 16 11:59:44.277800 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:44.277767 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hr6jx" event=&{ID:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Type:ContainerStarted Data:2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a} Nov 16 11:59:44.665661 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:44.665569151Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5973f89c60dd0ee15c6249c7913170a9097194ff6cde0a1d231512ad592d9559\"" Nov 16 11:59:44.678030 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:44.677930252Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:264660c55199b0faa7b19c04a437eb9cd9c6d6824d28fec7526f3656461e7cfe\"" Nov 16 11:59:45.437208 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:45.437073860Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9472b8291bcb75e928fe7853c8c53735fc34d909a79e0185396235205127b3b7\"" Nov 16 11:59:45.445089 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.82DMMY.mount: Succeeded. Nov 16 11:59:49.040723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.040683 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 11:59:49.041214 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.040770 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 11:59:49.051505 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod1f2eae2d_0f19_457b_ba37_10f8a1851015.slice. Nov 16 11:59:49.178754 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.178654 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.178754 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.178754 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmqlq\" (UniqueName: \"kubernetes.io/projected/1f2eae2d-0f19-457b-ba37-10f8a1851015-kube-api-access-qmqlq\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.179014 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.178822 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.179014 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.178942 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f2eae2d-0f19-457b-ba37-10f8a1851015-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.179234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.179200 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.179349 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.179334 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.179402 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.179380 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f2eae2d-0f19-457b-ba37-10f8a1851015-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.179441 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.179410 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-main-proxy\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.179441 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.179439 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f2eae2d-0f19-457b-ba37-10f8a1851015-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.179509 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.179475 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f2eae2d-0f19-457b-ba37-10f8a1851015-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.280471 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.280424 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f2eae2d-0f19-457b-ba37-10f8a1851015-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.280603 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.280534 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.280603 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.280579 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-qmqlq\" (UniqueName: \"kubernetes.io/projected/1f2eae2d-0f19-457b-ba37-10f8a1851015-kube-api-access-qmqlq\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.280705 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.280610 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.280705 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.280642 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f2eae2d-0f19-457b-ba37-10f8a1851015-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.280705 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.280675 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.280810 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.280720 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.280810 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.280758 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f2eae2d-0f19-457b-ba37-10f8a1851015-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.280810 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.280793 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-main-proxy\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.280912 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.280822 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f2eae2d-0f19-457b-ba37-10f8a1851015-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.280912 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.280863 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f2eae2d-0f19-457b-ba37-10f8a1851015-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.281545 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.281513 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f2eae2d-0f19-457b-ba37-10f8a1851015-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.281730 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.281694 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f2eae2d-0f19-457b-ba37-10f8a1851015-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.283325 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.283242 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.283493 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.283463 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-main-proxy\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.283647 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.283617 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.283785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.283759 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.284030 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.284005 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f2eae2d-0f19-457b-ba37-10f8a1851015-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.284274 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.284246 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.307263 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.307153 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmqlq\" (UniqueName: \"kubernetes.io/projected/1f2eae2d-0f19-457b-ba37-10f8a1851015-kube-api-access-qmqlq\") pod \"alertmanager-main-0\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.362024 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:49.361943 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 16 11:59:49.362548 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:49.362488557Z" level=info msg="Running pod sandbox: openshift-monitoring/alertmanager-main-0/POD" id=7540c6ca-0c64-4f4c-a314-e6d5c5adba8f name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 11:59:49.362870 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:49.362573158Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:50.348145 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.K0DMZW.mount: Succeeded. Nov 16 11:59:55.342991 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.uS0Ign.mount: Succeeded. Nov 16 11:59:58.597993 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:58.597870870Z" level=info msg="Got pod network &{Name:alertmanager-main-0 Namespace:openshift-monitoring ID:555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1 UID:1f2eae2d-0f19-457b-ba37-10f8a1851015 NetNS:/var/run/netns/161f6e4e-3505-4174-8213-5102bef5ea3a Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 11:59:58.597993 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:58.597946970Z" level=info msg="Adding pod openshift-monitoring_alertmanager-main-0 to CNI network \"multus-cni-network\" (type=multus)" Nov 16 11:59:58.645197 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 11:59:58.662177 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599998.6621] manager: (veth4df82f16): new Veth device (/org/freedesktop/NetworkManager/Devices/18) Nov 16 11:59:58.662987 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599998.6629] device (veth4df82f16): carrier: link connected Nov 16 11:59:58.678070 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth4df82f16: link is not ready Nov 16 11:59:58.678173 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth4df82f16: link becomes ready Nov 16 11:59:58.678207 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 11:59:58.685456 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[4971]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 11:59:58.685614 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[4971]: Could not generate persistent MAC address for veth4df82f16: No such file or directory Nov 16 11:59:58.706155 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668599998.7061] manager: (veth4df82f16): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/19) Nov 16 11:59:58.711358 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00067|bridge|INFO|bridge br0: added interface veth4df82f16 on port 7 Nov 16 11:59:58.711677 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth4df82f16 entered promiscuous mode Nov 16 11:59:58.745348 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00068|connmgr|INFO|br0<->unix#79: 5 flow_mods in the last 0 s (5 adds) Nov 16 11:59:58.748385 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:58.748345000Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6114d0ffa57dba48271a6d3cca48aada961cf8742ce3d499498343170f9b9c7" id=b5518a48-da34-435a-8fc2-28ad0bcc4061 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:58.749450 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:58.749408108Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6114d0ffa57dba48271a6d3cca48aada961cf8742ce3d499498343170f9b9c7" id=9fb8ac43-d948-4ac5-8f50-27dd4061af5a name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:58.806943 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00069|connmgr|INFO|br0<->unix#84: 2 flow_mods in the last 0 s (2 deletes) Nov 16 11:59:58.807436 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00070|connmgr|INFO|br0<->unix#85: 6 flow_mods in the last 0 s (2 adds, 4 deletes) Nov 16 11:59:59.034120 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.033898745Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a2070d322c3195865dc45d129bebc1ac75b0bdbc36265f4554bfe09785521cac" id=56afd114-3d10-402d-b3d3-991b70c54e58 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:59.035074 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.035002553Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a2070d322c3195865dc45d129bebc1ac75b0bdbc36265f4554bfe09785521cac" id=1578bc62-0cfb-4ccf-99cc-66de5751f610 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.044256 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.044211622Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:a1087e77a2641e277e14af20feabcf7a0199151b8cb4263380a0f28070d6d00f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c6114d0ffa57dba48271a6d3cca48aada961cf8742ce3d499498343170f9b9c7],Size_:300491033,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=9fb8ac43-d948-4ac5-8f50-27dd4061af5a name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.045186 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.045101129Z" level=info msg="Creating container: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-liveness-probe" id=c9681594-83ee-4072-9d8c-a1e8d2cb9d60 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.045272 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.045219730Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:59.059932 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.059874140Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9472b8291bcb75e928fe7853c8c53735fc34d909a79e0185396235205127b3b7" id=6369522a-d986-49de-8303-ff9e21d11e1e name=/runtime.v1.ImageService/PullImage Nov 16 11:59:59.060467 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.060429244Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5973f89c60dd0ee15c6249c7913170a9097194ff6cde0a1d231512ad592d9559" id=29812b9e-c903-4285-a61f-ef95de255e4c name=/runtime.v1.ImageService/PullImage Nov 16 11:59:59.060597 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.060555545Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:264660c55199b0faa7b19c04a437eb9cd9c6d6824d28fec7526f3656461e7cfe" id=394d5b0f-3cd9-408a-8026-f0abea1f1bf3 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:59.060835 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.060801147Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9472b8291bcb75e928fe7853c8c53735fc34d909a79e0185396235205127b3b7" id=f9dcd965-b798-4bb1-93df-43f36df343d8 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.061094 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.061047349Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:2c7db4f2afa3bf77d3eb7f1bd6dac03eaa8988c87e5ec25899b823dec324a8b0,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a2070d322c3195865dc45d129bebc1ac75b0bdbc36265f4554bfe09785521cac],Size_:289232107,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=1578bc62-0cfb-4ccf-99cc-66de5751f610 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.061205 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.061169650Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:264660c55199b0faa7b19c04a437eb9cd9c6d6824d28fec7526f3656461e7cfe" id=372d7854-aeff-4ee0-a679-a5c86413d72b name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.061310 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.061187950Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5973f89c60dd0ee15c6249c7913170a9097194ff6cde0a1d231512ad592d9559" id=153d5ef7-361a-4a54-b7a6-0d1042293dac name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.061801 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.061763854Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-q722j/bond-cni-plugin" id=802669ef-810c-457e-878d-bdf4011fd348 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.061902 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.061865555Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:59.063139 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.063048964Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:c6f0d5e198b0533e28772a341b91ce61bb562f76dcda9f0065fcc6a833b5f8f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9472b8291bcb75e928fe7853c8c53735fc34d909a79e0185396235205127b3b7],Size_:321607089,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=f9dcd965-b798-4bb1-93df-43f36df343d8 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.063988 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.063916170Z" level=info msg="Creating container: openshift-multus/network-metrics-daemon-hr6jx/network-metrics-daemon" id=a2ff511f-8414-4cec-91bd-1ba7314ef368 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.064075 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.064003571Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:59.093272 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.093182490Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d0242a6f8ab8d2f5fa748e55feafbefb0643e787e0dc796574e20cfb9a7162c0,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:264660c55199b0faa7b19c04a437eb9cd9c6d6824d28fec7526f3656461e7cfe],Size_:364801480,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=372d7854-aeff-4ee0-a679-a5c86413d72b name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.093272 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.093199790Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:17456051c7cc06538099675b0495b5f5ee0a3861cf65d754c5aa365a7d397090,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5973f89c60dd0ee15c6249c7913170a9097194ff6cde0a1d231512ad592d9559],Size_:449268556,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=153d5ef7-361a-4a54-b7a6-0d1042293dac name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.094189 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.094149997Z" level=info msg="Creating container: openshift-network-diagnostics/network-check-target-299n8/network-check-target-container" id=5e399e50-f206-474c-8d42-e7e7d16645f8 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.094264 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.094191598Z" level=info msg="Creating container: openshift-ingress-canary/ingress-canary-c77h2/serve-healthcheck-canary" id=51abc0e0-72a3-44bd-bf45-d1f6a38f43d3 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.094264 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.094245798Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:59.094339 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.094273198Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:59.103878 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.103839470Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:adb10bb2eed278eedcfa7aa9e65acce4467d8630ca37593a760788ceda16fc86" id=08c3a308-05f1-454d-ac63-ad6aa1ae19f6 name=/runtime.v1.ImageService/PullImage Nov 16 11:59:59.104588 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.104529675Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:adb10bb2eed278eedcfa7aa9e65acce4467d8630ca37593a760788ceda16fc86" id=78fc795e-3d66-4a82-909f-e7117ae2ce80 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.116172 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016.scope. Nov 16 11:59:59.140804 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016. Nov 16 11:59:59.146244 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.146185888Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:48a4a72a0964c46dcd106eccb452b0d2bff8e92de07b90af394b606aef4f0bef,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:adb10bb2eed278eedcfa7aa9e65acce4467d8630ca37593a760788ceda16fc86],Size_:353034994,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=78fc795e-3d66-4a82-909f-e7117ae2ce80 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.147257 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.147208396Z" level=info msg="Creating container: openshift-dns/dns-default-kg69s/dns" id=1bd157f9-4058-4273-8098-a3df88a6ba1f name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.147343 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.147307997Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:59.157733 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6.scope. Nov 16 11:59:59.170336 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:59.177301 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6. Nov 16 11:59:59.214336 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:59.238574 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.238460681Z" level=info msg="Created container c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-liveness-probe" id=c9681594-83ee-4072-9d8c-a1e8d2cb9d60 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.239080 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.239043686Z" level=info msg="Starting container: c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016" id=cd85a44a-c5ff-4451-964b-e8c581ab58d5 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:59.249876 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.249652865Z" level=info msg="Started container" PID=5013 containerID=c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016 description=openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-liveness-probe id=cd85a44a-c5ff-4451-964b-e8c581ab58d5 name=/runtime.v1.RuntimeService/StartContainer sandboxID=8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883 Nov 16 11:59:59.256758 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4.scope. Nov 16 11:59:59.261595 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc.scope. Nov 16 11:59:59.265685 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c.scope. Nov 16 11:59:59.276817 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4. Nov 16 11:59:59.283476 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.283425119Z" level=info msg="Created container 4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6: openshift-multus/multus-additional-cni-plugins-q722j/bond-cni-plugin" id=802669ef-810c-457e-878d-bdf4011fd348 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.284005 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.283935323Z" level=info msg="Starting container: 4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6" id=f237a7fc-8c33-4dae-a550-5dbeedddb4de name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:59.286209 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc. Nov 16 11:59:59.295168 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.295104607Z" level=info msg="Started container" PID=5032 containerID=4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6 description=openshift-multus/multus-additional-cni-plugins-q722j/bond-cni-plugin id=f237a7fc-8c33-4dae-a550-5dbeedddb4de name=/runtime.v1.RuntimeService/StartContainer sandboxID=8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3 Nov 16 11:59:59.296523 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c. Nov 16 11:59:59.302483 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.299303738Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/bond\": CREATE" Nov 16 11:59:59.309194 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9.scope. Nov 16 11:59:59.311689 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.311616431Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:59.311689 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.311659531Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:59.311689 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.311686331Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/bond\": WRITE" Nov 16 11:59:59.317105 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:59.318349 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6.scope: Succeeded. Nov 16 11:59:59.318990 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6.scope: Consumed 32ms CPU time Nov 16 11:59:59.321359 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:59.319889 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" event=&{ID:89deb745-c538-46ae-9d77-26643bf01a15 Type:ContainerStarted Data:c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016} Nov 16 11:59:59.325464 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.324361926Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:59.325464 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.324396227Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:59.325464 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.324413627Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/bond\": WRITE" Nov 16 11:59:59.323885 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:59.325683 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 11:59:59.324123 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q722j" event=&{ID:13c954d2-615d-4b55-be8a-e712b8fa30a8 Type:ContainerStarted Data:4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6} Nov 16 11:59:59.330894 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:59.331973 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6.scope: Succeeded. Nov 16 11:59:59.332654 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6.scope: Consumed 34ms CPU time Nov 16 11:59:59.335995 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9. Nov 16 11:59:59.339047 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.339000936Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 11:59:59.339047 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.339040737Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 11:59:59.367650 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:59.372017 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.371961884Z" level=info msg="Created container 7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4: openshift-ingress-canary/ingress-canary-c77h2/serve-healthcheck-canary" id=51abc0e0-72a3-44bd-bf45-d1f6a38f43d3 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.377073 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.377008522Z" level=info msg="Starting container: 7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4" id=d419bba8-e669-496c-9c47-afe56c5be02c name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:59.388011 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.387962104Z" level=info msg="Started container" PID=5110 containerID=7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4 description=openshift-ingress-canary/ingress-canary-c77h2/serve-healthcheck-canary id=d419bba8-e669-496c-9c47-afe56c5be02c name=/runtime.v1.RuntimeService/StartContainer sandboxID=4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171 Nov 16 11:59:59.388360 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.388322407Z" level=info msg="Created container 5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c: openshift-network-diagnostics/network-check-target-299n8/network-check-target-container" id=5e399e50-f206-474c-8d42-e7e7d16645f8 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.388898 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.388855911Z" level=info msg="Starting container: 5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c" id=3315ed04-2146-4309-9fb6-f3cae8e2054f name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:59.392046 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.391988034Z" level=info msg="Created container a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc: openshift-multus/network-metrics-daemon-hr6jx/network-metrics-daemon" id=a2ff511f-8414-4cec-91bd-1ba7314ef368 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.392480 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.392439438Z" level=info msg="Starting container: a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc" id=0cd188c6-b98b-4f2f-b354-5d54464c2198 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:59.402049 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.401995610Z" level=info msg="Started container" PID=5136 containerID=5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c description=openshift-network-diagnostics/network-check-target-299n8/network-check-target-container id=3315ed04-2146-4309-9fb6-f3cae8e2054f name=/runtime.v1.RuntimeService/StartContainer sandboxID=b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76 Nov 16 11:59:59.403515 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.403358020Z" level=info msg="Started container" PID=5117 containerID=a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc description=openshift-multus/network-metrics-daemon-hr6jx/network-metrics-daemon id=0cd188c6-b98b-4f2f-b354-5d54464c2198 name=/runtime.v1.RuntimeService/StartContainer sandboxID=2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a Nov 16 11:59:59.421438 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.421386055Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=a2da0dae-da2c-4fb3-85c2-ffabbcd26a5d name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.421685 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.421650857Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=a2da0dae-da2c-4fb3-85c2-ffabbcd26a5d name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.426569 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.426513894Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=979b4c36-6115-4491-b85c-bcf6705e9869 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.426778 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.426741395Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=979b4c36-6115-4491-b85c-bcf6705e9869 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.427718 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.427648902Z" level=info msg="Creating container: openshift-multus/network-metrics-daemon-hr6jx/kube-rbac-proxy" id=2060f036-8303-4898-b5cc-6a08a2da0475 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.427804 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.427751803Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:59.430203 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.430087621Z" level=info msg="Created container a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9: openshift-dns/dns-default-kg69s/dns" id=1bd157f9-4058-4273-8098-a3df88a6ba1f name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.430606 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.430566324Z" level=info msg="Starting container: a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9" id=3c18320c-2cfa-40e6-a594-ab9751357f9b name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:59.442663 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.442573814Z" level=info msg="Started container" PID=5191 containerID=a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9 description=openshift-dns/dns-default-kg69s/dns id=3c18320c-2cfa-40e6-a594-ab9751357f9b name=/runtime.v1.RuntimeService/StartContainer sandboxID=9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f Nov 16 11:59:59.464074 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.464010075Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=27c26bac-5789-4f98-bba6-f9549f3fc438 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.464312 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.464276177Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=27c26bac-5789-4f98-bba6-f9549f3fc438 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.469373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.469327315Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=77ac6a23-0de2-42b4-9c66-820a14cf9522 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.469588 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.469549617Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=77ac6a23-0de2-42b4-9c66-820a14cf9522 name=/runtime.v1.ImageService/ImageStatus Nov 16 11:59:59.469596 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2.scope. Nov 16 11:59:59.471150 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.470333623Z" level=info msg="Creating container: openshift-dns/dns-default-kg69s/kube-rbac-proxy" id=52d0abeb-4828-4360-9485-9e4bed1430a2 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.471150 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.470424524Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 11:59:59.490306 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2. Nov 16 11:59:59.518318 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68.scope. Nov 16 11:59:59.524012 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:59.537576 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68. Nov 16 11:59:59.570503 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 11:59:59.593941 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.593894551Z" level=info msg="Created container 4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2: openshift-multus/network-metrics-daemon-hr6jx/kube-rbac-proxy" id=2060f036-8303-4898-b5cc-6a08a2da0475 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.594645 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.594603456Z" level=info msg="Starting container: 4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2" id=4ccb7c47-b20f-489e-834d-2c7949c58227 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:59.605770 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.605690540Z" level=info msg="Started container" PID=5315 containerID=4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2 description=openshift-multus/network-metrics-daemon-hr6jx/kube-rbac-proxy id=4ccb7c47-b20f-489e-834d-2c7949c58227 name=/runtime.v1.RuntimeService/StartContainer sandboxID=2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a Nov 16 11:59:59.642920 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.642863519Z" level=info msg="Created container ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68: openshift-dns/dns-default-kg69s/kube-rbac-proxy" id=52d0abeb-4828-4360-9485-9e4bed1430a2 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 11:59:59.643274 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.643235622Z" level=info msg="Starting container: ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68" id=dfd25fcc-443d-4bc4-9717-eae05989ce55 name=/runtime.v1.RuntimeService/StartContainer Nov 16 11:59:59.654403 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 11:59:59.654358705Z" level=info msg="Started container" PID=5335 containerID=ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68 description=openshift-dns/dns-default-kg69s/kube-rbac-proxy id=dfd25fcc-443d-4bc4-9717-eae05989ce55 name=/runtime.v1.RuntimeService/StartContainer sandboxID=9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f Nov 16 12:00:00.328730 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:00.328684 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kg69s" event=&{ID:306e3397-e392-42af-b8d9-63e5f5dfae1b Type:ContainerStarted Data:ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68} Nov 16 12:00:00.328730 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:00.328734 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kg69s" event=&{ID:306e3397-e392-42af-b8d9-63e5f5dfae1b Type:ContainerStarted Data:a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9} Nov 16 12:00:00.329208 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:00.328795 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-dns/dns-default-kg69s" Nov 16 12:00:00.329806 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:00.329741 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-299n8" event=&{ID:1bd11256-096f-4190-9ee6-f23c6eaabf65 Type:ContainerStarted Data:5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c} Nov 16 12:00:00.329885 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:00.329854 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-network-diagnostics/network-check-target-299n8" Nov 16 12:00:00.330908 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:00.330876 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c77h2" event=&{ID:b0793e26-e08f-46cd-bee3-f0ec4509d53a Type:ContainerStarted Data:7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4} Nov 16 12:00:00.332489 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:00.332463 1563 generic.go:296] "Generic (PLEG): container finished" podID=13c954d2-615d-4b55-be8a-e712b8fa30a8 containerID="4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6" exitCode=0 Nov 16 12:00:00.332587 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:00.332532 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q722j" event=&{ID:13c954d2-615d-4b55-be8a-e712b8fa30a8 Type:ContainerDied Data:4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6} Nov 16 12:00:00.333080 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:00.333027858Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:521f03e789a084b6b55c27fd78b1c914bc7b0d8ad73b03565fd4c290922a880c" id=0c44c3e6-07cc-4dc4-a5e2-bf37bdd9d87b name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:00.333392 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:00.333353961Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:521f03e789a084b6b55c27fd78b1c914bc7b0d8ad73b03565fd4c290922a880c not found" id=0c44c3e6-07cc-4dc4-a5e2-bf37bdd9d87b name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:00.334379 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:00.333927 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hr6jx" event=&{ID:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Type:ContainerStarted Data:4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2} Nov 16 12:00:00.334379 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:00.333965 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hr6jx" event=&{ID:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Type:ContainerStarted Data:a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc} Nov 16 12:00:00.334564 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:00.333950265Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:521f03e789a084b6b55c27fd78b1c914bc7b0d8ad73b03565fd4c290922a880c" id=28552df5-4a38-4d1d-92ee-72c155c4e16a name=/runtime.v1.ImageService/PullImage Nov 16 12:00:00.337344 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:00.337307990Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:521f03e789a084b6b55c27fd78b1c914bc7b0d8ad73b03565fd4c290922a880c\"" Nov 16 12:00:00.829054 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T12:00:00Z [verbose] Add: openshift-monitoring:alertmanager-main-0:1f2eae2d-0f19-457b-ba37-10f8a1851015:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/161f6e4e-3505-4174-8213-5102bef5ea3a"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.6/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 12:00:00.829054 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 12:00:00.801027 4953 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-monitoring", Name:"alertmanager-main-0", UID:"1f2eae2d-0f19-457b-ba37-10f8a1851015", APIVersion:"v1", ResourceVersion:"25229", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.6/23] from openshift-sdn Nov 16 12:00:00.829980 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:00.829552322Z" level=info msg="Got pod network &{Name:alertmanager-main-0 Namespace:openshift-monitoring ID:555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1 UID:1f2eae2d-0f19-457b-ba37-10f8a1851015 NetNS:/var/run/netns/161f6e4e-3505-4174-8213-5102bef5ea3a Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:00:00.829980 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:00.829692223Z" level=info msg="Checking pod openshift-monitoring_alertmanager-main-0 for CNI network multus-cni-network (type=multus)" Nov 16 12:00:00.831276 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:00.831232 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 12:00:00.831870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 12:00:00.831835 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f2eae2d_0f19_457b_ba37_10f8a1851015.slice/crio-555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1.scope WatchSource:0}: Error finding container 555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1: Status 404 returned error &{%!s(*http.body=&{0xc0010b99f8 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 12:00:00.842664 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:00.842589619Z" level=info msg="Ran pod sandbox 555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1 with infra container: openshift-monitoring/alertmanager-main-0/POD" id=7540c6ca-0c64-4f4c-a314-e6d5c5adba8f name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:00:00.843660 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:00.843586126Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1cc7a9502746854b1057cc817aa8768dc5acabf36638f5d781dd38e16358f44" id=7efb9873-1c05-4215-b768-3eac8e082918 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:00.843919 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:00.843889328Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1cc7a9502746854b1057cc817aa8768dc5acabf36638f5d781dd38e16358f44 not found" id=7efb9873-1c05-4215-b768-3eac8e082918 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:00.844618 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:00.844537333Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1cc7a9502746854b1057cc817aa8768dc5acabf36638f5d781dd38e16358f44" id=063d66fa-95ab-421f-9703-374522437667 name=/runtime.v1.ImageService/PullImage Nov 16 12:00:00.847069 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:00.846999251Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1cc7a9502746854b1057cc817aa8768dc5acabf36638f5d781dd38e16358f44\"" Nov 16 12:00:01.336603 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:01.336560 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerStarted Data:555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1} Nov 16 12:00:01.602270 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:01.602012023Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:521f03e789a084b6b55c27fd78b1c914bc7b0d8ad73b03565fd4c290922a880c\"" Nov 16 12:00:02.666746 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:02.666654580Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1cc7a9502746854b1057cc817aa8768dc5acabf36638f5d781dd38e16358f44\"" Nov 16 12:00:04.149940 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.149785826Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:521f03e789a084b6b55c27fd78b1c914bc7b0d8ad73b03565fd4c290922a880c" id=28552df5-4a38-4d1d-92ee-72c155c4e16a name=/runtime.v1.ImageService/PullImage Nov 16 12:00:04.150994 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.150942734Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:521f03e789a084b6b55c27fd78b1c914bc7b0d8ad73b03565fd4c290922a880c" id=9ad204f3-a163-40a6-8f7d-0efc7840d1a5 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:04.152341 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.152298944Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:75cfe24d4c84067ecc35ed71ba7588effc7b234446dfbb71c3ae410babf9d7a8,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:521f03e789a084b6b55c27fd78b1c914bc7b0d8ad73b03565fd4c290922a880c],Size_:258534248,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=9ad204f3-a163-40a6-8f7d-0efc7840d1a5 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:04.153277 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.153233851Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-q722j/routeoverride-cni" id=a408aaca-1583-4cf4-95a6-5ca31b10c609 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:04.153367 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.153344552Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:00:04.208586 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6.scope. Nov 16 12:00:04.234463 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6. Nov 16 12:00:04.267768 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:00:04.333068 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.333009678Z" level=info msg="Created container ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6: openshift-multus/multus-additional-cni-plugins-q722j/routeoverride-cni" id=a408aaca-1583-4cf4-95a6-5ca31b10c609 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:04.333762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.333718883Z" level=info msg="Starting container: ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6" id=71c05110-b99b-4558-a808-b8dfe86837ae name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:00:04.344721 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.344659664Z" level=info msg="Started container" PID=5530 containerID=ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6 description=openshift-multus/multus-additional-cni-plugins-q722j/routeoverride-cni id=71c05110-b99b-4558-a808-b8dfe86837ae name=/runtime.v1.RuntimeService/StartContainer sandboxID=8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3 Nov 16 12:00:04.348415 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.348360891Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/route-override\": CREATE" Nov 16 12:00:04.361170 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.361072585Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 12:00:04.361170 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.361152986Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 12:00:04.361170 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.361172386Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/route-override\": WRITE" Nov 16 12:00:04.363865 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6.scope: Succeeded. Nov 16 12:00:04.364537 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6.scope: Consumed 33ms CPU time Nov 16 12:00:04.373196 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.373156874Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 12:00:04.373196 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.373192275Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 12:00:04.373317 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.373215475Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/route-override\": WRITE" Nov 16 12:00:04.378473 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6.scope: Succeeded. Nov 16 12:00:04.379109 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6.scope: Consumed 38ms CPU time Nov 16 12:00:04.386121 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.386072070Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 12:00:04.386121 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:04.386113470Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 12:00:05.349815 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:05.349777 1563 generic.go:296] "Generic (PLEG): container finished" podID=13c954d2-615d-4b55-be8a-e712b8fa30a8 containerID="ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6" exitCode=0 Nov 16 12:00:05.349815 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:05.349837 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q722j" event=&{ID:13c954d2-615d-4b55-be8a-e712b8fa30a8 Type:ContainerDied Data:ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6} Nov 16 12:00:05.350962 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:05.350530487Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30a1a116e4715186cf1bbc554f2c01b1a1709d6894659316aa217d27f1435513" id=0adab07d-4742-4034-b883-0a7c98643e97 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:05.350962 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:05.350880090Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30a1a116e4715186cf1bbc554f2c01b1a1709d6894659316aa217d27f1435513 not found" id=0adab07d-4742-4034-b883-0a7c98643e97 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:05.351649 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:05.351595995Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30a1a116e4715186cf1bbc554f2c01b1a1709d6894659316aa217d27f1435513" id=37bf3f20-3350-46e4-b2b3-51f74cb4ccfa name=/runtime.v1.ImageService/PullImage Nov 16 12:00:05.354524 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:05.354486317Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30a1a116e4715186cf1bbc554f2c01b1a1709d6894659316aa217d27f1435513\"" Nov 16 12:00:05.943892 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:05.943742765Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1cc7a9502746854b1057cc817aa8768dc5acabf36638f5d781dd38e16358f44" id=063d66fa-95ab-421f-9703-374522437667 name=/runtime.v1.ImageService/PullImage Nov 16 12:00:05.945056 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:05.944973874Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1cc7a9502746854b1057cc817aa8768dc5acabf36638f5d781dd38e16358f44" id=0f2dd664-9e30-4b53-8b9d-b6431ef895f7 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:05.946717 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:05.946636587Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:6f17b8d125c18ab4ead77265d71e95482e6a683cb1defa6bfed1e816acd1568b,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1cc7a9502746854b1057cc817aa8768dc5acabf36638f5d781dd38e16358f44],Size_:303678072,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=0f2dd664-9e30-4b53-8b9d-b6431ef895f7 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:05.948138 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:05.948041697Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/alertmanager" id=1dff53d6-d68f-4e3e-ac00-509d863ca10a name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:05.948246 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:05.948169398Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:00:06.000375 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f.scope. Nov 16 12:00:06.025476 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f. Nov 16 12:00:06.061938 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:00:06.130911 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:06.130807546Z" level=info msg="Created container b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f: openshift-monitoring/alertmanager-main-0/alertmanager" id=1dff53d6-d68f-4e3e-ac00-509d863ca10a name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:06.131507 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:06.131470151Z" level=info msg="Starting container: b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f" id=865ec454-0512-4d08-8af8-c608ab72f001 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:00:06.144587 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:06.144493747Z" level=info msg="Started container" PID=5682 containerID=b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f description=openshift-monitoring/alertmanager-main-0/alertmanager id=865ec454-0512-4d08-8af8-c608ab72f001 name=/runtime.v1.RuntimeService/StartContainer sandboxID=555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1 Nov 16 12:00:06.169285 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:06.169227529Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afbb7c73da4e694d007aea9c4f9b50e0ff32ce55baba5c82a7ad264a7c423fe6" id=ac6b3fe7-002d-46cf-8388-a9ffb16bc38e name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:06.169556 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:06.169526232Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afbb7c73da4e694d007aea9c4f9b50e0ff32ce55baba5c82a7ad264a7c423fe6 not found" id=ac6b3fe7-002d-46cf-8388-a9ffb16bc38e name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:06.170191 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:06.170119936Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afbb7c73da4e694d007aea9c4f9b50e0ff32ce55baba5c82a7ad264a7c423fe6" id=5d56b5b5-7f38-414c-8b5a-be066a50443d name=/runtime.v1.ImageService/PullImage Nov 16 12:00:06.173762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:06.173669362Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afbb7c73da4e694d007aea9c4f9b50e0ff32ce55baba5c82a7ad264a7c423fe6\"" Nov 16 12:00:06.352985 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:06.352946 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerStarted Data:b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f} Nov 16 12:00:06.566262 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:06.566203559Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30a1a116e4715186cf1bbc554f2c01b1a1709d6894659316aa217d27f1435513\"" Nov 16 12:00:07.409053 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:07.408948878Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afbb7c73da4e694d007aea9c4f9b50e0ff32ce55baba5c82a7ad264a7c423fe6\"" Nov 16 12:00:11.296095 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:11.296050 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kg69s" Nov 16 12:00:14.650062 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.649982416Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30a1a116e4715186cf1bbc554f2c01b1a1709d6894659316aa217d27f1435513" id=37bf3f20-3350-46e4-b2b3-51f74cb4ccfa name=/runtime.v1.ImageService/PullImage Nov 16 12:00:14.650612 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.650576720Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afbb7c73da4e694d007aea9c4f9b50e0ff32ce55baba5c82a7ad264a7c423fe6" id=5d56b5b5-7f38-414c-8b5a-be066a50443d name=/runtime.v1.ImageService/PullImage Nov 16 12:00:14.650961 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.650917823Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30a1a116e4715186cf1bbc554f2c01b1a1709d6894659316aa217d27f1435513" id=4a915567-62a0-4876-bdc5-78594f9737c9 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:14.652719 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.652657237Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:ac9b86543e73f475ab5ce976761e685359be0428c5b37c4cc97fb11a179fbff7,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30a1a116e4715186cf1bbc554f2c01b1a1709d6894659316aa217d27f1435513],Size_:423916388,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=4a915567-62a0-4876-bdc5-78594f9737c9 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:14.653994 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.653932146Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-q722j/whereabouts-cni-bincopy" id=2395cff2-5320-449e-93ff-c4a29f9915c2 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:14.654096 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.654050647Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:00:14.656792 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.656751268Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afbb7c73da4e694d007aea9c4f9b50e0ff32ce55baba5c82a7ad264a7c423fe6" id=98bbf809-3294-4586-af9e-1e66f1141d92 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:14.690461 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.690404530Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:bb96cd8019833a019b9413120d7c8a0214f5a032e44ee448d3d2cfc55791979f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afbb7c73da4e694d007aea9c4f9b50e0ff32ce55baba5c82a7ad264a7c423fe6],Size_:295948669,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=98bbf809-3294-4586-af9e-1e66f1141d92 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:14.691763 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.691721241Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/config-reloader" id=c4c2d0ef-5dac-4aa9-b19c-d8232421005c name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:14.691858 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.691821641Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:00:14.703449 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635.scope. Nov 16 12:00:14.728384 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635. Nov 16 12:00:14.763331 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:00:14.799823 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3.scope. Nov 16 12:00:14.828749 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3-runc.I9lzx1.mount: Succeeded. Nov 16 12:00:14.832281 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3. Nov 16 12:00:14.833670 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.833613445Z" level=info msg="Created container ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635: openshift-multus/multus-additional-cni-plugins-q722j/whereabouts-cni-bincopy" id=2395cff2-5320-449e-93ff-c4a29f9915c2 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:14.834237 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.834200650Z" level=info msg="Starting container: ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635" id=8376cff9-4a67-4fd8-8d32-ee9a2cae6f7f name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:00:14.845514 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.845467638Z" level=info msg="Started container" PID=5834 containerID=ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635 description=openshift-multus/multus-additional-cni-plugins-q722j/whereabouts-cni-bincopy id=8376cff9-4a67-4fd8-8d32-ee9a2cae6f7f name=/runtime.v1.RuntimeService/StartContainer sandboxID=8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3 Nov 16 12:00:14.849193 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.849091966Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/whereabouts\": CREATE" Nov 16 12:00:14.862328 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.862263169Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 12:00:14.862328 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.862320569Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 12:00:14.862328 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.862352269Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/whereabouts\": WRITE" Nov 16 12:00:14.868738 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:00:14.876601 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.876542480Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 12:00:14.876601 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.876594680Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 12:00:14.876786 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.876627680Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/whereabouts\": WRITE" Nov 16 12:00:14.889251 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.889198078Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 12:00:14.889251 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.889247579Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 12:00:14.889411 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.889277379Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/whereabouts\": WRITE" Nov 16 12:00:14.893004 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635.scope: Succeeded. Nov 16 12:00:14.893675 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635.scope: Consumed 59ms CPU time Nov 16 12:00:14.904829 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.904692099Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 12:00:14.904829 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.904735399Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 12:00:14.904829 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.904754399Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/whereabouts\": WRITE" Nov 16 12:00:14.906677 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635.scope: Succeeded. Nov 16 12:00:14.907372 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635.scope: Consumed 37ms CPU time Nov 16 12:00:14.918284 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.918236204Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 12:00:14.918284 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.918276405Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 12:00:14.928318 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.928267882Z" level=info msg="Created container ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3: openshift-monitoring/alertmanager-main-0/config-reloader" id=c4c2d0ef-5dac-4aa9-b19c-d8232421005c name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:14.928905 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.928856587Z" level=info msg="Starting container: ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3" id=ab23a48b-30f0-458d-8fdb-23ad6fb5de87 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:00:14.939083 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.938999966Z" level=info msg="Started container" PID=5867 containerID=ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3 description=openshift-monitoring/alertmanager-main-0/config-reloader id=ab23a48b-30f0-458d-8fdb-23ad6fb5de87 name=/runtime.v1.RuntimeService/StartContainer sandboxID=555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1 Nov 16 12:00:14.958210 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.958161015Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9" id=c7533b5f-8304-436a-9382-b659e27761bf name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:14.958445 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.958409617Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:cd35189fa8b001e06ada495b7c204aa0d2784a2ea23746527811e83e56f376b9,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9],Size_:296071171,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=c7533b5f-8304-436a-9382-b659e27761bf name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:14.962289 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.962248347Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9" id=97fa5ec4-3664-446c-82e9-77d1bad92dc3 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:14.962496 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.962462749Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:cd35189fa8b001e06ada495b7c204aa0d2784a2ea23746527811e83e56f376b9,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9],Size_:296071171,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=97fa5ec4-3664-446c-82e9-77d1bad92dc3 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:14.963711 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.963609558Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/alertmanager-proxy" id=adfbe830-2d00-4757-a1c8-e8b8c1ae4317 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:14.963826 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:14.963781659Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:00:15.020988 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9.scope. Nov 16 12:00:15.041987 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9. Nov 16 12:00:15.074638 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:00:15.134294 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.134230886Z" level=info msg="Created container eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9: openshift-monitoring/alertmanager-main-0/alertmanager-proxy" id=adfbe830-2d00-4757-a1c8-e8b8c1ae4317 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:15.134826 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.134749390Z" level=info msg="Starting container: eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9" id=e12905d7-8e09-4476-b797-8fa90139fe86 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:00:15.146762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.146671483Z" level=info msg="Started container" PID=5963 containerID=eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9 description=openshift-monitoring/alertmanager-main-0/alertmanager-proxy id=e12905d7-8e09-4476-b797-8fa90139fe86 name=/runtime.v1.RuntimeService/StartContainer sandboxID=555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1 Nov 16 12:00:15.185346 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.185197383Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=a36dfd62-cee7-4627-8a5b-b244ee0ab979 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:15.185515 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.185481685Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=a36dfd62-cee7-4627-8a5b-b244ee0ab979 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:15.186435 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.186376592Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=e0208ee3-42b5-47a7-ae0c-0254535f5aec name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:15.186698 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.186599294Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=e0208ee3-42b5-47a7-ae0c-0254535f5aec name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:15.187620 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.187579701Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy" id=f01119ee-9045-4117-a0ff-83a5432eed99 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:15.187699 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.187683902Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:00:15.270622 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34.scope. Nov 16 12:00:15.292555 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34. Nov 16 12:00:15.323808 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:00:15.373145 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:15.373076 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerStarted Data:eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9} Nov 16 12:00:15.373145 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:15.373156 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerStarted Data:ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3} Nov 16 12:00:15.376742 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:15.376717 1563 generic.go:296] "Generic (PLEG): container finished" podID=13c954d2-615d-4b55-be8a-e712b8fa30a8 containerID="ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635" exitCode=0 Nov 16 12:00:15.376829 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:15.376757 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q722j" event=&{ID:13c954d2-615d-4b55-be8a-e712b8fa30a8 Type:ContainerDied Data:ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635} Nov 16 12:00:15.377631 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.377574781Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30a1a116e4715186cf1bbc554f2c01b1a1709d6894659316aa217d27f1435513" id=7bf2b5c3-de1c-4708-980d-3d1905711d2f name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:15.379345 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.379305194Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:ac9b86543e73f475ab5ce976761e685359be0428c5b37c4cc97fb11a179fbff7,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30a1a116e4715186cf1bbc554f2c01b1a1709d6894659316aa217d27f1435513],Size_:423916388,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=7bf2b5c3-de1c-4708-980d-3d1905711d2f name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:15.380084 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.380032800Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30a1a116e4715186cf1bbc554f2c01b1a1709d6894659316aa217d27f1435513" id=3b8255fc-a4e8-44b0-9d00-4d9bbf90caaf name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:15.381672 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.381617512Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:ac9b86543e73f475ab5ce976761e685359be0428c5b37c4cc97fb11a179fbff7,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:30a1a116e4715186cf1bbc554f2c01b1a1709d6894659316aa217d27f1435513],Size_:423916388,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=3b8255fc-a4e8-44b0-9d00-4d9bbf90caaf name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:15.383180 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.383144424Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-q722j/whereabouts-cni" id=bd9eb0c7-049b-49ec-a051-6935d1efc768 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:15.383266 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.383243825Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:00:15.384096 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.384008031Z" level=info msg="Created container 9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy" id=f01119ee-9045-4117-a0ff-83a5432eed99 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:15.384528 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.384489135Z" level=info msg="Starting container: 9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34" id=7809c0fe-199a-4b4d-812b-2533ccc2626e name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:00:15.396504 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.396454028Z" level=info msg="Started container" PID=6033 containerID=9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34 description=openshift-monitoring/alertmanager-main-0/kube-rbac-proxy id=7809c0fe-199a-4b4d-812b-2533ccc2626e name=/runtime.v1.RuntimeService/StartContainer sandboxID=555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1 Nov 16 12:00:15.498523 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.498361221Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=0167b961-c31d-461b-a9de-d6700a4ac52c name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:15.548413 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.548301410Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=0167b961-c31d-461b-a9de-d6700a4ac52c name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:15.549879 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.549839122Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=441ac357-369f-4344-b95c-f2d22a335932 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:15.550093 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.550045524Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=441ac357-369f-4344-b95c-f2d22a335932 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:15.554000 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.553960854Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric" id=80e16cd2-5253-42be-a100-219ac402780b name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:15.554107 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.554071955Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:00:15.645841 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554.scope. Nov 16 12:00:15.665664 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554. Nov 16 12:00:15.681792 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83.scope. Nov 16 12:00:15.719755 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:00:15.721672 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83. Nov 16 12:00:15.754576 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:00:15.788637 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.788539481Z" level=info msg="Created container bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554: openshift-multus/multus-additional-cni-plugins-q722j/whereabouts-cni" id=bd9eb0c7-049b-49ec-a051-6935d1efc768 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:15.789241 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.789208686Z" level=info msg="Starting container: bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554" id=9135eae7-e5ab-493e-87d8-2332f6c76334 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:00:15.800734 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.800645775Z" level=info msg="Started container" PID=6090 containerID=bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554 description=openshift-multus/multus-additional-cni-plugins-q722j/whereabouts-cni id=9135eae7-e5ab-493e-87d8-2332f6c76334 name=/runtime.v1.RuntimeService/StartContainer sandboxID=8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3 Nov 16 12:00:15.804184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.804105902Z" level=info msg="CNI monitoring event \"/etc/kubernetes/cni/net.d/whereabouts.d\": CREATE" Nov 16 12:00:15.817186 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.817043403Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 12:00:15.817186 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.817106003Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 12:00:15.837112 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554.scope: Succeeded. Nov 16 12:00:15.837832 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554.scope: Consumed 51ms CPU time Nov 16 12:00:15.942197 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554.scope: Succeeded. Nov 16 12:00:15.942891 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554.scope: Consumed 39ms CPU time Nov 16 12:00:15.968885 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.968782084Z" level=info msg="Created container b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric" id=80e16cd2-5253-42be-a100-219ac402780b name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:15.969498 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.969462289Z" level=info msg="Starting container: b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83" id=026666b8-f7de-4f72-9f29-11451b7d065c name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:00:15.981361 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:15.981302981Z" level=info msg="Started container" PID=6108 containerID=b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83 description=openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric id=026666b8-f7de-4f72-9f29-11451b7d065c name=/runtime.v1.RuntimeService/StartContainer sandboxID=555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1 Nov 16 12:00:16.000567 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:16.000520031Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e4230f1af4f1da72e3bf7627a3e7203910e5aaf7e6758b7820244b5e6fc021d5" id=8b19dc65-5199-48e3-b1f8-da3cc37d6d9d name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:16.000891 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:16.000845334Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e4230f1af4f1da72e3bf7627a3e7203910e5aaf7e6758b7820244b5e6fc021d5 not found" id=8b19dc65-5199-48e3-b1f8-da3cc37d6d9d name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:16.001565 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:16.001523339Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e4230f1af4f1da72e3bf7627a3e7203910e5aaf7e6758b7820244b5e6fc021d5" id=5d5945e3-102e-4f05-91b6-ce2a866a1105 name=/runtime.v1.ImageService/PullImage Nov 16 12:00:16.006409 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:16.006301176Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e4230f1af4f1da72e3bf7627a3e7203910e5aaf7e6758b7820244b5e6fc021d5\"" Nov 16 12:00:16.382170 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:16.382115 1563 generic.go:296] "Generic (PLEG): container finished" podID=13c954d2-615d-4b55-be8a-e712b8fa30a8 containerID="bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554" exitCode=0 Nov 16 12:00:16.382623 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:16.382201 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q722j" event=&{ID:13c954d2-615d-4b55-be8a-e712b8fa30a8 Type:ContainerDied Data:bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554} Nov 16 12:00:16.382881 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:16.382827783Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1288cd9c16ae683a98a48a1dc79f451f0514648b85101f53d94fc0d0c9ca73b5" id=02fb2d36-19b3-4965-b49b-55284572ec8b name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:16.383120 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:16.383082985Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:46d6874c0fbc2d49165549a5a3bb17887206e64404f99ce6ee1456a174ac3db2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1288cd9c16ae683a98a48a1dc79f451f0514648b85101f53d94fc0d0c9ca73b5],Size_:425617347,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=02fb2d36-19b3-4965-b49b-55284572ec8b name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:16.385747 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:16.385584804Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1288cd9c16ae683a98a48a1dc79f451f0514648b85101f53d94fc0d0c9ca73b5" id=0f0da8c7-6f31-4d39-a508-fef6c823fc9a name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:16.385997 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:16.385961707Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:46d6874c0fbc2d49165549a5a3bb17887206e64404f99ce6ee1456a174ac3db2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1288cd9c16ae683a98a48a1dc79f451f0514648b85101f53d94fc0d0c9ca73b5],Size_:425617347,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=0f0da8c7-6f31-4d39-a508-fef6c823fc9a name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:16.386075 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:16.386051 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerStarted Data:b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83} Nov 16 12:00:16.386138 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:16.386086 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerStarted Data:9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34} Nov 16 12:00:16.386874 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:16.386824713Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-q722j/kube-multus-additional-cni-plugins" id=fb438086-8635-46ed-98d4-687212e64372 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:16.386988 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:16.386959514Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:00:16.590809 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e.scope. Nov 16 12:00:16.616303 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e. Nov 16 12:00:16.697276 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:16.697220625Z" level=info msg="Created container c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e: openshift-multus/multus-additional-cni-plugins-q722j/kube-multus-additional-cni-plugins" id=fb438086-8635-46ed-98d4-687212e64372 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:16.697854 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:16.697813029Z" level=info msg="Starting container: c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e" id=d0a6e49f-7268-4bb4-8d30-4792a88cca90 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:00:16.710762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:16.710686225Z" level=info msg="Started container" PID=6214 containerID=c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e description=openshift-multus/multus-additional-cni-plugins-q722j/kube-multus-additional-cni-plugins id=d0a6e49f-7268-4bb4-8d30-4792a88cca90 name=/runtime.v1.RuntimeService/StartContainer sandboxID=8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3 Nov 16 12:00:17.314986 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:17.314936126Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e4230f1af4f1da72e3bf7627a3e7203910e5aaf7e6758b7820244b5e6fc021d5\"" Nov 16 12:00:17.392503 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:17.392467 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q722j" event=&{ID:13c954d2-615d-4b55-be8a-e712b8fa30a8 Type:ContainerStarted Data:c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e} Nov 16 12:00:17.745622 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[1771]: In idle state; will auto-exit in 62 seconds Nov 16 12:00:17.759635 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: rpm-ostreed.service: Succeeded. Nov 16 12:00:17.760452 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: rpm-ostreed.service: Consumed 85ms CPU time Nov 16 12:00:21.918775 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:21.918606415Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e4230f1af4f1da72e3bf7627a3e7203910e5aaf7e6758b7820244b5e6fc021d5" id=5d5945e3-102e-4f05-91b6-ce2a866a1105 name=/runtime.v1.ImageService/PullImage Nov 16 12:00:21.920161 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:21.919999926Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e4230f1af4f1da72e3bf7627a3e7203910e5aaf7e6758b7820244b5e6fc021d5" id=53291d56-78cd-4004-9795-912414d11a2b name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:21.921712 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:21.921640238Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:cebf4fcd978280fc3b1176a74da8be80bbb11d0bf8bac2f3f104e431b1749586,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e4230f1af4f1da72e3bf7627a3e7203910e5aaf7e6758b7820244b5e6fc021d5],Size_:262947828,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=53291d56-78cd-4004-9795-912414d11a2b name=/runtime.v1.ImageService/ImageStatus Nov 16 12:00:21.923019 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:21.922932548Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/prom-label-proxy" id=8d529e1e-8fe7-4561-bb70-b6d61e86f6a7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:21.923106 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:21.923058948Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:00:21.979141 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982.scope. Nov 16 12:00:22.004367 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982. Nov 16 12:00:22.036315 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:00:22.100412 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:22.100357869Z" level=info msg="Created container f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982: openshift-monitoring/alertmanager-main-0/prom-label-proxy" id=8d529e1e-8fe7-4561-bb70-b6d61e86f6a7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:00:22.101048 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:22.101016074Z" level=info msg="Starting container: f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982" id=a6ca4261-f2c8-4245-a62c-3e961e3972dd name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:00:22.112995 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:00:22.112888962Z" level=info msg="Started container" PID=6321 containerID=f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982 description=openshift-monitoring/alertmanager-main-0/prom-label-proxy id=a6ca4261-f2c8-4245-a62c-3e961e3972dd name=/runtime.v1.RuntimeService/StartContainer sandboxID=555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1 Nov 16 12:00:22.408713 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:22.408661 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerStarted Data:f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982} Nov 16 12:00:29.362699 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:29.362607 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:00:29.397015 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f-runc.WPSVdZ.mount: Succeeded. Nov 16 12:00:29.438022 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:29.437985 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:00:32.121868 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:00:32.121746 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-299n8" Nov 16 12:00:45.571631 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.EIvLx7.mount: Succeeded. Nov 16 12:01:20.305724 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.dQW88h.mount: Succeeded. Nov 16 12:01:25.303136 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.zfWSTj.mount: Succeeded. Nov 16 12:01:30.303979 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.sW1PlX.mount: Succeeded. Nov 16 12:01:35.305207 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.JjCCTx.mount: Succeeded. Nov 16 12:01:40.555009 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:40.554919 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 12:01:40.555501 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:40.555259 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=1f2eae2d-0f19-457b-ba37-10f8a1851015 containerName="alertmanager" containerID="cri-o://b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f" gracePeriod=120 Nov 16 12:01:40.555501 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:40.555288 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=1f2eae2d-0f19-457b-ba37-10f8a1851015 containerName="kube-rbac-proxy-metric" containerID="cri-o://b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83" gracePeriod=120 Nov 16 12:01:40.555501 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:40.555390 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=1f2eae2d-0f19-457b-ba37-10f8a1851015 containerName="prom-label-proxy" containerID="cri-o://f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982" gracePeriod=120 Nov 16 12:01:40.555501 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:40.555406 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=1f2eae2d-0f19-457b-ba37-10f8a1851015 containerName="alertmanager-proxy" containerID="cri-o://eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9" gracePeriod=120 Nov 16 12:01:40.555501 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:40.555412 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=1f2eae2d-0f19-457b-ba37-10f8a1851015 containerName="config-reloader" containerID="cri-o://ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3" gracePeriod=120 Nov 16 12:01:40.555501 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:40.555452 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=1f2eae2d-0f19-457b-ba37-10f8a1851015 containerName="kube-rbac-proxy" containerID="cri-o://9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34" gracePeriod=120 Nov 16 12:01:40.555907 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:40.555859796Z" level=info msg="Stopping container: f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982 (timeout: 120s)" id=9b4b361f-b4bf-4d8a-ab5d-a40a7d898807 name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:01:40.555907 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:40.555926896Z" level=info msg="Stopping container: 9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34 (timeout: 120s)" id=4a74625e-6b62-4da2-9a5c-7dbb6511edd5 name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:01:40.556426 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:40.555968696Z" level=info msg="Stopping container: eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9 (timeout: 120s)" id=36943a79-d38f-4c37-bc25-dc5cc3c3ee49 name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:01:40.556426 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:40.556047397Z" level=info msg="Stopping container: b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83 (timeout: 120s)" id=b92702b5-cffa-4843-8707-6ed7b9553cd5 name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:01:40.556426 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:40.556075097Z" level=info msg="Stopping container: ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3 (timeout: 120s)" id=a332bf42-3b03-47c7-be61-d85a10fc4496 name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:01:40.556426 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:40.556258199Z" level=info msg="Stopping container: b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f (timeout: 120s)" id=7cacd86b-adb1-4144-ad3f-d5986f84ef4c name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:01:40.579280 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982.scope: Succeeded. Nov 16 12:01:40.579902 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982.scope: Consumed 34ms CPU time Nov 16 12:01:40.583428 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9.scope: Succeeded. Nov 16 12:01:40.583986 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9.scope: Consumed 313ms CPU time Nov 16 12:01:40.584254 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34.scope: Succeeded. Nov 16 12:01:40.584785 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34.scope: Consumed 104ms CPU time Nov 16 12:01:40.585111 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9.scope: Succeeded. Nov 16 12:01:40.585701 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9.scope: Consumed 39ms CPU time Nov 16 12:01:40.586986 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83.scope: Succeeded. Nov 16 12:01:40.588045 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83.scope: Consumed 127ms CPU time Nov 16 12:01:40.588344 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3.scope: Succeeded. Nov 16 12:01:40.589022 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3.scope: Consumed 42ms CPU time Nov 16 12:01:40.590006 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3.scope: Succeeded. Nov 16 12:01:40.590742 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3.scope: Consumed 40ms CPU time Nov 16 12:01:40.594712 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34.scope: Succeeded. Nov 16 12:01:40.595312 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34.scope: Consumed 37ms CPU time Nov 16 12:01:40.599604 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83.scope: Succeeded. Nov 16 12:01:40.600196 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83.scope: Consumed 44ms CPU time Nov 16 12:01:40.612980 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982.scope: Succeeded. Nov 16 12:01:40.613654 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982.scope: Consumed 37ms CPU time Nov 16 12:01:40.717843 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-21b884d13662d24c2dbc9b5894abe4749297ad31569881d3f7c2a178179f7737-merged.mount: Succeeded. Nov 16 12:01:40.944734 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-f212b06a3c42dc450649223514ff1fdbface066b60ef7cc105c6e47008954d44-merged.mount: Succeeded. Nov 16 12:01:40.948748 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:40.948637583Z" level=info msg="Stopped container eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9: openshift-monitoring/alertmanager-main-0/alertmanager-proxy" id=36943a79-d38f-4c37-bc25-dc5cc3c3ee49 name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:01:41.039166 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-48d981f1a2b52b0383a9dffed8ed9bca7a219988ca251149b5dded1857033fce-merged.mount: Succeeded. Nov 16 12:01:41.043444 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:41.043338579Z" level=info msg="Stopped container ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3: openshift-monitoring/alertmanager-main-0/config-reloader" id=a332bf42-3b03-47c7-be61-d85a10fc4496 name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:01:41.179894 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:41.179773582Z" level=info msg="Stopped container 9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy" id=4a74625e-6b62-4da2-9a5c-7dbb6511edd5 name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:01:41.249332 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:41.249103491Z" level=info msg="Stopped container b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric" id=b92702b5-cffa-4843-8707-6ed7b9553cd5 name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:01:41.297877 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-5e2e18dca5eb4058ca381f83ff933f849de4da0267f59ba23a01174eec176305-merged.mount: Succeeded. Nov 16 12:01:41.297925 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-ca43d128e9e6d17452348ec46f8eb0b85ca241c751586c7b44d39dc519804ec1-merged.mount: Succeeded. Nov 16 12:01:41.304162 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f.scope: Succeeded. Nov 16 12:01:41.304822 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f.scope: Consumed 485ms CPU time Nov 16 12:01:41.305073 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f.scope: Succeeded. Nov 16 12:01:41.305729 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f.scope: Consumed 39ms CPU time Nov 16 12:01:41.327625 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:41.327549568Z" level=info msg="Stopped container f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982: openshift-monitoring/alertmanager-main-0/prom-label-proxy" id=9b4b361f-b4bf-4d8a-ab5d-a40a7d898807 name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:01:41.414036 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6cc767f3901e71179be63e7dd0426e98bb4c933b04cf3c3c228b918685edfdd6-merged.mount: Succeeded. Nov 16 12:01:41.514415 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:41.514258840Z" level=info msg="Stopped container b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f: openshift-monitoring/alertmanager-main-0/alertmanager" id=7cacd86b-adb1-4144-ad3f-d5986f84ef4c name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:01:41.515002 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:41.514927245Z" level=info msg="Stopping pod sandbox: 555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1" id=034fd0e4-1c6c-4abd-9d10-2984002ded49 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:01:41.515298 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:41.515220647Z" level=info msg="Got pod network &{Name:alertmanager-main-0 Namespace:openshift-monitoring ID:555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1 UID:1f2eae2d-0f19-457b-ba37-10f8a1851015 NetNS:/var/run/netns/161f6e4e-3505-4174-8213-5102bef5ea3a Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:01:41.515403 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:41.515378149Z" level=info msg="Deleting pod openshift-monitoring_alertmanager-main-0 from CNI network \"multus-cni-network\" (type=multus)" Nov 16 12:01:41.589959 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00071|connmgr|INFO|br0<->unix#89: 2 flow_mods in the last 0 s (2 deletes) Nov 16 12:01:41.605528 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.605488 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1f2eae2d-0f19-457b-ba37-10f8a1851015/alertmanager-proxy/0.log" Nov 16 12:01:41.605977 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.605880 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1f2eae2d-0f19-457b-ba37-10f8a1851015/config-reloader/0.log" Nov 16 12:01:41.606318 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.606295 1563 generic.go:296] "Generic (PLEG): container finished" podID=1f2eae2d-0f19-457b-ba37-10f8a1851015 containerID="f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982" exitCode=0 Nov 16 12:01:41.606381 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.606324 1563 generic.go:296] "Generic (PLEG): container finished" podID=1f2eae2d-0f19-457b-ba37-10f8a1851015 containerID="b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83" exitCode=0 Nov 16 12:01:41.606381 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.606338 1563 generic.go:296] "Generic (PLEG): container finished" podID=1f2eae2d-0f19-457b-ba37-10f8a1851015 containerID="9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34" exitCode=0 Nov 16 12:01:41.606381 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.606353 1563 generic.go:296] "Generic (PLEG): container finished" podID=1f2eae2d-0f19-457b-ba37-10f8a1851015 containerID="eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9" exitCode=2 Nov 16 12:01:41.606381 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.606365 1563 generic.go:296] "Generic (PLEG): container finished" podID=1f2eae2d-0f19-457b-ba37-10f8a1851015 containerID="ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3" exitCode=2 Nov 16 12:01:41.606381 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.606377 1563 generic.go:296] "Generic (PLEG): container finished" podID=1f2eae2d-0f19-457b-ba37-10f8a1851015 containerID="b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f" exitCode=0 Nov 16 12:01:41.606529 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.606385 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerDied Data:f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982} Nov 16 12:01:41.606529 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.606420 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerDied Data:b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83} Nov 16 12:01:41.606529 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.606440 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerDied Data:9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34} Nov 16 12:01:41.606529 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.606455 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerDied Data:eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9} Nov 16 12:01:41.606529 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.606468 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerDied Data:ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3} Nov 16 12:01:41.606529 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.606483 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerDied Data:b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f} Nov 16 12:01:41.623730 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00072|connmgr|INFO|br0<->unix#92: 4 flow_mods in the last 0 s (4 deletes) Nov 16 12:01:41.654436 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00073|bridge|INFO|bridge br0: deleted interface veth4df82f16 on port 7 Nov 16 12:01:41.663159 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth4df82f16 left promiscuous mode Nov 16 12:01:41.680152 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T12:01:41Z [verbose] Del: openshift-monitoring:alertmanager-main-0:1f2eae2d-0f19-457b-ba37-10f8a1851015:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 12:01:41.686328 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-161f6e4e\x2d3505\x2d4174\x2d8213\x2d5102bef5ea3a.mount: Succeeded. Nov 16 12:01:41.696480 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-161f6e4e\x2d3505\x2d4174\x2d8213\x2d5102bef5ea3a.mount: Succeeded. Nov 16 12:01:41.712831 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-161f6e4e\x2d3505\x2d4174\x2d8213\x2d5102bef5ea3a.mount: Succeeded. Nov 16 12:01:41.880734 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:41.880659933Z" level=info msg="Stopped pod sandbox: 555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1" id=034fd0e4-1c6c-4abd-9d10-2984002ded49 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:01:41.890195 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.890103 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1f2eae2d-0f19-457b-ba37-10f8a1851015/alertmanager-proxy/0.log" Nov 16 12:01:41.890557 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:41.890532 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1f2eae2d-0f19-457b-ba37-10f8a1851015/config-reloader/0.log" Nov 16 12:01:42.004361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.004213 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-main-tls\") pod \"1f2eae2d-0f19-457b-ba37-10f8a1851015\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " Nov 16 12:01:42.004361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.004380 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f2eae2d-0f19-457b-ba37-10f8a1851015-alertmanager-trusted-ca-bundle\") pod \"1f2eae2d-0f19-457b-ba37-10f8a1851015\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " Nov 16 12:01:42.004604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.004413 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-config-volume\") pod \"1f2eae2d-0f19-457b-ba37-10f8a1851015\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " Nov 16 12:01:42.004604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.004442 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmqlq\" (UniqueName: \"kubernetes.io/projected/1f2eae2d-0f19-457b-ba37-10f8a1851015-kube-api-access-qmqlq\") pod \"1f2eae2d-0f19-457b-ba37-10f8a1851015\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " Nov 16 12:01:42.004604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.004483 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-main-proxy\") pod \"1f2eae2d-0f19-457b-ba37-10f8a1851015\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " Nov 16 12:01:42.004604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.004520 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f2eae2d-0f19-457b-ba37-10f8a1851015-alertmanager-main-db\") pod \"1f2eae2d-0f19-457b-ba37-10f8a1851015\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " Nov 16 12:01:42.004604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.004549 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-kube-rbac-proxy\") pod \"1f2eae2d-0f19-457b-ba37-10f8a1851015\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " Nov 16 12:01:42.004604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.004574 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f2eae2d-0f19-457b-ba37-10f8a1851015-metrics-client-ca\") pod \"1f2eae2d-0f19-457b-ba37-10f8a1851015\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " Nov 16 12:01:42.004604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.004607 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f2eae2d-0f19-457b-ba37-10f8a1851015-tls-assets\") pod \"1f2eae2d-0f19-457b-ba37-10f8a1851015\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " Nov 16 12:01:42.004847 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.004638 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1f2eae2d-0f19-457b-ba37-10f8a1851015\" (UID: \"1f2eae2d-0f19-457b-ba37-10f8a1851015\") " Nov 16 12:01:42.004847 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 12:01:42.004643 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/1f2eae2d-0f19-457b-ba37-10f8a1851015/volumes/kubernetes.io~configmap/alertmanager-trusted-ca-bundle: clearQuota called, but quotas disabled Nov 16 12:01:42.004847 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 12:01:42.004788 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/1f2eae2d-0f19-457b-ba37-10f8a1851015/volumes/kubernetes.io~empty-dir/alertmanager-main-db: clearQuota called, but quotas disabled Nov 16 12:01:42.004954 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.004869 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f2eae2d-0f19-457b-ba37-10f8a1851015-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1f2eae2d-0f19-457b-ba37-10f8a1851015" (UID: "1f2eae2d-0f19-457b-ba37-10f8a1851015"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 12:01:42.004954 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.004900 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f2eae2d-0f19-457b-ba37-10f8a1851015-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1f2eae2d-0f19-457b-ba37-10f8a1851015" (UID: "1f2eae2d-0f19-457b-ba37-10f8a1851015"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Nov 16 12:01:42.004954 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 12:01:42.004898 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/1f2eae2d-0f19-457b-ba37-10f8a1851015/volumes/kubernetes.io~configmap/metrics-client-ca: clearQuota called, but quotas disabled Nov 16 12:01:42.005158 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.005108 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f2eae2d-0f19-457b-ba37-10f8a1851015-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1f2eae2d-0f19-457b-ba37-10f8a1851015" (UID: "1f2eae2d-0f19-457b-ba37-10f8a1851015"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 12:01:42.013713 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.013604 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-config-volume" (OuterVolumeSpecName: "config-volume") pod "1f2eae2d-0f19-457b-ba37-10f8a1851015" (UID: "1f2eae2d-0f19-457b-ba37-10f8a1851015"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 12:01:42.017638 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.017595 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1f2eae2d-0f19-457b-ba37-10f8a1851015" (UID: "1f2eae2d-0f19-457b-ba37-10f8a1851015"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 12:01:42.017736 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.017638 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1f2eae2d-0f19-457b-ba37-10f8a1851015" (UID: "1f2eae2d-0f19-457b-ba37-10f8a1851015"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 12:01:42.017736 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.017683 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f2eae2d-0f19-457b-ba37-10f8a1851015-kube-api-access-qmqlq" (OuterVolumeSpecName: "kube-api-access-qmqlq") pod "1f2eae2d-0f19-457b-ba37-10f8a1851015" (UID: "1f2eae2d-0f19-457b-ba37-10f8a1851015"). InnerVolumeSpecName "kube-api-access-qmqlq". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 12:01:42.018755 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.018709 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-main-proxy" (OuterVolumeSpecName: "secret-alertmanager-main-proxy") pod "1f2eae2d-0f19-457b-ba37-10f8a1851015" (UID: "1f2eae2d-0f19-457b-ba37-10f8a1851015"). InnerVolumeSpecName "secret-alertmanager-main-proxy". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 12:01:42.019548 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.019508 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1f2eae2d-0f19-457b-ba37-10f8a1851015" (UID: "1f2eae2d-0f19-457b-ba37-10f8a1851015"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 12:01:42.019616 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.019581 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f2eae2d-0f19-457b-ba37-10f8a1851015-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1f2eae2d-0f19-457b-ba37-10f8a1851015" (UID: "1f2eae2d-0f19-457b-ba37-10f8a1851015"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 12:01:42.105246 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.105191 1563 reconciler.go:300] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-main-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 12:01:42.105246 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.105246 1563 reconciler.go:300] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f2eae2d-0f19-457b-ba37-10f8a1851015-alertmanager-trusted-ca-bundle\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 12:01:42.105246 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.105269 1563 reconciler.go:300] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-config-volume\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 12:01:42.105523 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.105288 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-qmqlq\" (UniqueName: \"kubernetes.io/projected/1f2eae2d-0f19-457b-ba37-10f8a1851015-kube-api-access-qmqlq\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 12:01:42.105523 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.105308 1563 reconciler.go:300] "Volume detached for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-main-proxy\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 12:01:42.105523 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.105325 1563 reconciler.go:300] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f2eae2d-0f19-457b-ba37-10f8a1851015-alertmanager-main-db\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 12:01:42.105523 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.105342 1563 reconciler.go:300] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-kube-rbac-proxy\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 12:01:42.105523 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.105356 1563 reconciler.go:300] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f2eae2d-0f19-457b-ba37-10f8a1851015-metrics-client-ca\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 12:01:42.105523 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.105372 1563 reconciler.go:300] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f2eae2d-0f19-457b-ba37-10f8a1851015-tls-assets\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 12:01:42.105523 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.105392 1563 reconciler.go:300] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f2eae2d-0f19-457b-ba37-10f8a1851015-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 12:01:42.297879 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-796b5a582f24ec91a0dec022efe6feee7fcfebe52f148e25a995e3ee3ade5b9d-merged.mount: Succeeded. Nov 16 12:01:42.297918 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1-userdata-shm.mount: Succeeded. Nov 16 12:01:42.297928 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-1f2eae2d\x2d0f19\x2d457b\x2dba37\x2d10f8a1851015-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dqmqlq.mount: Succeeded. Nov 16 12:01:42.297938 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-1f2eae2d\x2d0f19\x2d457b\x2dba37\x2d10f8a1851015-volumes-kubernetes.io\x7esecret-config\x2dvolume.mount: Succeeded. Nov 16 12:01:42.297946 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-1f2eae2d\x2d0f19\x2d457b\x2dba37\x2d10f8a1851015-volumes-kubernetes.io\x7eprojected-tls\x2dassets.mount: Succeeded. Nov 16 12:01:42.297955 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-1f2eae2d\x2d0f19\x2d457b\x2dba37\x2d10f8a1851015-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dmain\x2dtls.mount: Succeeded. Nov 16 12:01:42.297965 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-1f2eae2d\x2d0f19\x2d457b\x2dba37\x2d10f8a1851015-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dkube\x2drbac\x2dproxy\x2dmetric.mount: Succeeded. Nov 16 12:01:42.297973 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-1f2eae2d\x2d0f19\x2d457b\x2dba37\x2d10f8a1851015-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dmain\x2dproxy.mount: Succeeded. Nov 16 12:01:42.297988 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-1f2eae2d\x2d0f19\x2d457b\x2dba37\x2d10f8a1851015-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dkube\x2drbac\x2dproxy.mount: Succeeded. Nov 16 12:01:42.603480 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod1f2eae2d_0f19_457b_ba37_10f8a1851015.slice. Nov 16 12:01:42.603543 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod1f2eae2d_0f19_457b_ba37_10f8a1851015.slice: Consumed 1.346s CPU time Nov 16 12:01:42.610980 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.610910 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1f2eae2d-0f19-457b-ba37-10f8a1851015/alertmanager-proxy/0.log" Nov 16 12:01:42.611402 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.611266 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1f2eae2d-0f19-457b-ba37-10f8a1851015/config-reloader/0.log" Nov 16 12:01:42.611682 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.611655 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:1f2eae2d-0f19-457b-ba37-10f8a1851015 Type:ContainerDied Data:555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1} Nov 16 12:01:42.611751 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.611704 1563 scope.go:110] "RemoveContainer" containerID="f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982" Nov 16 12:01:42.613817 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:42.613773022Z" level=info msg="Removing container: f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982" id=07b8d294-2977-499d-9275-bf39065d3dd2 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:01:42.646762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:42.646666364Z" level=info msg="Removed container f54ba4f9f48c6baf34fb33c9e9210350dcc1df319fe4bf78ad75394a4d968982: openshift-monitoring/alertmanager-main-0/prom-label-proxy" id=07b8d294-2977-499d-9275-bf39065d3dd2 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:01:42.647075 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.647040 1563 scope.go:110] "RemoveContainer" containerID="b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83" Nov 16 12:01:42.648090 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:42.648052674Z" level=info msg="Removing container: b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83" id=40526ad2-0eca-4d2c-a08a-daa0afd4be3e name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:01:42.665863 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.665790 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 12:01:42.674697 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.674624 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 12:01:42.678462 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:42.678401897Z" level=info msg="Removed container b2c5d94c5ba86ac9c9191c8272a35e7b52131510bc0e6a49112df282e6444d83: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric" id=40526ad2-0eca-4d2c-a08a-daa0afd4be3e name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:01:42.678745 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.678718 1563 scope.go:110] "RemoveContainer" containerID="9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34" Nov 16 12:01:42.679862 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:42.679823708Z" level=info msg="Removing container: 9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34" id=241f5369-f637-4b56-aec7-d92f6e740de5 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:01:42.713012 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:42.712950951Z" level=info msg="Removed container 9bbbe575d5f397ad84f01fcfc05802f4e54f746314efaf5fc633f31d068d0e34: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy" id=241f5369-f637-4b56-aec7-d92f6e740de5 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:01:42.713437 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.713319 1563 scope.go:110] "RemoveContainer" containerID="eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9" Nov 16 12:01:42.714524 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:42.714461262Z" level=info msg="Removing container: eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9" id=b88df521-cc58-4dc1-b2ea-82b833515940 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:01:42.722035 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.721947 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 12:01:42.722035 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.722002 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 12:01:42.722234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 12:01:42.722086 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="kube-rbac-proxy-metric" Nov 16 12:01:42.722234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.722100 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="kube-rbac-proxy-metric" Nov 16 12:01:42.722234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 12:01:42.722114 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="kube-rbac-proxy" Nov 16 12:01:42.722234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.722123 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="kube-rbac-proxy" Nov 16 12:01:42.722234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 12:01:42.722153 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="prom-label-proxy" Nov 16 12:01:42.722234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.722162 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="prom-label-proxy" Nov 16 12:01:42.722234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 12:01:42.722172 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="alertmanager-proxy" Nov 16 12:01:42.722234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.722180 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="alertmanager-proxy" Nov 16 12:01:42.722234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 12:01:42.722189 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="config-reloader" Nov 16 12:01:42.722234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.722197 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="config-reloader" Nov 16 12:01:42.722234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 12:01:42.722208 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="alertmanager" Nov 16 12:01:42.722234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.722216 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="alertmanager" Nov 16 12:01:42.722558 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.722262 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="alertmanager" Nov 16 12:01:42.722558 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.722274 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="prom-label-proxy" Nov 16 12:01:42.722558 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.722284 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="config-reloader" Nov 16 12:01:42.722558 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.722293 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="kube-rbac-proxy-metric" Nov 16 12:01:42.722558 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.722302 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="alertmanager-proxy" Nov 16 12:01:42.722558 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.722313 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="1f2eae2d-0f19-457b-ba37-10f8a1851015" containerName="kube-rbac-proxy" Nov 16 12:01:42.734531 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-podd002c6da_b2c9_44ec_827f_97e4f83396f4.slice. Nov 16 12:01:42.757598 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:42.757497979Z" level=info msg="Removed container eb9ebb7ec1a9bbed23af14c311baaf6d2dcb3399bf512b123d50cf7e32d3b8b9: openshift-monitoring/alertmanager-main-0/alertmanager-proxy" id=b88df521-cc58-4dc1-b2ea-82b833515940 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:01:42.757834 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.757794 1563 scope.go:110] "RemoveContainer" containerID="ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3" Nov 16 12:01:42.759012 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:42.758976790Z" level=info msg="Removing container: ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3" id=598ecfe0-a770-42b5-a85e-a3f278cfb7f6 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:01:42.793694 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:42.793636044Z" level=info msg="Removed container ef436f0b7c74c50dccfe8992fa829eeb744bf2d0bc5e2739828472e93e467fc3: openshift-monitoring/alertmanager-main-0/config-reloader" id=598ecfe0-a770-42b5-a85e-a3f278cfb7f6 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:01:42.793988 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.793951 1563 scope.go:110] "RemoveContainer" containerID="b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f" Nov 16 12:01:42.795085 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:42.795047055Z" level=info msg="Removing container: b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f" id=e0bf4b7c-4cb8-4f6e-92a7-a628ac9cab94 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:01:42.838354 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:42.838297273Z" level=info msg="Removed container b54b78309c22a5ff12a5a1d86a4caa08fa1f61264fcde336e1ea418c35d9899f: openshift-monitoring/alertmanager-main-0/alertmanager" id=e0bf4b7c-4cb8-4f6e-92a7-a628ac9cab94 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:01:42.911698 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.911543 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:42.911698 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.911651 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-config-volume\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:42.911698 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.911701 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-main-proxy\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:42.911899 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.911741 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d002c6da-b2c9-44ec-827f-97e4f83396f4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:42.911899 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.911769 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:42.911899 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.911798 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d002c6da-b2c9-44ec-827f-97e4f83396f4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:42.911899 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.911861 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdd6h\" (UniqueName: \"kubernetes.io/projected/d002c6da-b2c9-44ec-827f-97e4f83396f4-kube-api-access-bdd6h\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:42.912030 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.911904 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d002c6da-b2c9-44ec-827f-97e4f83396f4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:42.912030 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.911940 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:42.912097 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:42.912032 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d002c6da-b2c9-44ec-827f-97e4f83396f4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.012415 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.012370 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.012415 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.012433 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-config-volume\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.012618 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.012468 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.012618 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.012507 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-main-proxy\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.012618 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.012541 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d002c6da-b2c9-44ec-827f-97e4f83396f4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.012618 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.012574 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d002c6da-b2c9-44ec-827f-97e4f83396f4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.012618 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.012604 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-bdd6h\" (UniqueName: \"kubernetes.io/projected/d002c6da-b2c9-44ec-827f-97e4f83396f4-kube-api-access-bdd6h\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.012765 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.012661 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d002c6da-b2c9-44ec-827f-97e4f83396f4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.012765 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.012705 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.012765 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.012735 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d002c6da-b2c9-44ec-827f-97e4f83396f4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.013259 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.013229 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d002c6da-b2c9-44ec-827f-97e4f83396f4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.013492 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.013455 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d002c6da-b2c9-44ec-827f-97e4f83396f4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.013660 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.013575 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d002c6da-b2c9-44ec-827f-97e4f83396f4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.015473 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.015285 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d002c6da-b2c9-44ec-827f-97e4f83396f4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.015473 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.015389 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.015854 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.015781 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-config-volume\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.015984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.015957 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.016299 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.016274 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.016361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.016300 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-main-proxy\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.042210 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.042101 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdd6h\" (UniqueName: \"kubernetes.io/projected/d002c6da-b2c9-44ec-827f-97e4f83396f4-kube-api-access-bdd6h\") pod \"alertmanager-main-0\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.051833 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:43.051764 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:01:43.052516 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:43.052405546Z" level=info msg="Running pod sandbox: openshift-monitoring/alertmanager-main-0/POD" id=1f0d30dd-87f0-41a1-8949-e43fd97a62ca name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:01:43.052516 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:43.052500547Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:01:43.106374 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:43.106319643Z" level=info msg="Got pod network &{Name:alertmanager-main-0 Namespace:openshift-monitoring ID:765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85 UID:d002c6da-b2c9-44ec-827f-97e4f83396f4 NetNS:/var/run/netns/104df8d8-bf96-4ded-830e-506de358626a Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:01:43.106374 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:43.106364243Z" level=info msg="Adding pod openshift-monitoring_alertmanager-main-0 to CNI network \"multus-cni-network\" (type=multus)" Nov 16 12:01:43.153212 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 12:01:43.166021 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 12:01:43.166952 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[8138]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 12:01:43.167149 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[8138]: Could not generate persistent MAC address for veth8eb10799: No such file or directory Nov 16 12:01:43.167209 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600103.1671] device (veth8eb10799): carrier: link connected Nov 16 12:01:43.167602 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600103.1675] manager: (veth8eb10799): new Veth device (/org/freedesktop/NetworkManager/Devices/20) Nov 16 12:01:43.184668 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600103.1846] manager: (veth8eb10799): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/21) Nov 16 12:01:43.191150 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth8eb10799 entered promiscuous mode Nov 16 12:01:43.191084 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00074|bridge|INFO|bridge br0: added interface veth8eb10799 on port 8 Nov 16 12:01:43.235739 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00075|connmgr|INFO|br0<->unix#95: 5 flow_mods in the last 0 s (5 adds) Nov 16 12:01:43.294957 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00076|connmgr|INFO|br0<->unix#98: 2 flow_mods in the last 0 s (2 deletes) Nov 16 12:01:44.363632 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T12:01:44Z [verbose] Add: openshift-monitoring:alertmanager-main-0:d002c6da-b2c9-44ec-827f-97e4f83396f4:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/104df8d8-bf96-4ded-830e-506de358626a"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.7/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 12:01:44.363632 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 12:01:44.332715 8121 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-monitoring", Name:"alertmanager-main-0", UID:"d002c6da-b2c9-44ec-827f-97e4f83396f4", APIVersion:"v1", ResourceVersion:"26871", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.7/23] from openshift-sdn Nov 16 12:01:44.364117 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:44.364046 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 12:01:44.364405 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.364215247Z" level=info msg="Got pod network &{Name:alertmanager-main-0 Namespace:openshift-monitoring ID:765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85 UID:d002c6da-b2c9-44ec-827f-97e4f83396f4 NetNS:/var/run/netns/104df8d8-bf96-4ded-830e-506de358626a Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:01:44.364405 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.364354548Z" level=info msg="Checking pod openshift-monitoring_alertmanager-main-0 for CNI network multus-cni-network (type=multus)" Nov 16 12:01:44.366791 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 12:01:44.366753 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd002c6da_b2c9_44ec_827f_97e4f83396f4.slice/crio-765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85.scope WatchSource:0}: Error finding container 765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85: Status 404 returned error &{%!s(*http.body=&{0xc001bbeea0 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 12:01:44.375355 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.375316426Z" level=info msg="Ran pod sandbox 765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85 with infra container: openshift-monitoring/alertmanager-main-0/POD" id=1f0d30dd-87f0-41a1-8949-e43fd97a62ca name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:01:44.376401 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.376353734Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1cc7a9502746854b1057cc817aa8768dc5acabf36638f5d781dd38e16358f44" id=1d4cd545-a94b-4a61-8f28-5b5039d75ae6 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:44.376634 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.376601236Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:6f17b8d125c18ab4ead77265d71e95482e6a683cb1defa6bfed1e816acd1568b,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1cc7a9502746854b1057cc817aa8768dc5acabf36638f5d781dd38e16358f44],Size_:303678072,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=1d4cd545-a94b-4a61-8f28-5b5039d75ae6 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:44.377494 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.377458042Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1cc7a9502746854b1057cc817aa8768dc5acabf36638f5d781dd38e16358f44" id=bab83b12-09fa-4918-a26b-57dcd49d20a9 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:44.377699 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.377665743Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:6f17b8d125c18ab4ead77265d71e95482e6a683cb1defa6bfed1e816acd1568b,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1cc7a9502746854b1057cc817aa8768dc5acabf36638f5d781dd38e16358f44],Size_:303678072,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=bab83b12-09fa-4918-a26b-57dcd49d20a9 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:44.378880 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.378804251Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/alertmanager" id=2e1cbce6-d5f1-4e6a-82e3-be47d8e5ac35 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:01:44.378952 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.378908652Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:01:44.439826 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a.scope. Nov 16 12:01:44.484279 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a. Nov 16 12:01:44.518284 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:01:44.582163 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.582067708Z" level=info msg="Created container 62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a: openshift-monitoring/alertmanager-main-0/alertmanager" id=2e1cbce6-d5f1-4e6a-82e3-be47d8e5ac35 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:01:44.582787 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.582715913Z" level=info msg="Starting container: 62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a" id=8fbe9781-8b38-4a19-b995-3fb23690bd92 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:01:44.594681 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:44.594598 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=1f2eae2d-0f19-457b-ba37-10f8a1851015 path="/var/lib/kubelet/pods/1f2eae2d-0f19-457b-ba37-10f8a1851015/volumes" Nov 16 12:01:44.594805 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.594704499Z" level=info msg="Started container" PID=8188 containerID=62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a description=openshift-monitoring/alertmanager-main-0/alertmanager id=8fbe9781-8b38-4a19-b995-3fb23690bd92 name=/runtime.v1.RuntimeService/StartContainer sandboxID=765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85 Nov 16 12:01:44.615791 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.615662249Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afbb7c73da4e694d007aea9c4f9b50e0ff32ce55baba5c82a7ad264a7c423fe6" id=aeac08ab-c320-4f95-a568-37b54860e1b8 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:44.615976 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.615927751Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:bb96cd8019833a019b9413120d7c8a0214f5a032e44ee448d3d2cfc55791979f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afbb7c73da4e694d007aea9c4f9b50e0ff32ce55baba5c82a7ad264a7c423fe6],Size_:295948669,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=aeac08ab-c320-4f95-a568-37b54860e1b8 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:44.618061 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:44.618032 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerStarted Data:62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a} Nov 16 12:01:44.618189 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:44.618076 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerStarted Data:765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85} Nov 16 12:01:44.622426 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.622364697Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afbb7c73da4e694d007aea9c4f9b50e0ff32ce55baba5c82a7ad264a7c423fe6" id=36f9e822-f6c5-48cf-9ad0-6426d6af03a4 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:44.622635 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.622599299Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:bb96cd8019833a019b9413120d7c8a0214f5a032e44ee448d3d2cfc55791979f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afbb7c73da4e694d007aea9c4f9b50e0ff32ce55baba5c82a7ad264a7c423fe6],Size_:295948669,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=36f9e822-f6c5-48cf-9ad0-6426d6af03a4 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:44.623861 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.623779107Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/config-reloader" id=8bd69204-b09b-4f5c-ab2d-3741724e1050 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:01:44.623936 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.623890108Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:01:44.676344 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba.scope. Nov 16 12:01:44.702561 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba. Nov 16 12:01:44.737651 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:01:44.808863 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.808752433Z" level=info msg="Created container 8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba: openshift-monitoring/alertmanager-main-0/config-reloader" id=8bd69204-b09b-4f5c-ab2d-3741724e1050 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:01:44.809441 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.809402538Z" level=info msg="Starting container: 8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba" id=a55182c7-735c-4b56-9767-83a4194c96f4 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:01:44.820960 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.820832620Z" level=info msg="Started container" PID=8232 containerID=8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba description=openshift-monitoring/alertmanager-main-0/config-reloader id=a55182c7-735c-4b56-9767-83a4194c96f4 name=/runtime.v1.RuntimeService/StartContainer sandboxID=765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85 Nov 16 12:01:44.840322 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.840268459Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9" id=3c867665-e0b9-454e-8294-54cb4f4d3d19 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:44.840568 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.840534361Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:cd35189fa8b001e06ada495b7c204aa0d2784a2ea23746527811e83e56f376b9,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9],Size_:296071171,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=3c867665-e0b9-454e-8294-54cb4f4d3d19 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:44.841453 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.841401567Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9" id=3f5cd6dd-c800-443e-a3de-c289a7773976 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:44.841658 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.841622169Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:cd35189fa8b001e06ada495b7c204aa0d2784a2ea23746527811e83e56f376b9,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4330299a44f3533bd3efed707c6213244e911e1de922187a4b7d56936cea46d9],Size_:296071171,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=3f5cd6dd-c800-443e-a3de-c289a7773976 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:44.842705 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.842676476Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/alertmanager-proxy" id=17126b21-ba9f-416e-be50-7244ab8c5d89 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:01:44.842831 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:44.842806577Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:01:44.888038 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953.scope. Nov 16 12:01:44.910055 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953. Nov 16 12:01:44.946414 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:01:45.016626 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.016538322Z" level=info msg="Created container f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953: openshift-monitoring/alertmanager-main-0/alertmanager-proxy" id=17126b21-ba9f-416e-be50-7244ab8c5d89 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:01:45.017235 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.017190327Z" level=info msg="Starting container: f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953" id=f7a20f9d-bcaa-455c-bd99-e1c568bc0aaa name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:01:45.028553 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.028461408Z" level=info msg="Started container" PID=8280 containerID=f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953 description=openshift-monitoring/alertmanager-main-0/alertmanager-proxy id=f7a20f9d-bcaa-455c-bd99-e1c568bc0aaa name=/runtime.v1.RuntimeService/StartContainer sandboxID=765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85 Nov 16 12:01:45.049670 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.049610260Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=0f9a1029-4726-40a0-85ce-ce4624df7cdf name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:45.049915 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.049877961Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=0f9a1029-4726-40a0-85ce-ce4624df7cdf name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:45.050856 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.050810868Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=78ec689a-cc9a-46fe-8987-e6354f05d062 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:45.051076 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.051026870Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=78ec689a-cc9a-46fe-8987-e6354f05d062 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:45.052052 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.052014677Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy" id=5a437a52-cf72-49ef-86b6-ba9f864b44c2 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:01:45.052213 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.052151478Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:01:45.099701 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc.scope. Nov 16 12:01:45.120822 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc. Nov 16 12:01:45.153831 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:01:45.223542 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.223482306Z" level=info msg="Created container dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy" id=5a437a52-cf72-49ef-86b6-ba9f864b44c2 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:01:45.224290 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.224242411Z" level=info msg="Starting container: dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc" id=b5ff361a-d984-4411-9cf4-d56dd5a4928a name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:01:45.235318 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.235269490Z" level=info msg="Started container" PID=8334 containerID=dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc description=openshift-monitoring/alertmanager-main-0/kube-rbac-proxy id=b5ff361a-d984-4411-9cf4-d56dd5a4928a name=/runtime.v1.RuntimeService/StartContainer sandboxID=765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85 Nov 16 12:01:45.259234 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.259178062Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=346682cb-7179-4e8d-a36b-8ed007774e97 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:45.259492 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.259455064Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=346682cb-7179-4e8d-a36b-8ed007774e97 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:45.260449 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.260379770Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f" id=59faa4ef-476f-4f97-a487-0bbc13545338 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:45.260638 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.260601372Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d884d78339fa28109a82f79677f9a140ad223bfcf24cbdf7adefd6a3d9be62f2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d6d9815fdd1a11959ec8b743c95bf856efd4d76e1783b593cc353c7d651b28f],Size_:328309728,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=59faa4ef-476f-4f97-a487-0bbc13545338 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:45.261757 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.261710780Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric" id=39fa1589-32b5-4551-a1d1-37d45d586228 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:01:45.261844 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.261820281Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:01:45.309793 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a.scope. Nov 16 12:01:45.338489 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a. Nov 16 12:01:45.386557 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:01:45.469444 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.469269368Z" level=info msg="Created container be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric" id=39fa1589-32b5-4551-a1d1-37d45d586228 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:01:45.469893 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.469853472Z" level=info msg="Starting container: be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a" id=c989354e-8e9a-4ae2-ac9b-c52ae8d94568 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:01:45.479818 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.479728943Z" level=info msg="Started container" PID=8401 containerID=be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a description=openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric id=c989354e-8e9a-4ae2-ac9b-c52ae8d94568 name=/runtime.v1.RuntimeService/StartContainer sandboxID=765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85 Nov 16 12:01:45.499000 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.498951680Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e4230f1af4f1da72e3bf7627a3e7203910e5aaf7e6758b7820244b5e6fc021d5" id=542ea1b5-9ead-41d6-8de6-89e5077304dc name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:45.499481 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.499439884Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:cebf4fcd978280fc3b1176a74da8be80bbb11d0bf8bac2f3f104e431b1749586,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e4230f1af4f1da72e3bf7627a3e7203910e5aaf7e6758b7820244b5e6fc021d5],Size_:262947828,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=542ea1b5-9ead-41d6-8de6-89e5077304dc name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:45.500458 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.500384691Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e4230f1af4f1da72e3bf7627a3e7203910e5aaf7e6758b7820244b5e6fc021d5" id=34473db1-ed59-44e7-b295-a7822c7c146d name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:45.500632 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.500597692Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:cebf4fcd978280fc3b1176a74da8be80bbb11d0bf8bac2f3f104e431b1749586,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e4230f1af4f1da72e3bf7627a3e7203910e5aaf7e6758b7820244b5e6fc021d5],Size_:262947828,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=34473db1-ed59-44e7-b295-a7822c7c146d name=/runtime.v1.ImageService/ImageStatus Nov 16 12:01:45.501476 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.501441898Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/prom-label-proxy" id=612d0912-558c-424b-837d-744e2196dcca name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:01:45.501565 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.501542799Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:01:45.555524 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f.scope. Nov 16 12:01:45.584233 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f. Nov 16 12:01:45.617244 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:01:45.623299 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:45.623264 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerStarted Data:be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a} Nov 16 12:01:45.623745 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:45.623310 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerStarted Data:dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc} Nov 16 12:01:45.623745 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:45.623328 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerStarted Data:f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953} Nov 16 12:01:45.623745 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:45.623345 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerStarted Data:8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba} Nov 16 12:01:45.682972 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.682885199Z" level=info msg="Created container d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f: openshift-monitoring/alertmanager-main-0/prom-label-proxy" id=612d0912-558c-424b-837d-744e2196dcca name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:01:45.683649 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.683580704Z" level=info msg="Starting container: d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f" id=8587e7c7-b09d-4738-8ba3-10dca8bf72f8 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:01:45.695564 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:01:45.695440789Z" level=info msg="Started container" PID=8449 containerID=d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f description=openshift-monitoring/alertmanager-main-0/prom-label-proxy id=8587e7c7-b09d-4738-8ba3-10dca8bf72f8 name=/runtime.v1.RuntimeService/StartContainer sandboxID=765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85 Nov 16 12:01:46.628945 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:46.628903 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerStarted Data:d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f} Nov 16 12:01:53.052670 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:01:53.052522 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:02:13.120777 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:02:13.120670 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/alertmanager-main-0" Nov 16 12:02:20.332716 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:02:20.332665517Z" level=info msg="Stopping pod sandbox: 555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1" id=f3523960-1a72-4b75-b596-3286f5b985de name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:02:20.332716 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:02:20.332719017Z" level=info msg="Stopped pod sandbox (already stopped): 555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1" id=f3523960-1a72-4b75-b596-3286f5b985de name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:02:20.333351 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:02:20.333269321Z" level=info msg="Removing pod sandbox: 555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1" id=7aea914d-b30c-4d2c-97f2-58355688336d name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 12:02:20.353357 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:02:20.353274175Z" level=info msg="Removed pod sandbox: 555f34cee3e05d5b0e1008458d44ce65e85b2ca4e7630ef5af711793e0b773c1" id=7aea914d-b30c-4d2c-97f2-58355688336d name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 12:03:20.325576 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:03:20.325511735Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=7ced0205-d9cb-4cd2-86d9-968b5f1590f6 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:03:20.335742 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:03:20.325771538Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=7ced0205-d9cb-4cd2-86d9-968b5f1590f6 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:04:25.304215 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.oODgTN.mount: Succeeded. Nov 16 12:04:30.302683 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.KqaTMi.mount: Succeeded. Nov 16 12:05:09.727721 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:05:09.727449 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-pgwdx_45cd7283-fcc9-4e61-b2b1-517445b7bd5a/sdn/0.log" Nov 16 12:05:09.906513 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:05:09.906472 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-pgwdx_45cd7283-fcc9-4e61-b2b1-517445b7bd5a/kube-rbac-proxy/0.log" Nov 16 12:05:10.110662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:05:10.110527 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-pgwdx_45cd7283-fcc9-4e61-b2b1-517445b7bd5a/drop-icmp/0.log" Nov 16 12:05:25.303298 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.1KHw3l.mount: Succeeded. Nov 16 12:05:30.303685 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.yqmS7p.mount: Succeeded. Nov 16 12:06:25.304437 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.ctBzON.mount: Succeeded. Nov 16 12:07:10.303594 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.CY4767.mount: Succeeded. Nov 16 12:07:35.307219 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.JfDKZ7.mount: Succeeded. Nov 16 12:08:00.304321 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting Cleanup of Temporary Directories... Nov 16 12:08:00.327697 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[13635]: [/usr/lib/tmpfiles.d/pkg-dbus-daemon.conf:1] Duplicate line for path "/var/lib/dbus", ignoring. Nov 16 12:08:00.335656 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[13635]: [/usr/lib/tmpfiles.d/tmp.conf:12] Duplicate line for path "/var/tmp", ignoring. Nov 16 12:08:00.335759 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[13635]: [/usr/lib/tmpfiles.d/var.conf:14] Duplicate line for path "/var/log", ignoring. Nov 16 12:08:00.336312 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[13635]: [/usr/lib/tmpfiles.d/var.conf:19] Duplicate line for path "/var/cache", ignoring. Nov 16 12:08:00.336325 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[13635]: [/usr/lib/tmpfiles.d/var.conf:21] Duplicate line for path "/var/lib", ignoring. Nov 16 12:08:00.336330 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-tmpfiles[13635]: [/usr/lib/tmpfiles.d/var.conf:23] Duplicate line for path "/var/spool", ignoring. Nov 16 12:08:00.389935 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-tmpfiles-clean.service: Succeeded. Nov 16 12:08:00.390641 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Cleanup of Temporary Directories. Nov 16 12:08:00.390807 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: systemd-tmpfiles-clean.service: Consumed 23ms CPU time Nov 16 12:08:20.304959 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.J90tdW.mount: Succeeded. Nov 16 12:08:20.329292 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:08:20.329120913Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=6f55cc41-de4e-49e5-8169-f33a3e0a9a94 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:08:20.408801 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:08:20.329390815Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=6f55cc41-de4e-49e5-8169-f33a3e0a9a94 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:10:00.303394 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.HYSWlL.mount: Succeeded. Nov 16 12:12:06.543982 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:06.543930 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-s74pp] Nov 16 12:12:06.543982 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:06.543984 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 12:12:06.554972 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod4b112725_da76_4917_a9b7_8d3bf5fa3423.slice. Nov 16 12:12:06.592786 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:06.592729 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-s74pp] Nov 16 12:12:06.619081 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:06.619023 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-s6mgf] Nov 16 12:12:06.619081 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:06.619059 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 12:12:06.628260 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-podba736212_a121_4f49_a957_3f6bdc1c986d.slice. Nov 16 12:12:06.686185 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:06.686148 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzbr5\" (UniqueName: \"kubernetes.io/projected/4b112725-da76-4917-a9b7-8d3bf5fa3423-kube-api-access-rzbr5\") pod \"qe-app-registry-s74pp\" (UID: \"4b112725-da76-4917-a9b7-8d3bf5fa3423\") " pod="openshift-marketplace/qe-app-registry-s74pp" Nov 16 12:12:06.686274 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:06.686210 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrcl8\" (UniqueName: \"kubernetes.io/projected/ba736212-a121-4f49-a957-3f6bdc1c986d-kube-api-access-qrcl8\") pod \"qe-app-registry-s6mgf\" (UID: \"ba736212-a121-4f49-a957-3f6bdc1c986d\") " pod="openshift-marketplace/qe-app-registry-s6mgf" Nov 16 12:12:06.787270 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:06.787213 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-rzbr5\" (UniqueName: \"kubernetes.io/projected/4b112725-da76-4917-a9b7-8d3bf5fa3423-kube-api-access-rzbr5\") pod \"qe-app-registry-s74pp\" (UID: \"4b112725-da76-4917-a9b7-8d3bf5fa3423\") " pod="openshift-marketplace/qe-app-registry-s74pp" Nov 16 12:12:06.787396 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:06.787317 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-qrcl8\" (UniqueName: \"kubernetes.io/projected/ba736212-a121-4f49-a957-3f6bdc1c986d-kube-api-access-qrcl8\") pod \"qe-app-registry-s6mgf\" (UID: \"ba736212-a121-4f49-a957-3f6bdc1c986d\") " pod="openshift-marketplace/qe-app-registry-s6mgf" Nov 16 12:12:06.815175 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:06.815036 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzbr5\" (UniqueName: \"kubernetes.io/projected/4b112725-da76-4917-a9b7-8d3bf5fa3423-kube-api-access-rzbr5\") pod \"qe-app-registry-s74pp\" (UID: \"4b112725-da76-4917-a9b7-8d3bf5fa3423\") " pod="openshift-marketplace/qe-app-registry-s74pp" Nov 16 12:12:06.817653 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:06.817590 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrcl8\" (UniqueName: \"kubernetes.io/projected/ba736212-a121-4f49-a957-3f6bdc1c986d-kube-api-access-qrcl8\") pod \"qe-app-registry-s6mgf\" (UID: \"ba736212-a121-4f49-a957-3f6bdc1c986d\") " pod="openshift-marketplace/qe-app-registry-s6mgf" Nov 16 12:12:06.865814 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:06.865732 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-s74pp" Nov 16 12:12:06.866603 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:06.866443748Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-s74pp/POD" id=6061d8ce-5dc8-4eef-a5ec-bf75696837b6 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:12:06.866971 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:06.866626150Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:12:06.915476 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:06.915427618Z" level=info msg="Got pod network &{Name:qe-app-registry-s74pp Namespace:openshift-marketplace ID:54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02 UID:4b112725-da76-4917-a9b7-8d3bf5fa3423 NetNS:/var/run/netns/48caaa7d-60a5-4a76-82be-e5bcf44693bb Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:12:06.915476 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:06.915467619Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-s74pp to CNI network \"multus-cni-network\" (type=multus)" Nov 16 12:12:06.941344 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:06.941254 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-s6mgf" Nov 16 12:12:06.941844 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:06.941758217Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-s6mgf/POD" id=21335739-4d0e-4d3c-b49d-6e8b7b78de71 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:12:06.941844 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:06.941834018Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:12:06.956910 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 12:12:06.967179 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600726.9671] manager: (veth0b926f1d): new Veth device (/org/freedesktop/NetworkManager/Devices/22) Nov 16 12:12:06.968182 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600726.9681] device (veth0b926f1d): carrier: link connected Nov 16 12:12:06.972200 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth0b926f1d: link is not ready Nov 16 12:12:06.972274 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth0b926f1d: link becomes ready Nov 16 12:12:06.985204 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 12:12:06.985369 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[16822]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 12:12:06.985508 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[16822]: Could not generate persistent MAC address for veth0b926f1d: No such file or directory Nov 16 12:12:07.018869 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600727.0188] manager: (veth0b926f1d): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/23) Nov 16 12:12:07.024763 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth0b926f1d entered promiscuous mode Nov 16 12:12:07.024712 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00077|bridge|INFO|bridge br0: added interface veth0b926f1d on port 9 Nov 16 12:12:07.055625 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00078|connmgr|INFO|br0<->unix#106: 5 flow_mods in the last 0 s (5 adds) Nov 16 12:12:07.058250 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:07.058205396Z" level=info msg="Got pod network &{Name:qe-app-registry-s6mgf Namespace:openshift-marketplace ID:50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc UID:ba736212-a121-4f49-a957-3f6bdc1c986d NetNS:/var/run/netns/158d06fe-08de-4558-9e81-9effea3a818d Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:12:07.058250 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:07.058246597Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-s6mgf to CNI network \"multus-cni-network\" (type=multus)" Nov 16 12:12:07.103227 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 12:12:07.105436 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00079|connmgr|INFO|br0<->unix#110: 6 flow_mods in the last 0 s (2 adds, 4 deletes) Nov 16 12:12:07.106835 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00080|connmgr|INFO|br0<->unix#112: 2 flow_mods in the last 0 s (2 deletes) Nov 16 12:12:07.114070 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600727.1140] manager: (veth910dcc54): new Veth device (/org/freedesktop/NetworkManager/Devices/24) Nov 16 12:12:07.115031 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600727.1150] device (veth910dcc54): carrier: link connected Nov 16 12:12:07.120774 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[16860]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 12:12:07.120919 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[16860]: Could not generate persistent MAC address for veth910dcc54: No such file or directory Nov 16 12:12:07.121206 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 12:12:07.131805 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600727.1317] manager: (veth910dcc54): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/25) Nov 16 12:12:07.136930 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00081|bridge|INFO|bridge br0: added interface veth910dcc54 on port 10 Nov 16 12:12:07.137148 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth910dcc54 entered promiscuous mode Nov 16 12:12:07.167188 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00082|connmgr|INFO|br0<->unix#115: 5 flow_mods in the last 0 s (5 adds) Nov 16 12:12:07.207570 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00083|connmgr|INFO|br0<->unix#118: 2 flow_mods in the last 0 s (2 deletes) Nov 16 12:12:08.574770 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T12:12:08Z [verbose] Add: openshift-marketplace:qe-app-registry-s74pp:4b112725-da76-4917-a9b7-8d3bf5fa3423:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/48caaa7d-60a5-4a76-82be-e5bcf44693bb"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.8/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 12:12:08.574770 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 12:12:08.551313 16804 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-s74pp", UID:"4b112725-da76-4917-a9b7-8d3bf5fa3423", APIVersion:"v1", ResourceVersion:"31777", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.8/23] from openshift-sdn Nov 16 12:12:08.575258 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:08.575033 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-s74pp] Nov 16 12:12:08.575519 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:08.575293351Z" level=info msg="Got pod network &{Name:qe-app-registry-s74pp Namespace:openshift-marketplace ID:54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02 UID:4b112725-da76-4917-a9b7-8d3bf5fa3423 NetNS:/var/run/netns/48caaa7d-60a5-4a76-82be-e5bcf44693bb Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:12:08.575519 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:08.575440252Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-s74pp for CNI network multus-cni-network (type=multus)" Nov 16 12:12:08.578076 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 12:12:08.578012 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b112725_da76_4917_a9b7_8d3bf5fa3423.slice/crio-54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02.scope WatchSource:0}: Error finding container 54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02: Status 404 returned error &{%!s(*http.body=&{0xc001c6f1a0 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 12:12:08.589926 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:08.589852061Z" level=info msg="Ran pod sandbox 54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02 with infra container: openshift-marketplace/qe-app-registry-s74pp/POD" id=6061d8ce-5dc8-4eef-a5ec-bf75696837b6 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:12:08.590888 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:08.590849069Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=df978fd2-6315-4e41-b0a4-b5d9e653890a name=/runtime.v1.ImageService/ImageStatus Nov 16 12:12:08.591117 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:08.591089470Z" level=info msg="Image quay.io/openshift-qe-optional-operators/aosqe-index:v4.10 not found" id=df978fd2-6315-4e41-b0a4-b5d9e653890a name=/runtime.v1.ImageService/ImageStatus Nov 16 12:12:08.591327 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:08.591300 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 12:12:08.592041 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:08.591957777Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=295691db-502c-4836-bf3e-c8818d500f8d name=/runtime.v1.ImageService/PullImage Nov 16 12:12:08.614121 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:08.614095544Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 12:12:09.173153 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:09.173101 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-s74pp" event=&{ID:4b112725-da76-4917-a9b7-8d3bf5fa3423 Type:ContainerStarted Data:54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02} Nov 16 12:12:09.173946 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:09.173878 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-marketplace/qe-app-registry-s6mgf] Nov 16 12:12:09.186994 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T12:12:09Z [verbose] Add: openshift-marketplace:qe-app-registry-s6mgf:ba736212-a121-4f49-a957-3f6bdc1c986d:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/158d06fe-08de-4558-9e81-9effea3a818d"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.9/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 12:12:09.186994 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 12:12:09.150088 16839 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-s6mgf", UID:"ba736212-a121-4f49-a957-3f6bdc1c986d", APIVersion:"v1", ResourceVersion:"31785", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.9/23] from openshift-sdn Nov 16 12:12:09.187566 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:09.187495974Z" level=info msg="Got pod network &{Name:qe-app-registry-s6mgf Namespace:openshift-marketplace ID:50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc UID:ba736212-a121-4f49-a957-3f6bdc1c986d NetNS:/var/run/netns/158d06fe-08de-4558-9e81-9effea3a818d Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:12:09.187648 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:09.187631575Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-s6mgf for CNI network multus-cni-network (type=multus)" Nov 16 12:12:09.189929 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 12:12:09.189895 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba736212_a121_4f49_a957_3f6bdc1c986d.slice/crio-50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc.scope WatchSource:0}: Error finding container 50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc: Status 404 returned error &{%!s(*http.body=&{0xc0019cb2a8 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 12:12:09.197436 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:09.197386248Z" level=info msg="Ran pod sandbox 50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc with infra container: openshift-marketplace/qe-app-registry-s6mgf/POD" id=21335739-4d0e-4d3c-b49d-6e8b7b78de71 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:12:09.198244 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:09.198208654Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=965cefba-b3ad-486b-abca-e42451086bf5 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:12:09.198483 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:09.198441656Z" level=info msg="Image quay.io/openshift-qe-optional-operators/aosqe-index:v4.10 not found" id=965cefba-b3ad-486b-abca-e42451086bf5 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:12:09.199300 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:09.199260362Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=e4e27359-b938-4442-9047-b4eed5806d75 name=/runtime.v1.ImageService/PullImage Nov 16 12:12:09.202192 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:09.202158184Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 12:12:10.176916 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:10.176859 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-s6mgf" event=&{ID:ba736212-a121-4f49-a957-3f6bdc1c986d Type:ContainerStarted Data:50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc} Nov 16 12:12:10.242938 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:10.242860142Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 12:12:10.830718 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:10.830682081Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 12:12:19.917875 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:19.917739793Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=295691db-502c-4836-bf3e-c8818d500f8d name=/runtime.v1.ImageService/PullImage Nov 16 12:12:19.918987 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:19.918880102Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=53d20153-8fcc-4c03-be1e-759499df31ab name=/runtime.v1.ImageService/ImageStatus Nov 16 12:12:19.920317 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:19.920276412Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=53d20153-8fcc-4c03-be1e-759499df31ab name=/runtime.v1.ImageService/ImageStatus Nov 16 12:12:19.921496 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:19.921459021Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-s74pp/registry-server" id=d78563ed-4856-4315-bacc-8cb4107292d3 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:12:19.921588 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:19.921561622Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:12:19.963427 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:19.963319237Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=e4e27359-b938-4442-9047-b4eed5806d75 name=/runtime.v1.ImageService/PullImage Nov 16 12:12:19.964335 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:19.964268545Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=4a192879-4c86-4163-b56e-1e8c303bb73b name=/runtime.v1.ImageService/ImageStatus Nov 16 12:12:19.965892 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:19.965737456Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=4a192879-4c86-4163-b56e-1e8c303bb73b name=/runtime.v1.ImageService/ImageStatus Nov 16 12:12:19.966995 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:19.966880964Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-s6mgf/registry-server" id=9ee6a0f7-b17f-4084-ba11-d06dacef0e58 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:12:19.967077 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:19.967021065Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:12:19.977204 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a.scope. Nov 16 12:12:20.001094 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a. Nov 16 12:12:20.018865 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259.scope. Nov 16 12:12:20.034728 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:20.043831 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259. Nov 16 12:12:20.079910 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:20.098940 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:20.098878961Z" level=info msg="Created container de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a: openshift-marketplace/qe-app-registry-s74pp/registry-server" id=d78563ed-4856-4315-bacc-8cb4107292d3 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:12:20.099577 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:20.099544166Z" level=info msg="Starting container: de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a" id=ab535514-232f-4293-b2cb-a00ab87a9b85 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:12:20.110561 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:20.110440948Z" level=info msg="Started container" PID=17126 containerID=de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a description=openshift-marketplace/qe-app-registry-s74pp/registry-server id=ab535514-232f-4293-b2cb-a00ab87a9b85 name=/runtime.v1.RuntimeService/StartContainer sandboxID=54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02 Nov 16 12:12:20.134629 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:20.134581131Z" level=info msg="Created container c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259: openshift-marketplace/qe-app-registry-s6mgf/registry-server" id=9ee6a0f7-b17f-4084-ba11-d06dacef0e58 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:12:20.135016 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:20.134979134Z" level=info msg="Starting container: c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259" id=2e178010-fcc1-4c2c-884a-6fab278d926e name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:12:20.144939 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:20.144899608Z" level=info msg="Started container" PID=17145 containerID=c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259 description=openshift-marketplace/qe-app-registry-s6mgf/registry-server id=2e178010-fcc1-4c2c-884a-6fab278d926e name=/runtime.v1.RuntimeService/StartContainer sandboxID=50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc Nov 16 12:12:20.208688 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:20.208542 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-s6mgf" event=&{ID:ba736212-a121-4f49-a957-3f6bdc1c986d Type:ContainerStarted Data:c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259} Nov 16 12:12:20.209694 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:20.209662 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-s74pp" event=&{ID:4b112725-da76-4917-a9b7-8d3bf5fa3423 Type:ContainerStarted Data:de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a} Nov 16 12:12:20.209895 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:20.209863 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-s74pp" podUID=4b112725-da76-4917-a9b7-8d3bf5fa3423 containerName="registry-server" containerID="cri-o://de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a" gracePeriod=1 Nov 16 12:12:20.210103 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:20.210058000Z" level=info msg="Stopping container: de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a (timeout: 1s)" id=c8edfd6d-0d56-4d1c-b1b5-9a4d151c2a42 name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:12:20.238694 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a.scope: Succeeded. Nov 16 12:12:20.239374 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a.scope: Consumed 160ms CPU time Nov 16 12:12:20.239672 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a.scope: Succeeded. Nov 16 12:12:20.240332 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a.scope: Consumed 35ms CPU time Nov 16 12:12:20.970008 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.E3dUKL.mount: Succeeded. Nov 16 12:12:20.970059 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-296b066186220f707fcc52de91e593a5bca0f3d18e303879bbc7f65939c79fe7-merged.mount: Succeeded. Nov 16 12:12:21.213070 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:21.213031 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_qe-app-registry-s74pp_4b112725-da76-4917-a9b7-8d3bf5fa3423/registry-server/0.log" Nov 16 12:12:21.213070 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:21.213092 1563 generic.go:296] "Generic (PLEG): container finished" podID=4b112725-da76-4917-a9b7-8d3bf5fa3423 containerID="de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a" exitCode=2 Nov 16 12:12:21.213652 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:21.213288 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-s74pp" event=&{ID:4b112725-da76-4917-a9b7-8d3bf5fa3423 Type:ContainerDied Data:de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a} Nov 16 12:12:25.347895 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.B4h0If.mount: Succeeded. Nov 16 12:12:25.786918 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:25.786659007Z" level=info msg="Stopped container de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a: openshift-marketplace/qe-app-registry-s74pp/registry-server" id=c8edfd6d-0d56-4d1c-b1b5-9a4d151c2a42 name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:12:25.787531 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:25.787259212Z" level=info msg="Stopping pod sandbox: 54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02" id=afedc986-419f-4aae-a34d-9d1bf79c61d9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:12:25.787627 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:25.787592914Z" level=info msg="Got pod network &{Name:qe-app-registry-s74pp Namespace:openshift-marketplace ID:54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02 UID:4b112725-da76-4917-a9b7-8d3bf5fa3423 NetNS:/var/run/netns/48caaa7d-60a5-4a76-82be-e5bcf44693bb Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:12:25.787799 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:25.787775016Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-s74pp from CNI network \"multus-cni-network\" (type=multus)" Nov 16 12:12:25.854461 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00084|connmgr|INFO|br0<->unix#121: 2 flow_mods in the last 0 s (2 deletes) Nov 16 12:12:25.883294 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00085|connmgr|INFO|br0<->unix#124: 4 flow_mods in the last 0 s (4 deletes) Nov 16 12:12:25.910208 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00086|bridge|INFO|bridge br0: deleted interface veth0b926f1d on port 9 Nov 16 12:12:25.916200 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth0b926f1d left promiscuous mode Nov 16 12:12:26.286538 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T12:12:25Z [verbose] Del: openshift-marketplace:qe-app-registry-s74pp:4b112725-da76-4917-a9b7-8d3bf5fa3423:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 12:12:26.497118 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-48caaa7d\x2d60a5\x2d4a76\x2d82be\x2de5bcf44693bb.mount: Succeeded. Nov 16 12:12:26.505146 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-48caaa7d\x2d60a5\x2d4a76\x2d82be\x2de5bcf44693bb.mount: Succeeded. Nov 16 12:12:26.520850 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-48caaa7d\x2d60a5\x2d4a76\x2d82be\x2de5bcf44693bb.mount: Succeeded. Nov 16 12:12:26.533337 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02-userdata-shm.mount: Succeeded. Nov 16 12:12:26.546376 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-9a8146212d4e991d20b18e632b1b988f1e896014cdfe3231a43c9e48545af19a-merged.mount: Succeeded. Nov 16 12:12:26.757307 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:26.757263736Z" level=info msg="Stopped pod sandbox: 54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02" id=afedc986-419f-4aae-a34d-9d1bf79c61d9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:12:26.764157 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:26.764101 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_qe-app-registry-s74pp_4b112725-da76-4917-a9b7-8d3bf5fa3423/registry-server/0.log" Nov 16 12:12:26.842259 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:26.842231 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzbr5\" (UniqueName: \"kubernetes.io/projected/4b112725-da76-4917-a9b7-8d3bf5fa3423-kube-api-access-rzbr5\") pod \"4b112725-da76-4917-a9b7-8d3bf5fa3423\" (UID: \"4b112725-da76-4917-a9b7-8d3bf5fa3423\") " Nov 16 12:12:26.853777 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:26.853660 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b112725-da76-4917-a9b7-8d3bf5fa3423-kube-api-access-rzbr5" (OuterVolumeSpecName: "kube-api-access-rzbr5") pod "4b112725-da76-4917-a9b7-8d3bf5fa3423" (UID: "4b112725-da76-4917-a9b7-8d3bf5fa3423"). InnerVolumeSpecName "kube-api-access-rzbr5". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 12:12:26.942350 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:26.942302 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/qe-app-registry-s6mgf" Nov 16 12:12:26.942494 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:26.942359 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/qe-app-registry-s6mgf" Nov 16 12:12:26.942577 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:26.942539 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-rzbr5\" (UniqueName: \"kubernetes.io/projected/4b112725-da76-4917-a9b7-8d3bf5fa3423-kube-api-access-rzbr5\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 12:12:26.994297 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:26.994260 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/qe-app-registry-s6mgf" Nov 16 12:12:27.234657 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:27.234454 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_qe-app-registry-s74pp_4b112725-da76-4917-a9b7-8d3bf5fa3423/registry-server/0.log" Nov 16 12:12:27.234657 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:27.234586 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-s74pp" event=&{ID:4b112725-da76-4917-a9b7-8d3bf5fa3423 Type:ContainerDied Data:54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02} Nov 16 12:12:27.234657 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:27.234660 1563 scope.go:110] "RemoveContainer" containerID="de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a" Nov 16 12:12:27.235649 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:27.235608948Z" level=info msg="Removing container: de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a" id=3dd721fe-bf25-449a-9dd6-374ba0424d26 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:12:27.241849 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod4b112725_da76_4917_a9b7_8d3bf5fa3423.slice. Nov 16 12:12:27.241981 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod4b112725_da76_4917_a9b7_8d3bf5fa3423.slice: Consumed 195ms CPU time Nov 16 12:12:27.264813 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:27.264747 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-s74pp] Nov 16 12:12:27.267898 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:27.267852091Z" level=info msg="Removed container de420100431701a0cbf3dfd633469ba79549b5f2296184b83c0e6edd6fc1749a: openshift-marketplace/qe-app-registry-s74pp/registry-server" id=3dd721fe-bf25-449a-9dd6-374ba0424d26 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:12:27.268785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:27.268676 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-s74pp] Nov 16 12:12:27.276072 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:27.275985 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/qe-app-registry-s6mgf" Nov 16 12:12:27.496819 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-4b112725\x2dda76\x2d4917\x2da9b7\x2d8d3bf5fa3423-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2drzbr5.mount: Succeeded. Nov 16 12:12:28.591337 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:28.591258984Z" level=info msg="Stopping pod sandbox: 54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02" id=addc9aeb-47e8-4693-a834-9fe6507191bd name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:12:28.591337 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:28.591337684Z" level=info msg="Stopped pod sandbox (already stopped): 54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02" id=addc9aeb-47e8-4693-a834-9fe6507191bd name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:12:28.592866 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:28.592798 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=4b112725-da76-4917-a9b7-8d3bf5fa3423 path="/var/lib/kubelet/pods/4b112725-da76-4917-a9b7-8d3bf5fa3423/volumes" Nov 16 12:12:32.830566 ropatil1611az-gs6zj-worker-westus-j4ngg root[17592]: machine-config-daemon[1735]: Starting update from rendered-worker-8b35090bacf1ad998c4e01e0fd1046d1 to rendered-worker-f6b2243eda256f645217bce72b92c7db: &{osUpdate:false kargs:false fips:false passwd:false files:true units:false kernelType:false extensions:false} Nov 16 12:12:32.960808 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:12:32.960766 1563 dynamic_cafile_content.go:210] "Failed to remove file watch, it may have been deleted" file="/etc/kubernetes/kubelet-ca.crt" err="can't remove non-existent inotify watch for: /etc/kubernetes/kubelet-ca.crt" Nov 16 12:12:33.155441 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reloading. Nov 16 12:12:33.211042 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-platform-chrony: /run/coreos-platform-chrony.conf already exists; skipping Nov 16 12:12:33.239019 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: /usr/lib/systemd/system/bootupd.service:22: Unknown lvalue 'ProtectHostname' in section 'Service' Nov 16 12:12:33.350145 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.354682 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.368285 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.372298 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.374484 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.377596 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.380209 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.383392 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.395235 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.396567 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.399646 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.405670 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.415644 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.424908 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.428502 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.437500 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.442087 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.447166 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.560476 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reloading. Nov 16 12:12:33.615223 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-platform-chrony: /run/coreos-platform-chrony.conf already exists; skipping Nov 16 12:12:33.640166 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: /usr/lib/systemd/system/bootupd.service:22: Unknown lvalue 'ProtectHostname' in section 'Service' Nov 16 12:12:33.751259 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.754331 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.767309 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.771011 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.772922 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.775989 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.778553 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.781228 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.793577 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.794899 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.798075 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.802589 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.810448 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.818349 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.821865 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.830875 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.835302 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.839921 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:33.891734 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reloading. Nov 16 12:12:33.942099 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-platform-chrony: /run/coreos-platform-chrony.conf already exists; skipping Nov 16 12:12:33.975103 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: /usr/lib/systemd/system/bootupd.service:22: Unknown lvalue 'ProtectHostname' in section 'Service' Nov 16 12:12:34.083972 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.087330 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.099907 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.103690 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.105577 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.108302 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.110632 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.113208 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.125642 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.127164 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.130693 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.135514 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.142257 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.150665 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.154278 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.162583 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.166894 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.171218 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.187804 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reloading. Nov 16 12:12:34.240538 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-platform-chrony: /run/coreos-platform-chrony.conf already exists; skipping Nov 16 12:12:34.270188 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: /usr/lib/systemd/system/bootupd.service:22: Unknown lvalue 'ProtectHostname' in section 'Service' Nov 16 12:12:34.383062 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.386582 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.399069 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.403561 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.405741 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.408528 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.410598 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.413016 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.424564 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.426462 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.429746 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.434696 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.441976 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.450394 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.453857 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.462354 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.467901 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.472520 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:12:34.508037 ropatil1611az-gs6zj-worker-westus-j4ngg logger[17829]: rendered-worker-f6b2243eda256f645217bce72b92c7db Nov 16 12:12:34.515623 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reloading Container Runtime Interface for OCI (CRI-O). Nov 16 12:12:34.595325 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:34.595269016Z" level=info msg="Reloading configuration" Nov 16 12:12:34.595648 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reloaded Container Runtime Interface for OCI (CRI-O). Nov 16 12:12:34.597095 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:34.597062130Z" level=info msg="Updating config from file /etc/crio/crio.conf" Nov 16 12:12:34.597324 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:34.597293931Z" level=info msg="Updating config from path /etc/crio/crio.conf.d" Nov 16 12:12:34.599498 ropatil1611az-gs6zj-worker-westus-j4ngg root[17839]: machine-config-daemon[1735]: crio config reloaded successfully! Desired config rendered-worker-f6b2243eda256f645217bce72b92c7db has been applied, skipping reboot Nov 16 12:12:34.616722 ropatil1611az-gs6zj-worker-westus-j4ngg logger[17840]: rendered-worker-f6b2243eda256f645217bce72b92c7db Nov 16 12:12:34.619572 ropatil1611az-gs6zj-worker-westus-j4ngg root[17841]: machine-config-daemon[1735]: Update completed for config rendered-worker-f6b2243eda256f645217bce72b92c7db and node has been successfully uncordoned Nov 16 12:12:34.638628 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:34.638512942Z" level=info msg="Applied new registry configuration: &{Registries:[{Prefix:registry-proxy.engineering.redhat.com Endpoint:{Location:registry-proxy.engineering.redhat.com Insecure:false} Mirrors:[{Location:brew.registry.redhat.io Insecure:false}] Blocked:false MirrorByDigestOnly:true} {Prefix:registry.redhat.io Endpoint:{Location:registry.redhat.io Insecure:false} Mirrors:[{Location:brew.registry.redhat.io Insecure:false}] Blocked:false MirrorByDigestOnly:true} {Prefix:registry.stage.redhat.io Endpoint:{Location:registry.stage.redhat.io Insecure:false} Mirrors:[{Location:brew.registry.redhat.io Insecure:false}] Blocked:false MirrorByDigestOnly:true}] UnqualifiedSearchRegistries:[registry.access.redhat.com docker.io] CredentialHelpers:[containers-auth.json] ShortNameMode: shortNameAliasConf:{Aliases:map[]}}" Nov 16 12:12:34.638628 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:34.638572543Z" level=info msg="No seccomp profile specified, using the internal default" Nov 16 12:12:34.638628 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:12:34.638584443Z" level=info msg="Set config seccomp_profile to \"\"" Nov 16 12:12:36.976813 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.qIxxMo.mount: Succeeded. Nov 16 12:13:08.309109 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:08.309049 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf] Nov 16 12:13:08.309109 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:08.309114 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 12:13:08.309727 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 12:13:08.309233 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="4b112725-da76-4917-a9b7-8d3bf5fa3423" containerName="registry-server" Nov 16 12:13:08.309727 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:08.309247 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b112725-da76-4917-a9b7-8d3bf5fa3423" containerName="registry-server" Nov 16 12:13:08.309727 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:08.309286 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="4b112725-da76-4917-a9b7-8d3bf5fa3423" containerName="registry-server" Nov 16 12:13:08.319825 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-besteffort-podb6d86f0b_e1d0_4ffe_b9c6_85f4a93f08dd.slice. Nov 16 12:13:08.441969 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:08.441867 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6zxl\" (UniqueName: \"kubernetes.io/projected/b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd-kube-api-access-z6zxl\") pod \"cluster-logging-operator-7f6c6bb67c-hmvsf\" (UID: \"b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd\") " pod="openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf" Nov 16 12:13:08.542642 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:08.542571 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-z6zxl\" (UniqueName: \"kubernetes.io/projected/b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd-kube-api-access-z6zxl\") pod \"cluster-logging-operator-7f6c6bb67c-hmvsf\" (UID: \"b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd\") " pod="openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf" Nov 16 12:13:08.574593 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:08.574436 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6zxl\" (UniqueName: \"kubernetes.io/projected/b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd-kube-api-access-z6zxl\") pod \"cluster-logging-operator-7f6c6bb67c-hmvsf\" (UID: \"b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd\") " pod="openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf" Nov 16 12:13:08.635731 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:08.635623 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf" Nov 16 12:13:08.636538 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:08.636340401Z" level=info msg="Running pod sandbox: openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf/POD" id=bc7e7163-0970-4fed-ac4d-63a87987e7e0 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:13:08.636538 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:08.636490202Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:13:08.682551 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:08.682482537Z" level=info msg="Got pod network &{Name:cluster-logging-operator-7f6c6bb67c-hmvsf Namespace:openshift-logging ID:6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d UID:b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd NetNS:/var/run/netns/98721d7b-dd12-41b3-993b-3bfd49aba10f Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:13:08.682551 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:08.682513637Z" level=info msg="Adding pod openshift-logging_cluster-logging-operator-7f6c6bb67c-hmvsf to CNI network \"multus-cni-network\" (type=multus)" Nov 16 12:13:08.730215 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 12:13:08.746745 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[18460]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 12:13:08.747018 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600788.7469] manager: (veth83bb3ade): new Veth device (/org/freedesktop/NetworkManager/Devices/26) Nov 16 12:13:08.747841 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600788.7478] device (veth83bb3ade): carrier: link connected Nov 16 12:13:08.754025 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth83bb3ade: link is not ready Nov 16 12:13:08.754162 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth83bb3ade: link becomes ready Nov 16 12:13:08.754196 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 12:13:08.759722 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[18460]: Could not generate persistent MAC address for veth83bb3ade: No such file or directory Nov 16 12:13:08.771983 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600788.7719] manager: (veth83bb3ade): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/27) Nov 16 12:13:08.777549 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00087|bridge|INFO|bridge br0: added interface veth83bb3ade on port 11 Nov 16 12:13:08.777891 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth83bb3ade entered promiscuous mode Nov 16 12:13:08.807459 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00088|connmgr|INFO|br0<->unix#127: 5 flow_mods in the last 0 s (5 adds) Nov 16 12:13:08.853706 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00089|connmgr|INFO|br0<->unix#131: 2 flow_mods in the last 0 s (2 deletes) Nov 16 12:13:08.855470 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00090|connmgr|INFO|br0<->unix#133: 6 flow_mods in the last 0 s (2 adds, 4 deletes) Nov 16 12:13:10.565240 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:10.565097 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf] Nov 16 12:13:10.565709 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T12:13:10Z [verbose] Add: openshift-logging:cluster-logging-operator-7f6c6bb67c-hmvsf:b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/98721d7b-dd12-41b3-993b-3bfd49aba10f"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.10/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 12:13:10.565709 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 12:13:10.542746 18443 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-logging", Name:"cluster-logging-operator-7f6c6bb67c-hmvsf", UID:"b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd", APIVersion:"v1", ResourceVersion:"32623", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.10/23] from openshift-sdn Nov 16 12:13:10.565974 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:10.565882071Z" level=info msg="Got pod network &{Name:cluster-logging-operator-7f6c6bb67c-hmvsf Namespace:openshift-logging ID:6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d UID:b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd NetNS:/var/run/netns/98721d7b-dd12-41b3-993b-3bfd49aba10f Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:13:10.566281 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:10.566229873Z" level=info msg="Checking pod openshift-logging_cluster-logging-operator-7f6c6bb67c-hmvsf for CNI network multus-cni-network (type=multus)" Nov 16 12:13:10.568958 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 12:13:10.568906 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6d86f0b_e1d0_4ffe_b9c6_85f4a93f08dd.slice/crio-6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d.scope WatchSource:0}: Error finding container 6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d: Status 404 returned error &{%!s(*http.body=&{0xc00102f350 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 12:13:10.581722 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:10.581646886Z" level=info msg="Ran pod sandbox 6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d with infra container: openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf/POD" id=bc7e7163-0970-4fed-ac4d-63a87987e7e0 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:13:10.583001 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:10.582897895Z" level=info msg="Checking image status: registry.redhat.io/openshift-logging/cluster-logging-rhel8-operator@sha256:1c7d0363374e4d1f285cdec7507825de57480d4c18e797782b3049f6719bb6e1" id=f45272fe-4596-4c88-9614-69d856bb7061 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:10.583356 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:10.583313898Z" level=info msg="Image registry.redhat.io/openshift-logging/cluster-logging-rhel8-operator@sha256:1c7d0363374e4d1f285cdec7507825de57480d4c18e797782b3049f6719bb6e1 not found" id=f45272fe-4596-4c88-9614-69d856bb7061 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:10.584477 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:10.584409606Z" level=info msg="Pulling image: registry.redhat.io/openshift-logging/cluster-logging-rhel8-operator@sha256:1c7d0363374e4d1f285cdec7507825de57480d4c18e797782b3049f6719bb6e1" id=8550c452-7deb-4160-b89e-4d4afcb6e031 name=/runtime.v1.ImageService/PullImage Nov 16 12:13:10.587092 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:10.587026825Z" level=info msg="Trying to access \"brew.registry.redhat.io/openshift-logging/cluster-logging-rhel8-operator@sha256:1c7d0363374e4d1f285cdec7507825de57480d4c18e797782b3049f6719bb6e1\"" Nov 16 12:13:11.351181 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:11.351141 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf" event=&{ID:b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd Type:ContainerStarted Data:6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d} Nov 16 12:13:11.823081 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:11.822868737Z" level=info msg="Trying to access \"brew.registry.redhat.io/openshift-logging/cluster-logging-rhel8-operator@sha256:1c7d0363374e4d1f285cdec7507825de57480d4c18e797782b3049f6719bb6e1\"" Nov 16 12:13:15.526061 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.526019 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv] Nov 16 12:13:15.526061 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.526074 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 12:13:15.536489 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-besteffort-pod9502a499_9f8b_4a15_99d8_714a0cc37bea.slice. Nov 16 12:13:15.594326 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.594290 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf6kf\" (UniqueName: \"kubernetes.io/projected/9502a499-9f8b-4a15-99d8-714a0cc37bea-kube-api-access-cf6kf\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.594438 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.594342 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-webhook-cert\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.594438 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.594399 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-apiservice-cert\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.594638 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.594528 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9502a499-9f8b-4a15-99d8-714a0cc37bea-manager-config\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.594638 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.594605 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"loki-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-loki-operator-metrics-cert\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.695810 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.695775 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"loki-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-loki-operator-metrics-cert\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.695942 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.695831 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-webhook-cert\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.695942 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.695883 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-apiservice-cert\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.695942 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.695914 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-cf6kf\" (UniqueName: \"kubernetes.io/projected/9502a499-9f8b-4a15-99d8-714a0cc37bea-kube-api-access-cf6kf\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.696031 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.695957 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9502a499-9f8b-4a15-99d8-714a0cc37bea-manager-config\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.696760 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.696727 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9502a499-9f8b-4a15-99d8-714a0cc37bea-manager-config\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.699327 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.699287 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-webhook-cert\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.699327 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.699309 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"loki-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-loki-operator-metrics-cert\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.700047 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.699912 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-apiservice-cert\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.726286 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.726241 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf6kf\" (UniqueName: \"kubernetes.io/projected/9502a499-9f8b-4a15-99d8-714a0cc37bea-kube-api-access-cf6kf\") pod \"loki-operator-controller-manager-67cc5866c9-m67tv\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.850833 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:15.850738 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:15.851613 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:15.851484003Z" level=info msg="Running pod sandbox: openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv/POD" id=4590367a-01dd-4ce7-a5a8-d287c55da892 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:13:15.851613 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:15.851606804Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:13:16.688967 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:16.688895 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt] Nov 16 12:13:16.688967 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:16.688951 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 12:13:16.699216 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-besteffort-pod220ce491_f766_41ee_ab02_aa1546eb7ed3.slice. Nov 16 12:13:16.803668 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:16.803560 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"elasticsearch-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/220ce491-f766-41ee-ab02-aa1546eb7ed3-elasticsearch-operator-metrics-cert\") pod \"elasticsearch-operator-66b6655fcb-ksdmt\" (UID: \"220ce491-f766-41ee-ab02-aa1546eb7ed3\") " pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" Nov 16 12:13:16.803668 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:16.803666 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj7t8\" (UniqueName: \"kubernetes.io/projected/220ce491-f766-41ee-ab02-aa1546eb7ed3-kube-api-access-xj7t8\") pod \"elasticsearch-operator-66b6655fcb-ksdmt\" (UID: \"220ce491-f766-41ee-ab02-aa1546eb7ed3\") " pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" Nov 16 12:13:16.904382 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:16.904343 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-xj7t8\" (UniqueName: \"kubernetes.io/projected/220ce491-f766-41ee-ab02-aa1546eb7ed3-kube-api-access-xj7t8\") pod \"elasticsearch-operator-66b6655fcb-ksdmt\" (UID: \"220ce491-f766-41ee-ab02-aa1546eb7ed3\") " pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" Nov 16 12:13:16.904542 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:16.904434 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"elasticsearch-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/220ce491-f766-41ee-ab02-aa1546eb7ed3-elasticsearch-operator-metrics-cert\") pod \"elasticsearch-operator-66b6655fcb-ksdmt\" (UID: \"220ce491-f766-41ee-ab02-aa1546eb7ed3\") " pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" Nov 16 12:13:16.907627 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:16.907565 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"elasticsearch-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/220ce491-f766-41ee-ab02-aa1546eb7ed3-elasticsearch-operator-metrics-cert\") pod \"elasticsearch-operator-66b6655fcb-ksdmt\" (UID: \"220ce491-f766-41ee-ab02-aa1546eb7ed3\") " pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" Nov 16 12:13:16.949029 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:16.948904 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj7t8\" (UniqueName: \"kubernetes.io/projected/220ce491-f766-41ee-ab02-aa1546eb7ed3-kube-api-access-xj7t8\") pod \"elasticsearch-operator-66b6655fcb-ksdmt\" (UID: \"220ce491-f766-41ee-ab02-aa1546eb7ed3\") " pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" Nov 16 12:13:16.977657 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.AGUKwj.mount: Succeeded. Nov 16 12:13:17.017582 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:17.017533 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" Nov 16 12:13:17.018171 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:17.018089174Z" level=info msg="Running pod sandbox: openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt/POD" id=59a40c51-4534-440e-9929-da32edd7faae name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:13:17.018508 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:17.018188374Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:13:18.136878 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:18.136751788Z" level=info msg="Got pod network &{Name:loki-operator-controller-manager-67cc5866c9-m67tv Namespace:openshift-operators-redhat ID:0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7 UID:9502a499-9f8b-4a15-99d8-714a0cc37bea NetNS:/var/run/netns/4d6a1d55-cb65-4ac7-b4f6-0ccbae6022e5 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:13:18.137373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:18.136910789Z" level=info msg="Adding pod openshift-operators-redhat_loki-operator-controller-manager-67cc5866c9-m67tv to CNI network \"multus-cni-network\" (type=multus)" Nov 16 12:13:18.185206 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 12:13:18.195112 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600798.1950] manager: (veth5cab1de5): new Veth device (/org/freedesktop/NetworkManager/Devices/28) Nov 16 12:13:18.196001 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600798.1959] device (veth5cab1de5): carrier: link connected Nov 16 12:13:18.202369 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[18865]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 12:13:18.210366 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth5cab1de5: link is not ready Nov 16 12:13:18.210440 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth5cab1de5: link becomes ready Nov 16 12:13:18.210469 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 12:13:18.216981 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[18865]: Could not generate persistent MAC address for veth5cab1de5: No such file or directory Nov 16 12:13:18.232049 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600798.2320] manager: (veth5cab1de5): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/29) Nov 16 12:13:18.237945 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00091|bridge|INFO|bridge br0: added interface veth5cab1de5 on port 12 Nov 16 12:13:18.238345 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth5cab1de5 entered promiscuous mode Nov 16 12:13:18.270636 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00092|connmgr|INFO|br0<->unix#136: 5 flow_mods in the last 0 s (5 adds) Nov 16 12:13:18.323228 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00093|connmgr|INFO|br0<->unix#140: 6 flow_mods in the last 0 s (2 adds, 4 deletes) Nov 16 12:13:18.324408 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00094|connmgr|INFO|br0<->unix#142: 2 flow_mods in the last 0 s (2 deletes) Nov 16 12:13:19.429539 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.429488896Z" level=info msg="Got pod network &{Name:elasticsearch-operator-66b6655fcb-ksdmt Namespace:openshift-operators-redhat ID:59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f UID:220ce491-f766-41ee-ab02-aa1546eb7ed3 NetNS:/var/run/netns/19b123c6-9795-499e-8d37-fae8e0b17a13 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:13:19.429539 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.429534797Z" level=info msg="Adding pod openshift-operators-redhat_elasticsearch-operator-66b6655fcb-ksdmt to CNI network \"multus-cni-network\" (type=multus)" Nov 16 12:13:19.482182 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 12:13:19.492249 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600799.4922] manager: (vetha69937be): new Veth device (/org/freedesktop/NetworkManager/Devices/30) Nov 16 12:13:19.493238 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600799.4932] device (vetha69937be): carrier: link connected Nov 16 12:13:19.506247 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): vetha69937be: link is not ready Nov 16 12:13:19.506315 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vetha69937be: link becomes ready Nov 16 12:13:19.506344 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 12:13:19.512562 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[18953]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 12:13:19.512715 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[18953]: Could not generate persistent MAC address for vetha69937be: No such file or directory Nov 16 12:13:19.533432 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668600799.5334] manager: (vetha69937be): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/31) Nov 16 12:13:19.538896 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00095|bridge|INFO|bridge br0: added interface vetha69937be on port 13 Nov 16 12:13:19.539325 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vetha69937be entered promiscuous mode Nov 16 12:13:19.570396 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00096|connmgr|INFO|br0<->unix#145: 5 flow_mods in the last 0 s (5 adds) Nov 16 12:13:19.584987 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.584944252Z" level=info msg="Pulled image: registry.redhat.io/openshift-logging/cluster-logging-rhel8-operator@sha256:1c7d0363374e4d1f285cdec7507825de57480d4c18e797782b3049f6719bb6e1" id=8550c452-7deb-4160-b89e-4d4afcb6e031 name=/runtime.v1.ImageService/PullImage Nov 16 12:13:19.585905 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.585867659Z" level=info msg="Checking image status: registry.redhat.io/openshift-logging/cluster-logging-rhel8-operator@sha256:1c7d0363374e4d1f285cdec7507825de57480d4c18e797782b3049f6719bb6e1" id=9cdd2b8b-fe20-4121-bf6d-7285060e8ffd name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:19.587384 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.587341570Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:15391db13fa04e5ab3461609b63f7725de48133bc3467a7a5ffbee5bb8d17a5e,RepoTags:[],RepoDigests:[registry.redhat.io/openshift-logging/cluster-logging-rhel8-operator@sha256:1c7d0363374e4d1f285cdec7507825de57480d4c18e797782b3049f6719bb6e1 registry.redhat.io/openshift-logging/cluster-logging-rhel8-operator@sha256:ef9a3b9cf96f17b4b51d07cab4f993a1bb1ccbe0d999f1b78dd10667479dd101],Size_:411291779,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=9cdd2b8b-fe20-4121-bf6d-7285060e8ffd name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:19.588403 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.588365777Z" level=info msg="Creating container: openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf/cluster-logging-operator" id=9984873f-0bed-4a81-978a-59874741824c name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:13:19.588489 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.588470578Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:13:19.612226 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00097|connmgr|INFO|br0<->unix#148: 2 flow_mods in the last 0 s (2 deletes) Nov 16 12:13:19.636873 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c.scope. Nov 16 12:13:19.661339 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c. Nov 16 12:13:19.693525 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:13:19.752444 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.752376896Z" level=info msg="Created container d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c: openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf/cluster-logging-operator" id=9984873f-0bed-4a81-978a-59874741824c name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:13:19.753148 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.753104502Z" level=info msg="Starting container: d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c" id=b991b42e-ce62-4f74-9f85-d66c4d3715f0 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:13:19.762633 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.762592472Z" level=info msg="Started container" PID=18986 containerID=d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c description=openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf/cluster-logging-operator id=b991b42e-ce62-4f74-9f85-d66c4d3715f0 name=/runtime.v1.RuntimeService/StartContainer sandboxID=6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d Nov 16 12:13:19.793030 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T12:13:19Z [verbose] Add: openshift-operators-redhat:loki-operator-controller-manager-67cc5866c9-m67tv:9502a499-9f8b-4a15-99d8-714a0cc37bea:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/4d6a1d55-cb65-4ac7-b4f6-0ccbae6022e5"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.11/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 12:13:19.793030 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 12:13:19.768287 18838 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-operators-redhat", Name:"loki-operator-controller-manager-67cc5866c9-m67tv", UID:"9502a499-9f8b-4a15-99d8-714a0cc37bea", APIVersion:"v1", ResourceVersion:"32958", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.11/23] from openshift-sdn Nov 16 12:13:19.793738 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.793674703Z" level=info msg="Got pod network &{Name:loki-operator-controller-manager-67cc5866c9-m67tv Namespace:openshift-operators-redhat ID:0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7 UID:9502a499-9f8b-4a15-99d8-714a0cc37bea NetNS:/var/run/netns/4d6a1d55-cb65-4ac7-b4f6-0ccbae6022e5 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:13:19.793874 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.793840604Z" level=info msg="Checking pod openshift-operators-redhat_loki-operator-controller-manager-67cc5866c9-m67tv for CNI network multus-cni-network (type=multus)" Nov 16 12:13:19.796434 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:19.796399 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv] Nov 16 12:13:19.802842 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 12:13:19.802809 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9502a499_9f8b_4a15_99d8_714a0cc37bea.slice/crio-0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7.scope WatchSource:0}: Error finding container 0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7: Status 404 returned error &{%!s(*http.body=&{0xc0023fc7e0 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 12:13:19.805698 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.805653092Z" level=info msg="Ran pod sandbox 0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7 with infra container: openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv/POD" id=4590367a-01dd-4ce7-a5a8-d287c55da892 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:13:19.806714 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.806659300Z" level=info msg="Checking image status: registry.redhat.io/openshift-logging/loki-rhel8-operator@sha256:790e863255d4301d4e8eebda467593a4efdf359c17dbe949ea470e2602412043" id=bbe08505-a0da-464f-b772-82a6e1f5e6e8 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:19.806987 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.806957202Z" level=info msg="Image registry.redhat.io/openshift-logging/loki-rhel8-operator@sha256:790e863255d4301d4e8eebda467593a4efdf359c17dbe949ea470e2602412043 not found" id=bbe08505-a0da-464f-b772-82a6e1f5e6e8 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:19.807876 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.807788208Z" level=info msg="Pulling image: registry.redhat.io/openshift-logging/loki-rhel8-operator@sha256:790e863255d4301d4e8eebda467593a4efdf359c17dbe949ea470e2602412043" id=d1816090-7556-4bbf-b5a1-47be2163e0c0 name=/runtime.v1.ImageService/PullImage Nov 16 12:13:19.812178 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:19.812100240Z" level=info msg="Trying to access \"brew.registry.redhat.io/openshift-logging/loki-rhel8-operator@sha256:790e863255d4301d4e8eebda467593a4efdf359c17dbe949ea470e2602412043\"" Nov 16 12:13:20.333401 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:20.333173113Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=a7eb4343-8b23-49c1-82b8-2a8160349e00 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:20.888550 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:20.333460215Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=a7eb4343-8b23-49c1-82b8-2a8160349e00 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:20.888550 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:20.393964065Z" level=info msg="Stopping pod sandbox: 54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02" id=64ec0ba1-320c-4eac-b85c-d3c8a45afcd5 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:13:20.888550 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:20.394020965Z" level=info msg="Stopped pod sandbox (already stopped): 54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02" id=64ec0ba1-320c-4eac-b85c-d3c8a45afcd5 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:13:20.888550 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:20.394269667Z" level=info msg="Removing pod sandbox: 54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02" id=35025f7d-d6d1-4b9d-ba2c-d9ba8aa4c4fb name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 12:13:20.889034 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:20.378701 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf" event=&{ID:b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd Type:ContainerStarted Data:d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c} Nov 16 12:13:20.889034 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:20.383545 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" event=&{ID:9502a499-9f8b-4a15-99d8-714a0cc37bea Type:ContainerStarted Data:0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7} Nov 16 12:13:20.889372 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:20.888840343Z" level=info msg="Removed pod sandbox: 54c6c140c944c9fd1d56021828655e96d8a9476a38d8919786d0d2e2301f4a02" id=35025f7d-d6d1-4b9d-ba2c-d9ba8aa4c4fb name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 12:13:21.189178 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:21.188968076Z" level=info msg="Trying to access \"brew.registry.redhat.io/openshift-logging/loki-rhel8-operator@sha256:790e863255d4301d4e8eebda467593a4efdf359c17dbe949ea470e2602412043\"" Nov 16 12:13:21.438671 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:21.438452 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt] Nov 16 12:13:21.439887 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T12:13:21Z [verbose] Add: openshift-operators-redhat:elasticsearch-operator-66b6655fcb-ksdmt:220ce491-f766-41ee-ab02-aa1546eb7ed3:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/19b123c6-9795-499e-8d37-fae8e0b17a13"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.12/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 12:13:21.439887 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 12:13:21.406858 18937 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-operators-redhat", Name:"elasticsearch-operator-66b6655fcb-ksdmt", UID:"220ce491-f766-41ee-ab02-aa1546eb7ed3", APIVersion:"v1", ResourceVersion:"33037", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.12/23] from openshift-sdn Nov 16 12:13:21.440526 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:21.440480853Z" level=info msg="Got pod network &{Name:elasticsearch-operator-66b6655fcb-ksdmt Namespace:openshift-operators-redhat ID:59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f UID:220ce491-f766-41ee-ab02-aa1546eb7ed3 NetNS:/var/run/netns/19b123c6-9795-499e-8d37-fae8e0b17a13 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:13:21.440711 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:21.440680954Z" level=info msg="Checking pod openshift-operators-redhat_elasticsearch-operator-66b6655fcb-ksdmt for CNI network multus-cni-network (type=multus)" Nov 16 12:13:21.443948 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 12:13:21.443883 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod220ce491_f766_41ee_ab02_aa1546eb7ed3.slice/crio-59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f.scope WatchSource:0}: Error finding container 59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f: Status 404 returned error &{%!s(*http.body=&{0xc00135a000 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 12:13:21.459586 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:21.459505094Z" level=info msg="Ran pod sandbox 59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f with infra container: openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt/POD" id=59a40c51-4534-440e-9929-da32edd7faae name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:13:21.460721 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:21.460660503Z" level=info msg="Checking image status: registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d" id=c1038b97-7034-4281-a815-39a4cacdd6bb name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:21.460985 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:21.460956605Z" level=info msg="Image registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d not found" id=c1038b97-7034-4281-a815-39a4cacdd6bb name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:21.465801 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:21.462450916Z" level=info msg="Pulling image: registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d" id=defc72f1-74ad-4360-acfd-b7365ed8fb10 name=/runtime.v1.ImageService/PullImage Nov 16 12:13:21.465801 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:21.464974935Z" level=info msg="Trying to access \"brew.registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d\"" Nov 16 12:13:22.389423 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:22.389384 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" event=&{ID:220ce491-f766-41ee-ab02-aa1546eb7ed3 Type:ContainerStarted Data:59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f} Nov 16 12:13:22.599755 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:22.599704600Z" level=info msg="Trying to access \"brew.registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d\"" Nov 16 12:13:24.362807 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:24.362697751Z" level=info msg="Pulled image: registry.redhat.io/openshift-logging/loki-rhel8-operator@sha256:790e863255d4301d4e8eebda467593a4efdf359c17dbe949ea470e2602412043" id=d1816090-7556-4bbf-b5a1-47be2163e0c0 name=/runtime.v1.ImageService/PullImage Nov 16 12:13:24.369380 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:24.369310800Z" level=info msg="Checking image status: registry.redhat.io/openshift-logging/loki-rhel8-operator@sha256:790e863255d4301d4e8eebda467593a4efdf359c17dbe949ea470e2602412043" id=6e7129a2-b950-4360-bded-da12d67b3f84 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:24.370867 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:24.370718711Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:6ecaccf6082ac76661e64ebd2d977518d9248e1923789c9e7e44939dec767fc5,RepoTags:[],RepoDigests:[registry.redhat.io/openshift-logging/loki-rhel8-operator@sha256:790e863255d4301d4e8eebda467593a4efdf359c17dbe949ea470e2602412043 registry.redhat.io/openshift-logging/loki-rhel8-operator@sha256:f8a35557c9eb27f703ffa9320a9d39a2361d93fdb0b8c874d97da458048e7eec],Size_:279652430,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=6e7129a2-b950-4360-bded-da12d67b3f84 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:24.372218 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:24.372183422Z" level=info msg="Creating container: openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv/manager" id=ba3bb1a6-215e-4677-bf5b-38a906d29ead name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:13:24.372328 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:24.372276822Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:13:24.424401 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4.scope. Nov 16 12:13:24.449459 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4. Nov 16 12:13:24.486727 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:13:24.550525 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:24.550444451Z" level=info msg="Created container 8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4: openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv/manager" id=ba3bb1a6-215e-4677-bf5b-38a906d29ead name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:13:24.551179 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:24.551059056Z" level=info msg="Starting container: 8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4" id=6d8e91fd-3a46-496d-abab-b7fa5820a270 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:13:24.563923 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:24.563850251Z" level=info msg="Started container" PID=19144 containerID=8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4 description=openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv/manager id=6d8e91fd-3a46-496d-abab-b7fa5820a270 name=/runtime.v1.RuntimeService/StartContainer sandboxID=0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7 Nov 16 12:13:24.584852 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:24.584755107Z" level=info msg="Checking image status: registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d" id=23ba7579-6ba2-414e-8bf0-9a062f9d356f name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:24.585143 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:24.585104510Z" level=info msg="Image registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d not found" id=23ba7579-6ba2-414e-8bf0-9a062f9d356f name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:24.586259 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:24.586186318Z" level=info msg="Pulling image: registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d" id=a556fd15-1d9f-4c8f-8507-2248415094bf name=/runtime.v1.ImageService/PullImage Nov 16 12:13:24.748815 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:24.748573529Z" level=info msg="Trying to access \"brew.registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d\"" Nov 16 12:13:25.398225 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:25.398179 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" event=&{ID:9502a499-9f8b-4a15-99d8-714a0cc37bea Type:ContainerStarted Data:8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4} Nov 16 12:13:25.905949 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:25.905887862Z" level=info msg="Trying to access \"brew.registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d\"" Nov 16 12:13:32.637656 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.637527998Z" level=info msg="Pulled image: registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:16d63be14c9478a29c9d6631d11ff5f0fba4bb052bce81bf186c6e1c0578442c" id=defc72f1-74ad-4360-acfd-b7365ed8fb10 name=/runtime.v1.ImageService/PullImage Nov 16 12:13:32.638620 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.638572306Z" level=info msg="Checking image status: registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d" id=33053a3e-0920-4f42-b784-90ea0f2494a5 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:32.640177 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.640048117Z" level=info msg="Pulled image: registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:16d63be14c9478a29c9d6631d11ff5f0fba4bb052bce81bf186c6e1c0578442c" id=a556fd15-1d9f-4c8f-8507-2248415094bf name=/runtime.v1.ImageService/PullImage Nov 16 12:13:32.640759 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.640717322Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:90334e8650b6c6c2f0496befbdb5c0f173a5be4e15ab509166ca486423fa2f83,RepoTags:[],RepoDigests:[registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:16d63be14c9478a29c9d6631d11ff5f0fba4bb052bce81bf186c6e1c0578442c registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d],Size_:418255233,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=33053a3e-0920-4f42-b784-90ea0f2494a5 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:32.641809 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.641775430Z" level=info msg="Creating container: openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt/kube-rbac-proxy" id=b8233e53-d6a8-4316-b02e-5d51e0d37a66 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:13:32.641949 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.641916231Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:13:32.645224 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.645175255Z" level=info msg="Checking image status: registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d" id=d57cc541-c6a5-41f2-a6be-dda9b86a716d name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:32.679276 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.679174809Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:90334e8650b6c6c2f0496befbdb5c0f173a5be4e15ab509166ca486423fa2f83,RepoTags:[],RepoDigests:[registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:16d63be14c9478a29c9d6631d11ff5f0fba4bb052bce81bf186c6e1c0578442c registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:cd779f7b8b12a4ef79f8473d44c01c8abae76e4b9017b2f795c33fd9e6f7a03d],Size_:418255233,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=d57cc541-c6a5-41f2-a6be-dda9b86a716d name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:32.680291 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.680248717Z" level=info msg="Creating container: openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv/kube-rbac-proxy" id=4f8c0120-77a3-4af8-a67e-5864ef72e910 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:13:32.680397 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.680351818Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:13:32.693529 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667.scope. Nov 16 12:13:32.720528 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667. Nov 16 12:13:32.733669 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb.scope. Nov 16 12:13:32.760812 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb. Nov 16 12:13:32.765868 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:13:32.789900 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:13:32.839500 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.839414305Z" level=info msg="Created container 881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667: openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt/kube-rbac-proxy" id=b8233e53-d6a8-4316-b02e-5d51e0d37a66 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:13:32.840104 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.840054610Z" level=info msg="Starting container: 881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667" id=6d85374d-e4c2-4016-8450-9432a55146d6 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:13:32.850861 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.850820591Z" level=info msg="Started container" PID=19420 containerID=881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667 description=openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt/kube-rbac-proxy id=6d85374d-e4c2-4016-8450-9432a55146d6 name=/runtime.v1.RuntimeService/StartContainer sandboxID=59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f Nov 16 12:13:32.854822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.854769420Z" level=info msg="Created container 2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb: openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv/kube-rbac-proxy" id=4f8c0120-77a3-4af8-a67e-5864ef72e910 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:13:32.857396 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.857301039Z" level=info msg="Starting container: 2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb" id=a3a284ff-5b54-4388-84a3-96e665c1e71d name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:13:32.868746 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.868633324Z" level=info msg="Started container" PID=19438 containerID=2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb description=openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv/kube-rbac-proxy id=a3a284ff-5b54-4388-84a3-96e665c1e71d name=/runtime.v1.RuntimeService/StartContainer sandboxID=0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7 Nov 16 12:13:32.871086 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.871037941Z" level=info msg="Checking image status: registry.redhat.io/openshift-logging/elasticsearch-rhel8-operator@sha256:9f01ed33db47ab5eefcf5fbe0418bd124ec8f244ad054cbf1bc1f5ecb55c42fa" id=857feab4-78c2-4c8a-86e5-2ebce8e8a7eb name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:32.871377 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.871343844Z" level=info msg="Image registry.redhat.io/openshift-logging/elasticsearch-rhel8-operator@sha256:9f01ed33db47ab5eefcf5fbe0418bd124ec8f244ad054cbf1bc1f5ecb55c42fa not found" id=857feab4-78c2-4c8a-86e5-2ebce8e8a7eb name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:32.872354 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.872316251Z" level=info msg="Pulling image: registry.redhat.io/openshift-logging/elasticsearch-rhel8-operator@sha256:9f01ed33db47ab5eefcf5fbe0418bd124ec8f244ad054cbf1bc1f5ecb55c42fa" id=9bb117f1-1019-4368-b460-636debeccf94 name=/runtime.v1.ImageService/PullImage Nov 16 12:13:32.880401 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:32.880350111Z" level=info msg="Trying to access \"brew.registry.redhat.io/openshift-logging/elasticsearch-rhel8-operator@sha256:9f01ed33db47ab5eefcf5fbe0418bd124ec8f244ad054cbf1bc1f5ecb55c42fa\"" Nov 16 12:13:33.419344 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:33.419286 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" event=&{ID:220ce491-f766-41ee-ab02-aa1546eb7ed3 Type:ContainerStarted Data:881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667} Nov 16 12:13:33.421013 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:33.420987 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" event=&{ID:9502a499-9f8b-4a15-99d8-714a0cc37bea Type:ContainerStarted Data:2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb} Nov 16 12:13:33.421774 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:33.421537 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:33.422579 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:33.422545 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" Nov 16 12:13:34.190783 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:34.190695093Z" level=info msg="Trying to access \"brew.registry.redhat.io/openshift-logging/elasticsearch-rhel8-operator@sha256:9f01ed33db47ab5eefcf5fbe0418bd124ec8f244ad054cbf1bc1f5ecb55c42fa\"" Nov 16 12:13:37.083281 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:37.083216887Z" level=info msg="Pulled image: registry.redhat.io/openshift-logging/elasticsearch-rhel8-operator@sha256:9f01ed33db47ab5eefcf5fbe0418bd124ec8f244ad054cbf1bc1f5ecb55c42fa" id=9bb117f1-1019-4368-b460-636debeccf94 name=/runtime.v1.ImageService/PullImage Nov 16 12:13:37.092004 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:37.091921652Z" level=info msg="Checking image status: registry.redhat.io/openshift-logging/elasticsearch-rhel8-operator@sha256:9f01ed33db47ab5eefcf5fbe0418bd124ec8f244ad054cbf1bc1f5ecb55c42fa" id=a713a58c-837d-4150-8270-51864a13a6f2 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:37.093568 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:37.093476664Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:6322f9b9f73758ddf090c5355d17d1017d70846845fceff9746877f2281884d2,RepoTags:[],RepoDigests:[registry.redhat.io/openshift-logging/elasticsearch-rhel8-operator@sha256:9f01ed33db47ab5eefcf5fbe0418bd124ec8f244ad054cbf1bc1f5ecb55c42fa registry.redhat.io/openshift-logging/elasticsearch-rhel8-operator@sha256:9fcbf76dcbc1cb8b2fa771748a3a8336d8f8b18ae77ba5e44c13e0bdce706223],Size_:266272596,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=a713a58c-837d-4150-8270-51864a13a6f2 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:13:37.094948 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:37.094845574Z" level=info msg="Creating container: openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt/elasticsearch-operator" id=b9873b77-878d-4dee-a018-179c2d92b2b1 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:13:37.095038 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:37.094995975Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:13:37.155312 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf.scope. Nov 16 12:13:37.190255 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf. Nov 16 12:13:37.225119 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:13:37.291395 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:37.291339729Z" level=info msg="Created container dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf: openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt/elasticsearch-operator" id=b9873b77-878d-4dee-a018-179c2d92b2b1 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:13:37.291972 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:37.291931433Z" level=info msg="Starting container: dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf" id=136363af-f465-47f2-b0fa-bb59adcf4bc2 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:13:37.302201 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:13:37.302044708Z" level=info msg="Started container" PID=19682 containerID=dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf description=openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt/elasticsearch-operator id=136363af-f465-47f2-b0fa-bb59adcf4bc2 name=/runtime.v1.RuntimeService/StartContainer sandboxID=59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f Nov 16 12:13:37.434426 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:37.434391 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" event=&{ID:220ce491-f766-41ee-ab02-aa1546eb7ed3 Type:ContainerStarted Data:dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf} Nov 16 12:13:37.434880 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:37.434529 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" Nov 16 12:13:47.019814 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:13:47.019722 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" Nov 16 12:13:50.398635 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.gsjFeZ.mount: Succeeded. Nov 16 12:14:00.450043 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.MHcS0X.mount: Succeeded. Nov 16 12:17:16.979212 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.fRqtuA.mount: Succeeded. Nov 16 12:17:56.979286 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.7PkMrK.mount: Succeeded. Nov 16 12:18:20.337853 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:18:20.337732303Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=829d1b0a-bafa-4ed1-b8fd-5fa139283b3f name=/runtime.v1.ImageService/ImageStatus Nov 16 12:18:20.402794 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:18:20.338010405Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=829d1b0a-bafa-4ed1-b8fd-5fa139283b3f name=/runtime.v1.ImageService/ImageStatus Nov 16 12:20:16.979307 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.9kBhVq.mount: Succeeded. Nov 16 12:21:35.308799 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.Q6V01o.mount: Succeeded. Nov 16 12:22:06.981606 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.14fm3j.mount: Succeeded. Nov 16 12:23:20.342301 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:23:20.342090316Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=5bead4b7-72a4-4276-8a23-a97754237373 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:23:20.361507 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:23:20.342394518Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=5bead4b7-72a4-4276-8a23-a97754237373 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:24:26.982353 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.KETf9z.mount: Succeeded. Nov 16 12:27:56.228104 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:27:56.227924 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-g5brb] Nov 16 12:27:56.228104 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:27:56.228044 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 12:27:56.238715 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod71cd415c_6580_4fd9_ab24_b9c1eaa87298.slice. Nov 16 12:27:56.380650 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:27:56.380546 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njmqb\" (UniqueName: \"kubernetes.io/projected/71cd415c-6580-4fd9-ab24-b9c1eaa87298-kube-api-access-njmqb\") pod \"qe-app-registry-g5brb\" (UID: \"71cd415c-6580-4fd9-ab24-b9c1eaa87298\") " pod="openshift-marketplace/qe-app-registry-g5brb" Nov 16 12:27:56.481842 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:27:56.481591 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-njmqb\" (UniqueName: \"kubernetes.io/projected/71cd415c-6580-4fd9-ab24-b9c1eaa87298-kube-api-access-njmqb\") pod \"qe-app-registry-g5brb\" (UID: \"71cd415c-6580-4fd9-ab24-b9c1eaa87298\") " pod="openshift-marketplace/qe-app-registry-g5brb" Nov 16 12:27:56.509089 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:27:56.509017 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-njmqb\" (UniqueName: \"kubernetes.io/projected/71cd415c-6580-4fd9-ab24-b9c1eaa87298-kube-api-access-njmqb\") pod \"qe-app-registry-g5brb\" (UID: \"71cd415c-6580-4fd9-ab24-b9c1eaa87298\") " pod="openshift-marketplace/qe-app-registry-g5brb" Nov 16 12:27:56.554291 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:27:56.554249 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-g5brb" Nov 16 12:27:56.555032 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:27:56.554920642Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-g5brb/POD" id=eacc3c23-d279-4f4c-b66d-5ffd57c37553 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:27:56.555521 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:27:56.555051243Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:27:56.614067 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:27:56.614019087Z" level=info msg="Got pod network &{Name:qe-app-registry-g5brb Namespace:openshift-marketplace ID:627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9 UID:71cd415c-6580-4fd9-ab24-b9c1eaa87298 NetNS:/var/run/netns/af8bffe8-14d9-444c-8d89-e1550e26f8bd Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:27:56.614067 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:27:56.614064887Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-g5brb to CNI network \"multus-cni-network\" (type=multus)" Nov 16 12:27:56.670167 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 12:27:56.682075 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668601676.6820] manager: (vethd7fdafb7): new Veth device (/org/freedesktop/NetworkManager/Devices/32) Nov 16 12:27:56.682879 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668601676.6828] device (vethd7fdafb7): carrier: link connected Nov 16 12:27:56.694725 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): vethd7fdafb7: link is not ready Nov 16 12:27:56.694816 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethd7fdafb7: link becomes ready Nov 16 12:27:56.694847 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 12:27:56.700103 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[33457]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 12:27:56.700263 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[33457]: Could not generate persistent MAC address for vethd7fdafb7: No such file or directory Nov 16 12:27:56.712356 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668601676.7123] manager: (vethd7fdafb7): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/33) Nov 16 12:27:56.716767 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00098|bridge|INFO|bridge br0: added interface vethd7fdafb7 on port 14 Nov 16 12:27:56.717199 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethd7fdafb7 entered promiscuous mode Nov 16 12:27:56.748087 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00099|connmgr|INFO|br0<->unix#159: 5 flow_mods in the last 0 s (5 adds) Nov 16 12:27:56.794913 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00100|connmgr|INFO|br0<->unix#162: 2 flow_mods in the last 0 s (2 deletes) Nov 16 12:27:58.710122 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T12:27:58Z [verbose] Add: openshift-marketplace:qe-app-registry-g5brb:71cd415c-6580-4fd9-ab24-b9c1eaa87298:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/af8bffe8-14d9-444c-8d89-e1550e26f8bd"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.13/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 12:27:58.710122 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 12:27:58.689180 33441 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-g5brb", UID:"71cd415c-6580-4fd9-ab24-b9c1eaa87298", APIVersion:"v1", ResourceVersion:"38944", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.13/23] from openshift-sdn Nov 16 12:27:58.710657 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:27:58.710621 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-marketplace/qe-app-registry-g5brb] Nov 16 12:27:58.710968 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:27:58.710655660Z" level=info msg="Got pod network &{Name:qe-app-registry-g5brb Namespace:openshift-marketplace ID:627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9 UID:71cd415c-6580-4fd9-ab24-b9c1eaa87298 NetNS:/var/run/netns/af8bffe8-14d9-444c-8d89-e1550e26f8bd Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:27:58.710968 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:27:58.710819162Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-g5brb for CNI network multus-cni-network (type=multus)" Nov 16 12:27:58.713630 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 12:27:58.713598 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71cd415c_6580_4fd9_ab24_b9c1eaa87298.slice/crio-627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9.scope WatchSource:0}: Error finding container 627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9: Status 404 returned error &{%!s(*http.body=&{0xc00305e900 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 12:27:58.727642 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:27:58.727603488Z" level=info msg="Ran pod sandbox 627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9 with infra container: openshift-marketplace/qe-app-registry-g5brb/POD" id=eacc3c23-d279-4f4c-b66d-5ffd57c37553 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:27:58.728754 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:27:58.728684196Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=fda548ac-05d8-4a1c-8e59-568a1e498bee name=/runtime.v1.ImageService/ImageStatus Nov 16 12:27:58.728932 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:27:58.728894698Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=fda548ac-05d8-4a1c-8e59-568a1e498bee name=/runtime.v1.ImageService/ImageStatus Nov 16 12:27:58.729427 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:27:58.729398 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 12:27:58.730042 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:27:58.730002606Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=8467ebc7-0204-423a-a676-cc3a909e172c name=/runtime.v1.ImageService/PullImage Nov 16 12:27:58.733670 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:27:58.733603333Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 12:27:58.815668 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:27:58.815627 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-g5brb" event=&{ID:71cd415c-6580-4fd9-ab24-b9c1eaa87298 Type:ContainerStarted Data:627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9} Nov 16 12:28:00.324020 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:00.323896197Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=8467ebc7-0204-423a-a676-cc3a909e172c name=/runtime.v1.ImageService/PullImage Nov 16 12:28:00.324943 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:00.324902705Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=63e8fe74-8251-4ee6-b7b3-7901cd9c5c72 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:28:00.325182 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:00.325144607Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=63e8fe74-8251-4ee6-b7b3-7901cd9c5c72 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:28:00.326268 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:00.326230615Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-g5brb/registry-server" id=6e1d132c-5f43-4e75-a6b4-f27f3d8b00c7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:28:00.326369 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:00.326342916Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:28:00.385792 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8.scope. Nov 16 12:28:00.412326 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8. Nov 16 12:28:00.444322 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:28:00.504215 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:00.504090453Z" level=info msg="Created container 7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8: openshift-marketplace/qe-app-registry-g5brb/registry-server" id=6e1d132c-5f43-4e75-a6b4-f27f3d8b00c7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:28:00.504806 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:00.504769558Z" level=info msg="Starting container: 7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8" id=35db1a8c-c35b-45dd-9cbc-1ed04555b9a0 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:28:00.514733 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:00.514690833Z" level=info msg="Started container" PID=33598 containerID=7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8 description=openshift-marketplace/qe-app-registry-g5brb/registry-server id=35db1a8c-c35b-45dd-9cbc-1ed04555b9a0 name=/runtime.v1.RuntimeService/StartContainer sandboxID=627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9 Nov 16 12:28:00.822511 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:00.822471 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-g5brb" event=&{ID:71cd415c-6580-4fd9-ab24-b9c1eaa87298 Type:ContainerStarted Data:7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8} Nov 16 12:28:06.554557 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:06.554463 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/qe-app-registry-g5brb" Nov 16 12:28:06.554557 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:06.554527 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/qe-app-registry-g5brb" Nov 16 12:28:06.615521 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:06.615450 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/qe-app-registry-g5brb" Nov 16 12:28:06.893333 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:06.893289 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/qe-app-registry-g5brb" Nov 16 12:28:06.940459 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:06.940372 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-g5brb] Nov 16 12:28:08.844413 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:08.844305 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-g5brb" podUID=71cd415c-6580-4fd9-ab24-b9c1eaa87298 containerName="registry-server" containerID="cri-o://7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8" gracePeriod=1 Nov 16 12:28:08.844940 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:08.844616891Z" level=info msg="Stopping container: 7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8 (timeout: 1s)" id=145eb357-aa7b-42dc-bc3f-82bedca2ec2d name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:28:08.870656 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8.scope: Succeeded. Nov 16 12:28:08.871444 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8.scope: Consumed 682ms CPU time Nov 16 12:28:08.879188 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8.scope: Succeeded. Nov 16 12:28:08.879786 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8.scope: Consumed 38ms CPU time Nov 16 12:28:09.005073 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-1c9affa0a249e36421f178793744db5b2569b37f61fdf592021d6b55e5df1777-merged.mount: Succeeded. Nov 16 12:28:09.362803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:09.362708301Z" level=info msg="Stopped container 7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8: openshift-marketplace/qe-app-registry-g5brb/registry-server" id=145eb357-aa7b-42dc-bc3f-82bedca2ec2d name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:28:09.363522 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:09.363472106Z" level=info msg="Stopping pod sandbox: 627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9" id=13b2f7d5-c4d4-437b-b375-78e58b511c49 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:28:09.363823 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:09.363774308Z" level=info msg="Got pod network &{Name:qe-app-registry-g5brb Namespace:openshift-marketplace ID:627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9 UID:71cd415c-6580-4fd9-ab24-b9c1eaa87298 NetNS:/var/run/netns/af8bffe8-14d9-444c-8d89-e1550e26f8bd Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:28:09.363981 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:09.363957810Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-g5brb from CNI network \"multus-cni-network\" (type=multus)" Nov 16 12:28:09.432417 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00101|connmgr|INFO|br0<->unix#165: 2 flow_mods in the last 0 s (2 deletes) Nov 16 12:28:09.465858 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00102|connmgr|INFO|br0<->unix#168: 4 flow_mods in the last 0 s (4 deletes) Nov 16 12:28:09.496461 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00103|bridge|INFO|bridge br0: deleted interface vethd7fdafb7 on port 14 Nov 16 12:28:09.506206 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethd7fdafb7 left promiscuous mode Nov 16 12:28:09.515912 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T12:28:09Z [verbose] Del: openshift-marketplace:qe-app-registry-g5brb:71cd415c-6580-4fd9-ab24-b9c1eaa87298:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 12:28:09.523169 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-af8bffe8\x2d14d9\x2d444c\x2d8d89\x2de1550e26f8bd.mount: Succeeded. Nov 16 12:28:09.530212 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-af8bffe8\x2d14d9\x2d444c\x2d8d89\x2de1550e26f8bd.mount: Succeeded. Nov 16 12:28:09.550456 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-af8bffe8\x2d14d9\x2d444c\x2d8d89\x2de1550e26f8bd.mount: Succeeded. Nov 16 12:28:09.558140 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9-userdata-shm.mount: Succeeded. Nov 16 12:28:09.709190 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:09.708899046Z" level=info msg="Stopped pod sandbox: 627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9" id=13b2f7d5-c4d4-437b-b375-78e58b511c49 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:28:09.848309 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:09.848264 1563 generic.go:296] "Generic (PLEG): container finished" podID=71cd415c-6580-4fd9-ab24-b9c1eaa87298 containerID="7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8" exitCode=0 Nov 16 12:28:09.848309 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:09.848316 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-g5brb" event=&{ID:71cd415c-6580-4fd9-ab24-b9c1eaa87298 Type:ContainerDied Data:7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8} Nov 16 12:28:09.848945 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:09.848366 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-g5brb" event=&{ID:71cd415c-6580-4fd9-ab24-b9c1eaa87298 Type:ContainerDied Data:627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9} Nov 16 12:28:09.848945 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:09.848389 1563 scope.go:110] "RemoveContainer" containerID="7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8" Nov 16 12:28:09.849626 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:09.849581380Z" level=info msg="Removing container: 7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8" id=70fc2324-c54a-4ed6-8bb0-35b0a0138a5b name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:28:09.883185 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:09.883097627Z" level=info msg="Removed container 7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8: openshift-marketplace/qe-app-registry-g5brb/registry-server" id=70fc2324-c54a-4ed6-8bb0-35b0a0138a5b name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:28:09.883440 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:09.883407 1563 scope.go:110] "RemoveContainer" containerID="7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8" Nov 16 12:28:09.883885 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 12:28:09.883820 1563 remote_runtime.go:597] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8\": container with ID starting with 7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8 not found: ID does not exist" containerID="7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8" Nov 16 12:28:09.883885 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:09.883873 1563 pod_container_deletor.go:52] "DeleteContainer returned error" containerID={Type:cri-o ID:7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8} err="failed to get container status \"7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8\": rpc error: code = NotFound desc = could not find container \"7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8\": container with ID starting with 7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8 not found: ID does not exist" Nov 16 12:28:09.902493 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:09.902400 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njmqb\" (UniqueName: \"kubernetes.io/projected/71cd415c-6580-4fd9-ab24-b9c1eaa87298-kube-api-access-njmqb\") pod \"71cd415c-6580-4fd9-ab24-b9c1eaa87298\" (UID: \"71cd415c-6580-4fd9-ab24-b9c1eaa87298\") " Nov 16 12:28:09.913855 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:09.913806 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71cd415c-6580-4fd9-ab24-b9c1eaa87298-kube-api-access-njmqb" (OuterVolumeSpecName: "kube-api-access-njmqb") pod "71cd415c-6580-4fd9-ab24-b9c1eaa87298" (UID: "71cd415c-6580-4fd9-ab24-b9c1eaa87298"). InnerVolumeSpecName "kube-api-access-njmqb". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 12:28:10.003637 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:10.003395 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-njmqb\" (UniqueName: \"kubernetes.io/projected/71cd415c-6580-4fd9-ab24-b9c1eaa87298-kube-api-access-njmqb\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 12:28:10.003776 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-43ffd1122e55c3b5e420e6f63a29c2f314b913e39909cb4afb7bf94c615a8b25-merged.mount: Succeeded. Nov 16 12:28:10.003811 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-71cd415c\x2d6580\x2d4fd9\x2dab24\x2db9c1eaa87298-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dnjmqb.mount: Succeeded. Nov 16 12:28:10.156437 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod71cd415c_6580_4fd9_ab24_b9c1eaa87298.slice. Nov 16 12:28:10.156500 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod71cd415c_6580_4fd9_ab24_b9c1eaa87298.slice: Consumed 720ms CPU time Nov 16 12:28:10.178340 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:10.178267 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-g5brb] Nov 16 12:28:10.181196 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:10.181122 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-g5brb] Nov 16 12:28:10.591959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:10.591897 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-g5brb" podUID=71cd415c-6580-4fd9-ab24-b9c1eaa87298 containerName="registry-server" containerID="cri-o://7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8" gracePeriod=1 Nov 16 12:28:10.592498 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:10.592253746Z" level=info msg="Stopping container: 7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8 (timeout: 1s)" id=879550b2-50f9-4bec-81bf-286c55db8e2e name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:28:10.618376 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 12:28:10.618307 1563 remote_runtime.go:479] "StopContainer from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8\": container with ID starting with 7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8 not found: ID does not exist" containerID="7af53fd90ca0a22605fe481e86835ecb0b3d0312b8979f881e45482163df7be8" Nov 16 12:28:10.618711 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:10.618666840Z" level=info msg="Stopping pod sandbox: 627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9" id=b2e97e58-0cd5-4f95-b24b-fd27b16b95b3 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:28:10.618834 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:10.618717341Z" level=info msg="Stopped pod sandbox (already stopped): 627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9" id=b2e97e58-0cd5-4f95-b24b-fd27b16b95b3 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:28:10.619570 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:28:10.619538 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=71cd415c-6580-4fd9-ab24-b9c1eaa87298 path="/var/lib/kubelet/pods/71cd415c-6580-4fd9-ab24-b9c1eaa87298/volumes" Nov 16 12:28:20.347301 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:20.347101189Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=c8f3b357-9c9d-4ae7-a925-22e1f3b3fbc8 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:28:20.514138 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:20.347356391Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=c8f3b357-9c9d-4ae7-a925-22e1f3b3fbc8 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:28:20.954739 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:20.954686756Z" level=info msg="Stopping pod sandbox: 627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9" id=ac72531c-0f86-4d66-948f-fca4bffdd7c3 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:28:20.954739 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:20.954742256Z" level=info msg="Stopped pod sandbox (already stopped): 627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9" id=ac72531c-0f86-4d66-948f-fca4bffdd7c3 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:28:20.955046 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:20.955016358Z" level=info msg="Removing pod sandbox: 627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9" id=552f1378-479e-4b41-a5ff-1320b64a98b8 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 12:28:20.980750 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:28:20.980660942Z" level=info msg="Removed pod sandbox: 627bd198243356c321f3673c2492c99a14c4385f2b69697335727bbc004dbec9" id=552f1378-479e-4b41-a5ff-1320b64a98b8 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 12:33:20.352323 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:33:20.352147987Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=0feb5a4e-f2de-4e06-bab8-faf34185b57e name=/runtime.v1.ImageService/ImageStatus Nov 16 12:33:20.372739 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:33:20.352452189Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=0feb5a4e-f2de-4e06-bab8-faf34185b57e name=/runtime.v1.ImageService/ImageStatus Nov 16 12:37:46.980344 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.YOjwGN.mount: Succeeded. Nov 16 12:38:20.357200 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:38:20.356958819Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=44ae3504-0fb6-4d92-89e1-df1574a8f994 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:38:20.403668 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:38:20.357297722Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=44ae3504-0fb6-4d92-89e1-df1574a8f994 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:39:16.979544 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.O7Zg56.mount: Succeeded. Nov 16 12:43:20.362267 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:20.362061391Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=bc7da566-c87c-4b80-80bb-b940bddb240c name=/runtime.v1.ImageService/ImageStatus Nov 16 12:43:20.380083 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:20.362362393Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=bc7da566-c87c-4b80-80bb-b940bddb240c name=/runtime.v1.ImageService/ImageStatus Nov 16 12:43:24.678242 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:24.677247 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-rm2mf] Nov 16 12:43:24.678242 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:24.677302 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 12:43:24.678242 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 12:43:24.677391 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="71cd415c-6580-4fd9-ab24-b9c1eaa87298" containerName="registry-server" Nov 16 12:43:24.678242 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:24.677404 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="71cd415c-6580-4fd9-ab24-b9c1eaa87298" containerName="registry-server" Nov 16 12:43:24.678242 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:24.677466 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="71cd415c-6580-4fd9-ab24-b9c1eaa87298" containerName="registry-server" Nov 16 12:43:24.694071 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod207220f9_d557_45ae_b108_c5b283973d99.slice. Nov 16 12:43:24.867457 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:24.867402 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwn7j\" (UniqueName: \"kubernetes.io/projected/207220f9-d557-45ae-b108-c5b283973d99-kube-api-access-wwn7j\") pod \"qe-app-registry-rm2mf\" (UID: \"207220f9-d557-45ae-b108-c5b283973d99\") " pod="openshift-marketplace/qe-app-registry-rm2mf" Nov 16 12:43:24.968070 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:24.967798 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-wwn7j\" (UniqueName: \"kubernetes.io/projected/207220f9-d557-45ae-b108-c5b283973d99-kube-api-access-wwn7j\") pod \"qe-app-registry-rm2mf\" (UID: \"207220f9-d557-45ae-b108-c5b283973d99\") " pod="openshift-marketplace/qe-app-registry-rm2mf" Nov 16 12:43:24.996359 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:24.996283 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwn7j\" (UniqueName: \"kubernetes.io/projected/207220f9-d557-45ae-b108-c5b283973d99-kube-api-access-wwn7j\") pod \"qe-app-registry-rm2mf\" (UID: \"207220f9-d557-45ae-b108-c5b283973d99\") " pod="openshift-marketplace/qe-app-registry-rm2mf" Nov 16 12:43:25.009033 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:25.008969 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-rm2mf" Nov 16 12:43:25.009772 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:25.009588045Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-rm2mf/POD" id=425cc117-e2cc-47f5-997f-be5527af81e2 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:43:25.009772 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:25.009761846Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:43:25.079733 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:25.079649425Z" level=info msg="Got pod network &{Name:qe-app-registry-rm2mf Namespace:openshift-marketplace ID:0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8 UID:207220f9-d557-45ae-b108-c5b283973d99 NetNS:/var/run/netns/dcf7b3b4-2d92-4ee3-9883-ccfb751fbd93 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:43:25.079733 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:25.079696225Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-rm2mf to CNI network \"multus-cni-network\" (type=multus)" Nov 16 12:43:25.129003 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 12:43:25.147626 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668602605.1476] manager: (veth32b6e0ff): new Veth device (/org/freedesktop/NetworkManager/Devices/34) Nov 16 12:43:25.148508 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668602605.1484] device (veth32b6e0ff): carrier: link connected Nov 16 12:43:25.153755 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth32b6e0ff: link is not ready Nov 16 12:43:25.153847 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth32b6e0ff: link becomes ready Nov 16 12:43:25.153989 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 12:43:25.161282 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[48439]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 12:43:25.161480 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[48439]: Could not generate persistent MAC address for veth32b6e0ff: No such file or directory Nov 16 12:43:25.174359 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668602605.1743] manager: (veth32b6e0ff): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/35) Nov 16 12:43:25.179513 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00104|bridge|INFO|bridge br0: added interface veth32b6e0ff on port 15 Nov 16 12:43:25.180208 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth32b6e0ff entered promiscuous mode Nov 16 12:43:25.212311 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00105|connmgr|INFO|br0<->unix#178: 5 flow_mods in the last 0 s (5 adds) Nov 16 12:43:25.257391 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00106|connmgr|INFO|br0<->unix#181: 2 flow_mods in the last 0 s (2 deletes) Nov 16 12:43:26.872512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:26.871108 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-marketplace/qe-app-registry-rm2mf] Nov 16 12:43:26.872999 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T12:43:26Z [verbose] Add: openshift-marketplace:qe-app-registry-rm2mf:207220f9-d557-45ae-b108-c5b283973d99:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/dcf7b3b4-2d92-4ee3-9883-ccfb751fbd93"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.14/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 12:43:26.872999 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 12:43:26.849915 48421 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-rm2mf", UID:"207220f9-d557-45ae-b108-c5b283973d99", APIVersion:"v1", ResourceVersion:"44144", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.14/23] from openshift-sdn Nov 16 12:43:26.872999 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:26.872438696Z" level=info msg="Got pod network &{Name:qe-app-registry-rm2mf Namespace:openshift-marketplace ID:0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8 UID:207220f9-d557-45ae-b108-c5b283973d99 NetNS:/var/run/netns/dcf7b3b4-2d92-4ee3-9883-ccfb751fbd93 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:43:26.872999 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:26.872580197Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-rm2mf for CNI network multus-cni-network (type=multus)" Nov 16 12:43:26.885176 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 12:43:26.885100 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod207220f9_d557_45ae_b108_c5b283973d99.slice/crio-0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8.scope WatchSource:0}: Error finding container 0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8: Status 404 returned error &{%!s(*http.body=&{0xc0024d0558 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 12:43:26.888205 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:26.887349898Z" level=info msg="Ran pod sandbox 0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8 with infra container: openshift-marketplace/qe-app-registry-rm2mf/POD" id=425cc117-e2cc-47f5-997f-be5527af81e2 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 12:43:26.889351 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:26.889297811Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=dc8b1b47-044d-4ab0-b243-17ea9f487f38 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:43:26.889574 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:26.889535513Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=dc8b1b47-044d-4ab0-b243-17ea9f487f38 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:43:26.890047 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:26.890019 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 12:43:26.890719 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:26.890682321Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=0346b03f-d604-4254-87ab-c8473e5cc8a9 name=/runtime.v1.ImageService/PullImage Nov 16 12:43:26.893846 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:26.893771642Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 12:43:27.305310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:27.305157 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-rm2mf" event=&{ID:207220f9-d557-45ae-b108-c5b283973d99 Type:ContainerStarted Data:0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8} Nov 16 12:43:28.441737 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:28.441598936Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=0346b03f-d604-4254-87ab-c8473e5cc8a9 name=/runtime.v1.ImageService/PullImage Nov 16 12:43:28.442932 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:28.442853944Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=2636fb90-9132-4804-b2ea-8dbbf4e2f984 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:43:28.443108 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:28.443069946Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=2636fb90-9132-4804-b2ea-8dbbf4e2f984 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:43:28.444207 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:28.444167153Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-rm2mf/registry-server" id=6e83f48e-9911-495e-b347-c370f6d4e2ad name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:43:28.444299 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:28.444277854Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 12:43:28.496946 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07.scope. Nov 16 12:43:28.531626 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07. Nov 16 12:43:28.567724 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 12:43:28.645738 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:28.645638632Z" level=info msg="Created container 9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07: openshift-marketplace/qe-app-registry-rm2mf/registry-server" id=6e83f48e-9911-495e-b347-c370f6d4e2ad name=/runtime.v1.RuntimeService/CreateContainer Nov 16 12:43:28.646279 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:28.646243137Z" level=info msg="Starting container: 9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07" id=283c3670-4fca-47f9-946f-33430b80caa2 name=/runtime.v1.RuntimeService/StartContainer Nov 16 12:43:28.657270 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:28.657224212Z" level=info msg="Started container" PID=48568 containerID=9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07 description=openshift-marketplace/qe-app-registry-rm2mf/registry-server id=283c3670-4fca-47f9-946f-33430b80caa2 name=/runtime.v1.RuntimeService/StartContainer sandboxID=0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8 Nov 16 12:43:29.311884 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:29.311848 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-rm2mf" event=&{ID:207220f9-d557-45ae-b108-c5b283973d99 Type:ContainerStarted Data:9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07} Nov 16 12:43:35.009567 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:35.009509 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/qe-app-registry-rm2mf" Nov 16 12:43:35.009567 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:35.009567 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/qe-app-registry-rm2mf" Nov 16 12:43:35.065853 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:35.065800 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/qe-app-registry-rm2mf" Nov 16 12:43:35.379115 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:35.379068 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/qe-app-registry-rm2mf" Nov 16 12:43:35.423752 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:35.423685 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-rm2mf] Nov 16 12:43:37.332709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:37.332649 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-rm2mf" podUID=207220f9-d557-45ae-b108-c5b283973d99 containerName="registry-server" containerID="cri-o://9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07" gracePeriod=1 Nov 16 12:43:37.333277 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:37.332900544Z" level=info msg="Stopping container: 9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07 (timeout: 1s)" id=4d913a79-02b6-4262-981d-2555489fa21a name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:43:37.356321 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07.scope: Succeeded. Nov 16 12:43:37.357024 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07.scope: Consumed 705ms CPU time Nov 16 12:43:37.371230 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07.scope: Succeeded. Nov 16 12:43:37.371852 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07.scope: Consumed 41ms CPU time Nov 16 12:43:37.485817 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-c286dfb36957d364bdc6076cfc2ab9ed27543547e1d5a69507d1b9b73f488b1f-merged.mount: Succeeded. Nov 16 12:43:37.809180 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:37.809027283Z" level=info msg="Stopped container 9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07: openshift-marketplace/qe-app-registry-rm2mf/registry-server" id=4d913a79-02b6-4262-981d-2555489fa21a name=/runtime.v1.RuntimeService/StopContainer Nov 16 12:43:37.809766 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:37.809728589Z" level=info msg="Stopping pod sandbox: 0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8" id=230ccf66-9fa6-4753-b4df-1555ff091cc4 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:43:37.810067 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:37.810025691Z" level=info msg="Got pod network &{Name:qe-app-registry-rm2mf Namespace:openshift-marketplace ID:0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8 UID:207220f9-d557-45ae-b108-c5b283973d99 NetNS:/var/run/netns/dcf7b3b4-2d92-4ee3-9883-ccfb751fbd93 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 12:43:37.810244 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:37.810201392Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-rm2mf from CNI network \"multus-cni-network\" (type=multus)" Nov 16 12:43:37.878304 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00107|connmgr|INFO|br0<->unix#185: 2 flow_mods in the last 0 s (2 deletes) Nov 16 12:43:37.908879 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00108|connmgr|INFO|br0<->unix#188: 4 flow_mods in the last 0 s (4 deletes) Nov 16 12:43:37.938357 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00109|bridge|INFO|bridge br0: deleted interface veth32b6e0ff on port 15 Nov 16 12:43:37.945151 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth32b6e0ff left promiscuous mode Nov 16 12:43:37.953428 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T12:43:37Z [verbose] Del: openshift-marketplace:qe-app-registry-rm2mf:207220f9-d557-45ae-b108-c5b283973d99:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 12:43:37.960720 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-dcf7b3b4\x2d2d92\x2d4ee3\x2d9883\x2dccfb751fbd93.mount: Succeeded. Nov 16 12:43:37.969219 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-dcf7b3b4\x2d2d92\x2d4ee3\x2d9883\x2dccfb751fbd93.mount: Succeeded. Nov 16 12:43:37.985435 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-dcf7b3b4\x2d2d92\x2d4ee3\x2d9883\x2dccfb751fbd93.mount: Succeeded. Nov 16 12:43:37.993486 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8-userdata-shm.mount: Succeeded. Nov 16 12:43:38.157440 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:38.157380846Z" level=info msg="Stopped pod sandbox: 0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8" id=230ccf66-9fa6-4753-b4df-1555ff091cc4 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:43:38.280041 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:38.279982 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwn7j\" (UniqueName: \"kubernetes.io/projected/207220f9-d557-45ae-b108-c5b283973d99-kube-api-access-wwn7j\") pod \"207220f9-d557-45ae-b108-c5b283973d99\" (UID: \"207220f9-d557-45ae-b108-c5b283973d99\") " Nov 16 12:43:38.293610 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:38.293575 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/207220f9-d557-45ae-b108-c5b283973d99-kube-api-access-wwn7j" (OuterVolumeSpecName: "kube-api-access-wwn7j") pod "207220f9-d557-45ae-b108-c5b283973d99" (UID: "207220f9-d557-45ae-b108-c5b283973d99"). InnerVolumeSpecName "kube-api-access-wwn7j". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 12:43:38.336589 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:38.336562 1563 generic.go:296] "Generic (PLEG): container finished" podID=207220f9-d557-45ae-b108-c5b283973d99 containerID="9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07" exitCode=0 Nov 16 12:43:38.337010 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:38.336609 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-rm2mf" event=&{ID:207220f9-d557-45ae-b108-c5b283973d99 Type:ContainerDied Data:9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07} Nov 16 12:43:38.337010 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:38.336636 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-rm2mf" event=&{ID:207220f9-d557-45ae-b108-c5b283973d99 Type:ContainerDied Data:0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8} Nov 16 12:43:38.337010 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:38.336657 1563 scope.go:110] "RemoveContainer" containerID="9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07" Nov 16 12:43:38.337811 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:38.337721125Z" level=info msg="Removing container: 9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07" id=ceff24c5-b1b9-4173-81de-2ccf3c99b902 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:43:38.344021 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod207220f9_d557_45ae_b108_c5b283973d99.slice. Nov 16 12:43:38.344079 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod207220f9_d557_45ae_b108_c5b283973d99.slice: Consumed 747ms CPU time Nov 16 12:43:38.369321 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:43:38.369269066Z" level=info msg="Removed container 9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07: openshift-marketplace/qe-app-registry-rm2mf/registry-server" id=ceff24c5-b1b9-4173-81de-2ccf3c99b902 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 12:43:38.369576 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:38.369526 1563 scope.go:110] "RemoveContainer" containerID="9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07" Nov 16 12:43:38.370152 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 12:43:38.370062 1563 remote_runtime.go:597] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07\": container with ID starting with 9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07 not found: ID does not exist" containerID="9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07" Nov 16 12:43:38.370152 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:38.370109 1563 pod_container_deletor.go:52] "DeleteContainer returned error" containerID={Type:cri-o ID:9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07} err="failed to get container status \"9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07\": rpc error: code = NotFound desc = could not find container \"9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07\": container with ID starting with 9b9f50a5d10f709cf07cdb006c54514e67f80de4c567d729b72d2a2e100f9d07 not found: ID does not exist" Nov 16 12:43:38.370423 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:38.370392 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-rm2mf] Nov 16 12:43:38.377674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:38.377605 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-rm2mf] Nov 16 12:43:38.381254 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:38.381221 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-wwn7j\" (UniqueName: \"kubernetes.io/projected/207220f9-d557-45ae-b108-c5b283973d99-kube-api-access-wwn7j\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 12:43:38.485200 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-2dfe164981050cf9bdc24192b09a636cdda1df95cbb74fe18770f37eb567cb4a-merged.mount: Succeeded. Nov 16 12:43:38.485234 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-207220f9\x2dd557\x2d45ae\x2db108\x2dc5b283973d99-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dwwn7j.mount: Succeeded. Nov 16 12:43:38.596189 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 12:43:38.596090 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=207220f9-d557-45ae-b108-c5b283973d99 path="/var/lib/kubelet/pods/207220f9-d557-45ae-b108-c5b283973d99/volumes" Nov 16 12:44:21.045351 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:44:21.045198032Z" level=info msg="Stopping pod sandbox: 0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8" id=b1b0f041-1846-4780-b265-57f6aa1f2a64 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:44:21.045351 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:44:21.045265632Z" level=info msg="Stopped pod sandbox (already stopped): 0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8" id=b1b0f041-1846-4780-b265-57f6aa1f2a64 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 12:44:21.045931 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:44:21.045579035Z" level=info msg="Removing pod sandbox: 0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8" id=4e006ec7-42a3-42c5-857a-5aabdb596cc0 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 12:44:21.067379 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:44:21.067264600Z" level=info msg="Removed pod sandbox: 0bb10f28ebb132cbd5afc5184e83c96cfb05ca96e6db5515dba84537f2ab7fc8" id=4e006ec7-42a3-42c5-857a-5aabdb596cc0 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 12:48:06.979473 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.cl0N7V.mount: Succeeded. Nov 16 12:48:20.367195 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:48:20.367023052Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=1513c843-0d52-4121-8bf5-2fc87a619c79 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:48:20.499057 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:48:20.367304054Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=1513c843-0d52-4121-8bf5-2fc87a619c79 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:49:36.978806 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.SrdHcU.mount: Succeeded. Nov 16 12:53:20.372070 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:53:20.371904007Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=7845ab2f-34d3-4e22-bbd8-e8071d6cc178 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:53:20.395323 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:53:20.372632313Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=7845ab2f-34d3-4e22-bbd8-e8071d6cc178 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:58:20.378729 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:58:20.378504063Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=6ccd70a8-48ee-4507-9afc-5bb92850a078 name=/runtime.v1.ImageService/ImageStatus Nov 16 12:58:20.398826 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 12:58:20.378812066Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=6ccd70a8-48ee-4507-9afc-5bb92850a078 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:00:11.668366 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:11.667374 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-gwwph] Nov 16 13:00:11.668366 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:11.667502 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 13:00:11.668366 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 13:00:11.667586 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="207220f9-d557-45ae-b108-c5b283973d99" containerName="registry-server" Nov 16 13:00:11.668366 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:11.667600 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="207220f9-d557-45ae-b108-c5b283973d99" containerName="registry-server" Nov 16 13:00:11.668366 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:11.667657 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="207220f9-d557-45ae-b108-c5b283973d99" containerName="registry-server" Nov 16 13:00:11.682436 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod5d8b226e_0a01_4c17_a02b_bc7cf29173ea.slice. Nov 16 13:00:11.812724 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:11.812659 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt7bn\" (UniqueName: \"kubernetes.io/projected/5d8b226e-0a01-4c17-a02b-bc7cf29173ea-kube-api-access-rt7bn\") pod \"qe-app-registry-gwwph\" (UID: \"5d8b226e-0a01-4c17-a02b-bc7cf29173ea\") " pod="openshift-marketplace/qe-app-registry-gwwph" Nov 16 13:00:11.913471 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:11.913347 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-rt7bn\" (UniqueName: \"kubernetes.io/projected/5d8b226e-0a01-4c17-a02b-bc7cf29173ea-kube-api-access-rt7bn\") pod \"qe-app-registry-gwwph\" (UID: \"5d8b226e-0a01-4c17-a02b-bc7cf29173ea\") " pod="openshift-marketplace/qe-app-registry-gwwph" Nov 16 13:00:11.941580 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:11.941455 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt7bn\" (UniqueName: \"kubernetes.io/projected/5d8b226e-0a01-4c17-a02b-bc7cf29173ea-kube-api-access-rt7bn\") pod \"qe-app-registry-gwwph\" (UID: \"5d8b226e-0a01-4c17-a02b-bc7cf29173ea\") " pod="openshift-marketplace/qe-app-registry-gwwph" Nov 16 13:00:12.008819 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:12.008709 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-gwwph" Nov 16 13:00:12.009844 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:12.009729105Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-gwwph/POD" id=bd6abca6-6ef7-4d76-9afe-8ba56418bcf9 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 13:00:12.009844 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:12.009836106Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 13:00:12.093840 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:12.093766950Z" level=info msg="Got pod network &{Name:qe-app-registry-gwwph Namespace:openshift-marketplace ID:b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874 UID:5d8b226e-0a01-4c17-a02b-bc7cf29173ea NetNS:/var/run/netns/d0b03e11-6b39-410d-bf29-f560447ab631 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 13:00:12.093840 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:12.093806550Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-gwwph to CNI network \"multus-cni-network\" (type=multus)" Nov 16 13:00:12.141282 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 13:00:12.165384 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth701efe26: link is not ready Nov 16 13:00:12.165474 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth701efe26: link becomes ready Nov 16 13:00:12.165505 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 13:00:12.170875 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[64704]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 13:00:12.171025 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[64704]: Could not generate persistent MAC address for veth701efe26: No such file or directory Nov 16 13:00:12.171202 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668603612.1711] manager: (veth701efe26): new Veth device (/org/freedesktop/NetworkManager/Devices/36) Nov 16 13:00:12.172267 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668603612.1722] device (veth701efe26): carrier: link connected Nov 16 13:00:12.191924 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668603612.1919] manager: (veth701efe26): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/37) Nov 16 13:00:12.197185 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth701efe26 entered promiscuous mode Nov 16 13:00:12.197243 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00110|bridge|INFO|bridge br0: added interface veth701efe26 on port 16 Nov 16 13:00:12.228673 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00111|connmgr|INFO|br0<->unix#200: 5 flow_mods in the last 0 s (5 adds) Nov 16 13:00:12.272276 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00112|connmgr|INFO|br0<->unix#203: 2 flow_mods in the last 0 s (2 deletes) Nov 16 13:00:14.244443 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T13:00:14Z [verbose] Add: openshift-marketplace:qe-app-registry-gwwph:5d8b226e-0a01-4c17-a02b-bc7cf29173ea:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/d0b03e11-6b39-410d-bf29-f560447ab631"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.15/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 13:00:14.244443 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 13:00:14.215893 64686 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-gwwph", UID:"5d8b226e-0a01-4c17-a02b-bc7cf29173ea", APIVersion:"v1", ResourceVersion:"49885", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.15/23] from openshift-sdn Nov 16 13:00:14.244934 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:14.244521 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-marketplace/qe-app-registry-gwwph] Nov 16 13:00:14.245697 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:14.245597399Z" level=info msg="Got pod network &{Name:qe-app-registry-gwwph Namespace:openshift-marketplace ID:b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874 UID:5d8b226e-0a01-4c17-a02b-bc7cf29173ea NetNS:/var/run/netns/d0b03e11-6b39-410d-bf29-f560447ab631 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 13:00:14.245854 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:14.245825101Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-gwwph for CNI network multus-cni-network (type=multus)" Nov 16 13:00:14.248466 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 13:00:14.248428 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d8b226e_0a01_4c17_a02b_bc7cf29173ea.slice/crio-b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874.scope WatchSource:0}: Error finding container b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874: Status 404 returned error &{%!s(*http.body=&{0xc0028b3098 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 13:00:14.257532 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:14.257457985Z" level=info msg="Ran pod sandbox b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874 with infra container: openshift-marketplace/qe-app-registry-gwwph/POD" id=bd6abca6-6ef7-4d76-9afe-8ba56418bcf9 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 13:00:14.258591 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:14.258517693Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=9eaf70f8-63c8-413e-8143-6ded7ac54f02 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:00:14.258780 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:14.258743394Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=9eaf70f8-63c8-413e-8143-6ded7ac54f02 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:00:14.259275 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:14.259247 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 13:00:14.259940 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:14.259825102Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=52cb9ecf-c6fc-40b2-bc4f-1a88b9a6c3bf name=/runtime.v1.ImageService/PullImage Nov 16 13:00:14.263212 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:14.263176326Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 13:00:15.097140 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:15.097086 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-gwwph" event=&{ID:5d8b226e-0a01-4c17-a02b-bc7cf29173ea Type:ContainerStarted Data:b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874} Nov 16 13:00:15.799665 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:15.799518608Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=52cb9ecf-c6fc-40b2-bc4f-1a88b9a6c3bf name=/runtime.v1.ImageService/PullImage Nov 16 13:00:15.800731 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:15.800687517Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=032c95a4-5358-4046-a4d6-2c180294b03b name=/runtime.v1.ImageService/ImageStatus Nov 16 13:00:15.800966 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:15.800926419Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=032c95a4-5358-4046-a4d6-2c180294b03b name=/runtime.v1.ImageService/ImageStatus Nov 16 13:00:15.802196 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:15.802155727Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-gwwph/registry-server" id=5c60c9b7-de23-47ee-affc-be891c62515b name=/runtime.v1.RuntimeService/CreateContainer Nov 16 13:00:15.802295 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:15.802266728Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 13:00:15.947870 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f.scope. Nov 16 13:00:15.974684 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f. Nov 16 13:00:16.010145 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 13:00:16.156891 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:16.156779186Z" level=info msg="Created container be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f: openshift-marketplace/qe-app-registry-gwwph/registry-server" id=5c60c9b7-de23-47ee-affc-be891c62515b name=/runtime.v1.RuntimeService/CreateContainer Nov 16 13:00:16.157677 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:16.157633192Z" level=info msg="Starting container: be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f" id=fa432cb6-cba9-4832-bb7b-98174cf80ea8 name=/runtime.v1.RuntimeService/StartContainer Nov 16 13:00:16.169594 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:16.169480777Z" level=info msg="Started container" PID=64792 containerID=be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f description=openshift-marketplace/qe-app-registry-gwwph/registry-server id=fa432cb6-cba9-4832-bb7b-98174cf80ea8 name=/runtime.v1.RuntimeService/StartContainer sandboxID=b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874 Nov 16 13:00:17.105445 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:17.105398 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-gwwph" event=&{ID:5d8b226e-0a01-4c17-a02b-bc7cf29173ea Type:ContainerStarted Data:be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f} Nov 16 13:00:22.009536 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:22.009403 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/qe-app-registry-gwwph" Nov 16 13:00:22.009536 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:22.009481 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/qe-app-registry-gwwph" Nov 16 13:00:22.078937 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:22.078888 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/qe-app-registry-gwwph" Nov 16 13:00:22.177839 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:22.177789 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/qe-app-registry-gwwph" Nov 16 13:00:22.223175 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:22.223071 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-gwwph] Nov 16 13:00:24.127574 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:24.127467 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-gwwph" podUID=5d8b226e-0a01-4c17-a02b-bc7cf29173ea containerName="registry-server" containerID="cri-o://be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f" gracePeriod=1 Nov 16 13:00:24.128065 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:24.127770428Z" level=info msg="Stopping container: be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f (timeout: 1s)" id=014631f1-ec8e-41a9-95db-e1d55f3ba185 name=/runtime.v1.RuntimeService/StopContainer Nov 16 13:00:24.156250 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f.scope: Succeeded. Nov 16 13:00:24.157035 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f.scope: Consumed 658ms CPU time Nov 16 13:00:24.168730 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f.scope: Succeeded. Nov 16 13:00:24.169367 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f.scope: Consumed 38ms CPU time Nov 16 13:00:24.284823 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-de6cef58839dabe236ccceb4cc9d98210ace06575ae33fc2c65e8924c0d0d20f-merged.mount: Succeeded. Nov 16 13:00:24.621846 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:24.621785274Z" level=info msg="Stopped container be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f: openshift-marketplace/qe-app-registry-gwwph/registry-server" id=014631f1-ec8e-41a9-95db-e1d55f3ba185 name=/runtime.v1.RuntimeService/StopContainer Nov 16 13:00:24.622233 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:24.622195777Z" level=info msg="Stopping pod sandbox: b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874" id=853d831c-171c-444b-924f-bfa91c58a6a6 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:00:24.622475 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:24.622433579Z" level=info msg="Got pod network &{Name:qe-app-registry-gwwph Namespace:openshift-marketplace ID:b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874 UID:5d8b226e-0a01-4c17-a02b-bc7cf29173ea NetNS:/var/run/netns/d0b03e11-6b39-410d-bf29-f560447ab631 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 13:00:24.622661 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:24.622636781Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-gwwph from CNI network \"multus-cni-network\" (type=multus)" Nov 16 13:00:24.692736 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00113|connmgr|INFO|br0<->unix#206: 2 flow_mods in the last 0 s (2 deletes) Nov 16 13:00:24.723099 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00114|connmgr|INFO|br0<->unix#209: 4 flow_mods in the last 0 s (4 deletes) Nov 16 13:00:24.750922 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00115|bridge|INFO|bridge br0: deleted interface veth701efe26 on port 16 Nov 16 13:00:24.757189 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth701efe26 left promiscuous mode Nov 16 13:00:24.764737 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T13:00:24Z [verbose] Del: openshift-marketplace:qe-app-registry-gwwph:5d8b226e-0a01-4c17-a02b-bc7cf29173ea:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 13:00:24.771550 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-d0b03e11\x2d6b39\x2d410d\x2dbf29\x2df560447ab631.mount: Succeeded. Nov 16 13:00:24.777458 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-d0b03e11\x2d6b39\x2d410d\x2dbf29\x2df560447ab631.mount: Succeeded. Nov 16 13:00:24.798566 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-d0b03e11\x2d6b39\x2d410d\x2dbf29\x2df560447ab631.mount: Succeeded. Nov 16 13:00:24.805266 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874-userdata-shm.mount: Succeeded. Nov 16 13:00:25.132185 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:25.132141 1563 generic.go:296] "Generic (PLEG): container finished" podID=5d8b226e-0a01-4c17-a02b-bc7cf29173ea containerID="be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f" exitCode=0 Nov 16 13:00:25.132185 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:25.132198 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-gwwph" event=&{ID:5d8b226e-0a01-4c17-a02b-bc7cf29173ea Type:ContainerDied Data:be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f} Nov 16 13:00:25.282113 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:25.282001781Z" level=info msg="Stopped pod sandbox: b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874" id=853d831c-171c-444b-924f-bfa91c58a6a6 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:00:25.285263 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-dd147fda05cbd17a80bc18810ae9ff7f39b7cf63c8f9e5d6678de09953627ab5-merged.mount: Succeeded. Nov 16 13:00:25.427541 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:25.427350 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt7bn\" (UniqueName: \"kubernetes.io/projected/5d8b226e-0a01-4c17-a02b-bc7cf29173ea-kube-api-access-rt7bn\") pod \"5d8b226e-0a01-4c17-a02b-bc7cf29173ea\" (UID: \"5d8b226e-0a01-4c17-a02b-bc7cf29173ea\") " Nov 16 13:00:25.436436 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-5d8b226e\x2d0a01\x2d4c17\x2da02b\x2dbc7cf29173ea-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2drt7bn.mount: Succeeded. Nov 16 13:00:25.437678 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:25.437590 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d8b226e-0a01-4c17-a02b-bc7cf29173ea-kube-api-access-rt7bn" (OuterVolumeSpecName: "kube-api-access-rt7bn") pod "5d8b226e-0a01-4c17-a02b-bc7cf29173ea" (UID: "5d8b226e-0a01-4c17-a02b-bc7cf29173ea"). InnerVolumeSpecName "kube-api-access-rt7bn". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 13:00:25.528824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:25.528716 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-rt7bn\" (UniqueName: \"kubernetes.io/projected/5d8b226e-0a01-4c17-a02b-bc7cf29173ea-kube-api-access-rt7bn\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 13:00:26.137203 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:26.137153 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-gwwph" event=&{ID:5d8b226e-0a01-4c17-a02b-bc7cf29173ea Type:ContainerDied Data:b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874} Nov 16 13:00:26.137203 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:26.137231 1563 scope.go:110] "RemoveContainer" containerID="be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f" Nov 16 13:00:26.138376 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:26.138320575Z" level=info msg="Removing container: be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f" id=42bcfc06-2d8f-4d78-acf0-3724474a42c5 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 13:00:26.144896 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod5d8b226e_0a01_4c17_a02b_bc7cf29173ea.slice. Nov 16 13:00:26.144956 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod5d8b226e_0a01_4c17_a02b_bc7cf29173ea.slice: Consumed 696ms CPU time Nov 16 13:00:26.168295 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:26.168260 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-gwwph] Nov 16 13:00:26.172439 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:26.172407 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-gwwph] Nov 16 13:00:26.465634 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:00:26.465431756Z" level=info msg="Removed container be5d343ca9ebc7c06441487f8cfc41093aefffbee1da6e88fe9ddbc7edf5395f: openshift-marketplace/qe-app-registry-gwwph/registry-server" id=42bcfc06-2d8f-4d78-acf0-3724474a42c5 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 13:00:26.595902 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:00:26.595828 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=5d8b226e-0a01-4c17-a02b-bc7cf29173ea path="/var/lib/kubelet/pods/5d8b226e-0a01-4c17-a02b-bc7cf29173ea/volumes" Nov 16 13:01:21.132206 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:01:21.132149432Z" level=info msg="Stopping pod sandbox: b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874" id=dc3cc3d7-d07c-4b5d-b9d1-00901ec47a8a name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:01:21.132206 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:01:21.132205432Z" level=info msg="Stopped pod sandbox (already stopped): b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874" id=dc3cc3d7-d07c-4b5d-b9d1-00901ec47a8a name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:01:21.132824 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:01:21.132475534Z" level=info msg="Removing pod sandbox: b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874" id=fc8befea-a073-4a23-a576-d76ddc7aec10 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 13:01:21.149100 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:01:21.149063357Z" level=info msg="Removed pod sandbox: b9ed2f46fa624d16ae8bfe4d62621ba90117aaf9d2470b5d3cc8ed42daba9874" id=fc8befea-a073-4a23-a576-d76ddc7aec10 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 13:03:20.384330 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:03:20.384157590Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=29929ec4-1f8e-488d-86d7-c257494e23bf name=/runtime.v1.ImageService/ImageStatus Nov 16 13:03:20.402290 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:03:20.384434092Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=29929ec4-1f8e-488d-86d7-c257494e23bf name=/runtime.v1.ImageService/ImageStatus Nov 16 13:04:06.980308 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.5NLeWu.mount: Succeeded. Nov 16 13:06:16.978738 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.p05I7P.mount: Succeeded. Nov 16 13:08:20.389539 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:08:20.389209546Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=78efc2c1-77ab-4f30-81b1-e3d2e65ebf08 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:08:20.407348 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:08:20.389627849Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=78efc2c1-77ab-4f30-81b1-e3d2e65ebf08 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:13:18.348344 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00116|connmgr|INFO|br0<->unix#218: 2 flow_mods in the last 0 s (2 deletes) Nov 16 13:13:20.394498 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:13:20.394258927Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=9a43ae92-bde9-4e7a-93ed-9931191a10a0 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:13:20.421749 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:13:20.394604030Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=9a43ae92-bde9-4e7a-93ed-9931191a10a0 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:15:49.334281 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:49.334151 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-2pdbh] Nov 16 13:15:49.334281 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:49.334223 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 13:15:49.334923 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 13:15:49.334301 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="5d8b226e-0a01-4c17-a02b-bc7cf29173ea" containerName="registry-server" Nov 16 13:15:49.334923 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:49.334315 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d8b226e-0a01-4c17-a02b-bc7cf29173ea" containerName="registry-server" Nov 16 13:15:49.334923 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:49.334368 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="5d8b226e-0a01-4c17-a02b-bc7cf29173ea" containerName="registry-server" Nov 16 13:15:49.355093 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod847e27ee_d034_4a3d_89dd_da8f1ebb90b1.slice. Nov 16 13:15:49.436231 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:49.436101 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzglk\" (UniqueName: \"kubernetes.io/projected/847e27ee-d034-4a3d-89dd-da8f1ebb90b1-kube-api-access-lzglk\") pod \"qe-app-registry-2pdbh\" (UID: \"847e27ee-d034-4a3d-89dd-da8f1ebb90b1\") " pod="openshift-marketplace/qe-app-registry-2pdbh" Nov 16 13:15:49.537300 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:49.537194 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-lzglk\" (UniqueName: \"kubernetes.io/projected/847e27ee-d034-4a3d-89dd-da8f1ebb90b1-kube-api-access-lzglk\") pod \"qe-app-registry-2pdbh\" (UID: \"847e27ee-d034-4a3d-89dd-da8f1ebb90b1\") " pod="openshift-marketplace/qe-app-registry-2pdbh" Nov 16 13:15:49.571729 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:49.571668 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzglk\" (UniqueName: \"kubernetes.io/projected/847e27ee-d034-4a3d-89dd-da8f1ebb90b1-kube-api-access-lzglk\") pod \"qe-app-registry-2pdbh\" (UID: \"847e27ee-d034-4a3d-89dd-da8f1ebb90b1\") " pod="openshift-marketplace/qe-app-registry-2pdbh" Nov 16 13:15:49.678039 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:49.677992 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-2pdbh" Nov 16 13:15:49.679072 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:49.678937195Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-2pdbh/POD" id=da1a9a04-e9a7-49e2-bd92-720c31630edb name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 13:15:49.679552 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:49.679088296Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 13:15:49.728292 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:49.728173880Z" level=info msg="Got pod network &{Name:qe-app-registry-2pdbh Namespace:openshift-marketplace ID:7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26 UID:847e27ee-d034-4a3d-89dd-da8f1ebb90b1 NetNS:/var/run/netns/fe304235-e28f-433b-a10c-deb886ff92a3 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 13:15:49.728292 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:49.728226880Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-2pdbh to CNI network \"multus-cni-network\" (type=multus)" Nov 16 13:15:49.771175 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 13:15:49.796190 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth92fac283: link is not ready Nov 16 13:15:49.796279 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth92fac283: link becomes ready Nov 16 13:15:49.796312 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 13:15:49.802463 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[79819]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 13:15:49.802687 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[79819]: Could not generate persistent MAC address for veth92fac283: No such file or directory Nov 16 13:15:49.802894 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668604549.8028] device (veth92fac283): carrier: link connected Nov 16 13:15:49.803286 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668604549.8032] manager: (veth92fac283): new Veth device (/org/freedesktop/NetworkManager/Devices/38) Nov 16 13:15:49.814016 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668604549.8139] manager: (veth92fac283): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/39) Nov 16 13:15:49.819195 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth92fac283 entered promiscuous mode Nov 16 13:15:49.819177 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00117|bridge|INFO|bridge br0: added interface veth92fac283 on port 17 Nov 16 13:15:49.851563 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00118|connmgr|INFO|br0<->unix#223: 5 flow_mods in the last 0 s (5 adds) Nov 16 13:15:49.894976 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00119|connmgr|INFO|br0<->unix#226: 2 flow_mods in the last 0 s (2 deletes) Nov 16 13:15:51.938358 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:51.938316 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-marketplace/qe-app-registry-2pdbh] Nov 16 13:15:51.938837 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T13:15:51Z [verbose] Add: openshift-marketplace:qe-app-registry-2pdbh:847e27ee-d034-4a3d-89dd-da8f1ebb90b1:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/fe304235-e28f-433b-a10c-deb886ff92a3"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.16/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 13:15:51.938837 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 13:15:51.915487 79802 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-2pdbh", UID:"847e27ee-d034-4a3d-89dd-da8f1ebb90b1", APIVersion:"v1", ResourceVersion:"55356", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.16/23] from openshift-sdn Nov 16 13:15:51.939521 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:51.939471371Z" level=info msg="Got pod network &{Name:qe-app-registry-2pdbh Namespace:openshift-marketplace ID:7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26 UID:847e27ee-d034-4a3d-89dd-da8f1ebb90b1 NetNS:/var/run/netns/fe304235-e28f-433b-a10c-deb886ff92a3 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 13:15:51.939650 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:51.939623271Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-2pdbh for CNI network multus-cni-network (type=multus)" Nov 16 13:15:51.956403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 13:15:51.956346 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod847e27ee_d034_4a3d_89dd_da8f1ebb90b1.slice/crio-7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26.scope WatchSource:0}: Error finding container 7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26: Status 404 returned error &{%!s(*http.body=&{0xc0041677d0 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 13:15:51.958294 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:51.958257164Z" level=info msg="Ran pod sandbox 7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26 with infra container: openshift-marketplace/qe-app-registry-2pdbh/POD" id=da1a9a04-e9a7-49e2-bd92-720c31630edb name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 13:15:51.959358 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:51.959316870Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=5771b87b-63c6-4bea-bbdd-aa18c320e7a7 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:15:51.959697 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:51.959644171Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=5771b87b-63c6-4bea-bbdd-aa18c320e7a7 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:15:51.960478 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:51.960381 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 13:15:51.961076 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:51.961021378Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=59cb457d-b74b-4c10-9948-d37dac96e866 name=/runtime.v1.ImageService/PullImage Nov 16 13:15:51.964243 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:51.964208094Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 13:15:52.710334 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:52.710264 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-2pdbh" event=&{ID:847e27ee-d034-4a3d-89dd-da8f1ebb90b1 Type:ContainerStarted Data:7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26} Nov 16 13:15:53.566917 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:53.566789694Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=59cb457d-b74b-4c10-9948-d37dac96e866 name=/runtime.v1.ImageService/PullImage Nov 16 13:15:53.568309 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:53.568226601Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=c02a8fd3-15bb-41bf-907e-e43ef15b773e name=/runtime.v1.ImageService/ImageStatus Nov 16 13:15:53.568530 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:53.568480103Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=c02a8fd3-15bb-41bf-907e-e43ef15b773e name=/runtime.v1.ImageService/ImageStatus Nov 16 13:15:53.569771 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:53.569728809Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-2pdbh/registry-server" id=b6d2ceb6-4e5a-4eaa-b22d-d5e2f782fb33 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 13:15:53.569863 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:53.569831009Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 13:15:53.648396 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a.scope. Nov 16 13:15:53.675436 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a. Nov 16 13:15:53.713058 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 13:15:53.773571 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:53.773492126Z" level=info msg="Created container ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a: openshift-marketplace/qe-app-registry-2pdbh/registry-server" id=b6d2ceb6-4e5a-4eaa-b22d-d5e2f782fb33 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 13:15:53.773984 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:53.773950328Z" level=info msg="Starting container: ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a" id=cbf90dcf-7390-4972-89ae-6a75f24a0d19 name=/runtime.v1.RuntimeService/StartContainer Nov 16 13:15:53.783591 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:15:53.783550276Z" level=info msg="Started container" PID=79906 containerID=ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a description=openshift-marketplace/qe-app-registry-2pdbh/registry-server id=cbf90dcf-7390-4972-89ae-6a75f24a0d19 name=/runtime.v1.RuntimeService/StartContainer sandboxID=7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26 Nov 16 13:15:54.716045 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:54.716007 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-2pdbh" event=&{ID:847e27ee-d034-4a3d-89dd-da8f1ebb90b1 Type:ContainerStarted Data:ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a} Nov 16 13:15:59.679053 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:59.678933 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/qe-app-registry-2pdbh" Nov 16 13:15:59.679053 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:59.679015 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/qe-app-registry-2pdbh" Nov 16 13:15:59.730780 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:15:59.730745 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/qe-app-registry-2pdbh" Nov 16 13:16:00.789817 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:00.789742 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/qe-app-registry-2pdbh" Nov 16 13:16:00.830790 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:00.830695 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-2pdbh] Nov 16 13:16:02.736537 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:02.736394 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-2pdbh" podUID=847e27ee-d034-4a3d-89dd-da8f1ebb90b1 containerName="registry-server" containerID="cri-o://ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a" gracePeriod=1 Nov 16 13:16:02.737177 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:16:02.736792004Z" level=info msg="Stopping container: ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a (timeout: 1s)" id=73682fc7-7183-4ece-b92a-f27fbe44a629 name=/runtime.v1.RuntimeService/StopContainer Nov 16 13:16:02.765609 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a.scope: Succeeded. Nov 16 13:16:02.766407 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a.scope: Consumed 643ms CPU time Nov 16 13:16:02.774917 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a.scope: Succeeded. Nov 16 13:16:02.775509 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a.scope: Consumed 38ms CPU time Nov 16 13:16:02.892330 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-e886d9598c50cbe7a9fbe4a21cbe31770a3e49d0110622d067b7dfa64b30856e-merged.mount: Succeeded. Nov 16 13:16:03.243717 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:16:03.243600240Z" level=info msg="Stopped container ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a: openshift-marketplace/qe-app-registry-2pdbh/registry-server" id=73682fc7-7183-4ece-b92a-f27fbe44a629 name=/runtime.v1.RuntimeService/StopContainer Nov 16 13:16:03.244383 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:16:03.244349945Z" level=info msg="Stopping pod sandbox: 7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26" id=4dcba57b-a733-42ce-989d-b451cf6a278c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:16:03.244648 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:16:03.244599947Z" level=info msg="Got pod network &{Name:qe-app-registry-2pdbh Namespace:openshift-marketplace ID:7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26 UID:847e27ee-d034-4a3d-89dd-da8f1ebb90b1 NetNS:/var/run/netns/fe304235-e28f-433b-a10c-deb886ff92a3 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 13:16:03.244827 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:16:03.244799648Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-2pdbh from CNI network \"multus-cni-network\" (type=multus)" Nov 16 13:16:03.312320 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00120|connmgr|INFO|br0<->unix#229: 2 flow_mods in the last 0 s (2 deletes) Nov 16 13:16:03.340921 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00121|connmgr|INFO|br0<->unix#232: 4 flow_mods in the last 0 s (4 deletes) Nov 16 13:16:03.368107 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00122|bridge|INFO|bridge br0: deleted interface veth92fac283 on port 17 Nov 16 13:16:03.375157 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth92fac283 left promiscuous mode Nov 16 13:16:03.381900 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T13:16:03Z [verbose] Del: openshift-marketplace:qe-app-registry-2pdbh:847e27ee-d034-4a3d-89dd-da8f1ebb90b1:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 13:16:03.388639 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-fe304235\x2de28f\x2d433b\x2da10c\x2ddeb886ff92a3.mount: Succeeded. Nov 16 13:16:03.400214 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-fe304235\x2de28f\x2d433b\x2da10c\x2ddeb886ff92a3.mount: Succeeded. Nov 16 13:16:03.419531 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-fe304235\x2de28f\x2d433b\x2da10c\x2ddeb886ff92a3.mount: Succeeded. Nov 16 13:16:03.427548 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26-userdata-shm.mount: Succeeded. Nov 16 13:16:03.599595 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:16:03.599552954Z" level=info msg="Stopped pod sandbox: 7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26" id=4dcba57b-a733-42ce-989d-b451cf6a278c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:16:03.740563 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:03.740521 1563 generic.go:296] "Generic (PLEG): container finished" podID=847e27ee-d034-4a3d-89dd-da8f1ebb90b1 containerID="ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a" exitCode=0 Nov 16 13:16:03.740563 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:03.740570 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-2pdbh" event=&{ID:847e27ee-d034-4a3d-89dd-da8f1ebb90b1 Type:ContainerDied Data:ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a} Nov 16 13:16:03.741112 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:03.740613 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-2pdbh" event=&{ID:847e27ee-d034-4a3d-89dd-da8f1ebb90b1 Type:ContainerDied Data:7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26} Nov 16 13:16:03.741112 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:03.740636 1563 scope.go:110] "RemoveContainer" containerID="ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a" Nov 16 13:16:03.741908 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:16:03.741788418Z" level=info msg="Removing container: ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a" id=6937e79e-be9a-4776-b29c-52f6c920499b name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 13:16:03.751598 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:03.751560 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzglk\" (UniqueName: \"kubernetes.io/projected/847e27ee-d034-4a3d-89dd-da8f1ebb90b1-kube-api-access-lzglk\") pod \"847e27ee-d034-4a3d-89dd-da8f1ebb90b1\" (UID: \"847e27ee-d034-4a3d-89dd-da8f1ebb90b1\") " Nov 16 13:16:03.758731 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:03.758644 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/847e27ee-d034-4a3d-89dd-da8f1ebb90b1-kube-api-access-lzglk" (OuterVolumeSpecName: "kube-api-access-lzglk") pod "847e27ee-d034-4a3d-89dd-da8f1ebb90b1" (UID: "847e27ee-d034-4a3d-89dd-da8f1ebb90b1"). InnerVolumeSpecName "kube-api-access-lzglk". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 13:16:03.772314 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:16:03.772154824Z" level=info msg="Removed container ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a: openshift-marketplace/qe-app-registry-2pdbh/registry-server" id=6937e79e-be9a-4776-b29c-52f6c920499b name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 13:16:03.772528 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:03.772500 1563 scope.go:110] "RemoveContainer" containerID="ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a" Nov 16 13:16:03.772953 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 13:16:03.772892 1563 remote_runtime.go:597] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a\": container with ID starting with ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a not found: ID does not exist" containerID="ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a" Nov 16 13:16:03.772953 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:03.772935 1563 pod_container_deletor.go:52] "DeleteContainer returned error" containerID={Type:cri-o ID:ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a} err="failed to get container status \"ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a\": rpc error: code = NotFound desc = could not find container \"ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a\": container with ID starting with ec0fae3c68c34513605b70eede5b459199834898bd4ee8e3acc44b48c6f9683a not found: ID does not exist" Nov 16 13:16:03.852567 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:03.852322 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-lzglk\" (UniqueName: \"kubernetes.io/projected/847e27ee-d034-4a3d-89dd-da8f1ebb90b1-kube-api-access-lzglk\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 13:16:03.891932 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-1e59ed1c241c882e19d56c3c9a80bae66d74d3757c3c8748cddd0c0d8b98fb42-merged.mount: Succeeded. Nov 16 13:16:03.892000 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-847e27ee\x2dd034\x2d4a3d\x2d89dd\x2dda8f1ebb90b1-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dlzglk.mount: Succeeded. Nov 16 13:16:04.047764 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod847e27ee_d034_4a3d_89dd_da8f1ebb90b1.slice. Nov 16 13:16:04.047836 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod847e27ee_d034_4a3d_89dd_da8f1ebb90b1.slice: Consumed 682ms CPU time Nov 16 13:16:04.067891 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:04.067809 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-2pdbh] Nov 16 13:16:04.071657 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:04.071598 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-2pdbh] Nov 16 13:16:04.595184 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:16:04.595087 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=847e27ee-d034-4a3d-89dd-da8f1ebb90b1 path="/var/lib/kubelet/pods/847e27ee-d034-4a3d-89dd-da8f1ebb90b1/volumes" Nov 16 13:16:21.205111 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:16:21.205011140Z" level=info msg="Stopping pod sandbox: 7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26" id=4d9455c4-3cb8-42c5-b53e-2c97efce149f name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:16:21.205111 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:16:21.205070740Z" level=info msg="Stopped pod sandbox (already stopped): 7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26" id=4d9455c4-3cb8-42c5-b53e-2c97efce149f name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:16:21.205626 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:16:21.205411843Z" level=info msg="Removing pod sandbox: 7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26" id=7bc10b10-14a6-4a99-b306-781db280b0a8 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 13:16:21.224296 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:16:21.224249689Z" level=info msg="Removed pod sandbox: 7c067ed067cdea69c8e7068b01d9ce926cc31a3ab357e8522ce33812d8db9c26" id=7bc10b10-14a6-4a99-b306-781db280b0a8 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 13:18:20.399424 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:18:20.399275703Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=caa43557-faf6-4085-9364-56ae4d07b849 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:18:20.420086 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:18:20.399500605Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=caa43557-faf6-4085-9364-56ae4d07b849 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:18:56.978513 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.gmoAUO.mount: Succeeded. Nov 16 13:22:56.981413 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.7Ak3Nw.mount: Succeeded. Nov 16 13:23:06.978449 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.cJDkat.mount: Succeeded. Nov 16 13:23:20.404400 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:23:20.404213313Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=2769e217-5f87-4bd6-b46c-cf6aa0812635 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:23:20.459591 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:23:20.404527215Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=2769e217-5f87-4bd6-b46c-cf6aa0812635 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:28:20.409913 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:28:20.409684721Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=7811b32b-f3b9-4962-ba36-1580628af23b name=/runtime.v1.ImageService/ImageStatus Nov 16 13:28:20.427552 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:28:20.410025424Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=7811b32b-f3b9-4962-ba36-1580628af23b name=/runtime.v1.ImageService/ImageStatus Nov 16 13:29:26.980371 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.bzM9FT.mount: Succeeded. Nov 16 13:29:56.979287 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.uIoNI3.mount: Succeeded. Nov 16 13:31:27.026597 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:27.026509 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-m24cr] Nov 16 13:31:27.026597 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:27.026587 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 13:31:27.027223 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 13:31:27.026658 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="847e27ee-d034-4a3d-89dd-da8f1ebb90b1" containerName="registry-server" Nov 16 13:31:27.027223 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:27.026673 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="847e27ee-d034-4a3d-89dd-da8f1ebb90b1" containerName="registry-server" Nov 16 13:31:27.027223 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:27.026733 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="847e27ee-d034-4a3d-89dd-da8f1ebb90b1" containerName="registry-server" Nov 16 13:31:27.036409 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-podf4a98b32_1c98_4039_9bb6_6cf0748e3c43.slice. Nov 16 13:31:27.166504 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:27.166390 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fgth\" (UniqueName: \"kubernetes.io/projected/f4a98b32-1c98-4039-9bb6-6cf0748e3c43-kube-api-access-7fgth\") pod \"qe-app-registry-m24cr\" (UID: \"f4a98b32-1c98-4039-9bb6-6cf0748e3c43\") " pod="openshift-marketplace/qe-app-registry-m24cr" Nov 16 13:31:27.267830 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:27.267674 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-7fgth\" (UniqueName: \"kubernetes.io/projected/f4a98b32-1c98-4039-9bb6-6cf0748e3c43-kube-api-access-7fgth\") pod \"qe-app-registry-m24cr\" (UID: \"f4a98b32-1c98-4039-9bb6-6cf0748e3c43\") " pod="openshift-marketplace/qe-app-registry-m24cr" Nov 16 13:31:27.297033 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:27.296786 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fgth\" (UniqueName: \"kubernetes.io/projected/f4a98b32-1c98-4039-9bb6-6cf0748e3c43-kube-api-access-7fgth\") pod \"qe-app-registry-m24cr\" (UID: \"f4a98b32-1c98-4039-9bb6-6cf0748e3c43\") " pod="openshift-marketplace/qe-app-registry-m24cr" Nov 16 13:31:27.349440 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:27.349407 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-m24cr" Nov 16 13:31:27.350265 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:27.350112846Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-m24cr/POD" id=16a36577-f291-4df1-809a-91f6c68cb66e name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 13:31:27.350713 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:27.350282247Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 13:31:27.418829 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:27.418710983Z" level=info msg="Got pod network &{Name:qe-app-registry-m24cr Namespace:openshift-marketplace ID:f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4 UID:f4a98b32-1c98-4039-9bb6-6cf0748e3c43 NetNS:/var/run/netns/583a3160-a337-4c01-915b-b4291262cbc6 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 13:31:27.418829 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:27.418765583Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-m24cr to CNI network \"multus-cni-network\" (type=multus)" Nov 16 13:31:27.470164 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 13:31:27.495273 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): vetheb728b2e: link is not ready Nov 16 13:31:27.495374 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vetheb728b2e: link becomes ready Nov 16 13:31:27.495406 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 13:31:27.500674 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[95019]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 13:31:27.501081 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668605487.5010] manager: (vetheb728b2e): new Veth device (/org/freedesktop/NetworkManager/Devices/40) Nov 16 13:31:27.501191 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[95019]: Could not generate persistent MAC address for vetheb728b2e: No such file or directory Nov 16 13:31:27.501992 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668605487.5019] device (vetheb728b2e): carrier: link connected Nov 16 13:31:27.521895 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668605487.5218] manager: (vetheb728b2e): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/41) Nov 16 13:31:27.529954 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00123|bridge|INFO|bridge br0: added interface vetheb728b2e on port 18 Nov 16 13:31:27.530406 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vetheb728b2e entered promiscuous mode Nov 16 13:31:27.562180 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00124|connmgr|INFO|br0<->unix#242: 5 flow_mods in the last 0 s (5 adds) Nov 16 13:31:27.606855 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00125|connmgr|INFO|br0<->unix#245: 2 flow_mods in the last 0 s (2 deletes) Nov 16 13:31:29.269171 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T13:31:29Z [verbose] Add: openshift-marketplace:qe-app-registry-m24cr:f4a98b32-1c98-4039-9bb6-6cf0748e3c43:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/583a3160-a337-4c01-915b-b4291262cbc6"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.17/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 13:31:29.269171 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 13:31:29.246851 95002 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-m24cr", UID:"f4a98b32-1c98-4039-9bb6-6cf0748e3c43", APIVersion:"v1", ResourceVersion:"60603", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.17/23] from openshift-sdn Nov 16 13:31:29.269660 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:29.269338 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-marketplace/qe-app-registry-m24cr] Nov 16 13:31:29.269908 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:29.269731561Z" level=info msg="Got pod network &{Name:qe-app-registry-m24cr Namespace:openshift-marketplace ID:f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4 UID:f4a98b32-1c98-4039-9bb6-6cf0748e3c43 NetNS:/var/run/netns/583a3160-a337-4c01-915b-b4291262cbc6 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 13:31:29.269908 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:29.269879662Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-m24cr for CNI network multus-cni-network (type=multus)" Nov 16 13:31:29.272396 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 13:31:29.272358 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4a98b32_1c98_4039_9bb6_6cf0748e3c43.slice/crio-f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4.scope WatchSource:0}: Error finding container f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4: Status 404 returned error &{%!s(*http.body=&{0xc002f2e0a8 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 13:31:29.280816 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:29.280749366Z" level=info msg="Ran pod sandbox f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4 with infra container: openshift-marketplace/qe-app-registry-m24cr/POD" id=16a36577-f291-4df1-809a-91f6c68cb66e name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 13:31:29.281952 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:29.281905377Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=e13cb71a-f734-491c-8080-9dbd26060a99 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:31:29.282225 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:29.282183179Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=e13cb71a-f734-491c-8080-9dbd26060a99 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:31:29.282708 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:29.282684 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 13:31:29.283318 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:29.283276590Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=89e88d0d-f5e8-47be-ab18-bdc59155ff72 name=/runtime.v1.ImageService/PullImage Nov 16 13:31:29.286516 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:29.286483820Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 13:31:29.357433 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:29.357387 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-m24cr" event=&{ID:f4a98b32-1c98-4039-9bb6-6cf0748e3c43 Type:ContainerStarted Data:f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4} Nov 16 13:31:30.940800 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:30.940708257Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=89e88d0d-f5e8-47be-ab18-bdc59155ff72 name=/runtime.v1.ImageService/PullImage Nov 16 13:31:30.941993 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:30.941912469Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=2c61c17d-47b0-4af5-ad7d-39d631dbd75b name=/runtime.v1.ImageService/ImageStatus Nov 16 13:31:30.942305 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:30.942260472Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=2c61c17d-47b0-4af5-ad7d-39d631dbd75b name=/runtime.v1.ImageService/ImageStatus Nov 16 13:31:30.943398 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:30.943356683Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-m24cr/registry-server" id=33973b57-91ce-4f8d-bc01-58b354526eba name=/runtime.v1.RuntimeService/CreateContainer Nov 16 13:31:30.943508 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:30.943480784Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 13:31:31.002681 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29.scope. Nov 16 13:31:31.029032 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29. Nov 16 13:31:31.070337 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 13:31:31.137951 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:31.137884533Z" level=info msg="Created container 3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29: openshift-marketplace/qe-app-registry-m24cr/registry-server" id=33973b57-91ce-4f8d-bc01-58b354526eba name=/runtime.v1.RuntimeService/CreateContainer Nov 16 13:31:31.140848 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:31.140764561Z" level=info msg="Starting container: 3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29" id=d873b35d-5b41-4c33-9276-241381bc29af name=/runtime.v1.RuntimeService/StartContainer Nov 16 13:31:31.153323 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:31.153277880Z" level=info msg="Started container" PID=95128 containerID=3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29 description=openshift-marketplace/qe-app-registry-m24cr/registry-server id=d873b35d-5b41-4c33-9276-241381bc29af name=/runtime.v1.RuntimeService/StartContainer sandboxID=f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4 Nov 16 13:31:31.363759 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:31.363695 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-m24cr" event=&{ID:f4a98b32-1c98-4039-9bb6-6cf0748e3c43 Type:ContainerStarted Data:3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29} Nov 16 13:31:37.350352 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:37.350215 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/qe-app-registry-m24cr" Nov 16 13:31:37.350352 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:37.350285 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/qe-app-registry-m24cr" Nov 16 13:31:37.415688 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:37.415650 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/qe-app-registry-m24cr" Nov 16 13:31:38.443310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:38.443270 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/qe-app-registry-m24cr" Nov 16 13:31:38.494150 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:38.494043 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-m24cr] Nov 16 13:31:40.388857 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:40.388793 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-m24cr" podUID=f4a98b32-1c98-4039-9bb6-6cf0748e3c43 containerName="registry-server" containerID="cri-o://3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29" gracePeriod=1 Nov 16 13:31:40.389461 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:40.389171363Z" level=info msg="Stopping container: 3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29 (timeout: 1s)" id=96c1a448-fdea-460b-b8e4-bcb655562dd9 name=/runtime.v1.RuntimeService/StopContainer Nov 16 13:31:40.417103 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29.scope: Succeeded. Nov 16 13:31:40.417802 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29.scope: Consumed 673ms CPU time Nov 16 13:31:40.431433 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29.scope: Succeeded. Nov 16 13:31:40.432114 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29.scope: Consumed 36ms CPU time Nov 16 13:31:40.539819 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-ba49b791b5833f28ae2da4da3fa9645fccd0efaee952a75c06adaf62837cc2a2-merged.mount: Succeeded. Nov 16 13:31:40.891494 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:40.891395438Z" level=info msg="Stopped container 3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29: openshift-marketplace/qe-app-registry-m24cr/registry-server" id=96c1a448-fdea-460b-b8e4-bcb655562dd9 name=/runtime.v1.RuntimeService/StopContainer Nov 16 13:31:40.891949 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:40.891910342Z" level=info msg="Stopping pod sandbox: f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4" id=68391101-19e8-4fa0-af7c-20d1de95a115 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:31:40.892220 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:40.892183944Z" level=info msg="Got pod network &{Name:qe-app-registry-m24cr Namespace:openshift-marketplace ID:f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4 UID:f4a98b32-1c98-4039-9bb6-6cf0748e3c43 NetNS:/var/run/netns/583a3160-a337-4c01-915b-b4291262cbc6 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 13:31:40.892430 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:40.892397546Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-m24cr from CNI network \"multus-cni-network\" (type=multus)" Nov 16 13:31:40.960343 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00126|connmgr|INFO|br0<->unix#249: 2 flow_mods in the last 0 s (2 deletes) Nov 16 13:31:40.992203 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00127|connmgr|INFO|br0<->unix#252: 4 flow_mods in the last 0 s (4 deletes) Nov 16 13:31:41.022510 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00128|bridge|INFO|bridge br0: deleted interface vetheb728b2e on port 18 Nov 16 13:31:41.028163 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vetheb728b2e left promiscuous mode Nov 16 13:31:41.036401 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T13:31:40Z [verbose] Del: openshift-marketplace:qe-app-registry-m24cr:f4a98b32-1c98-4039-9bb6-6cf0748e3c43:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 13:31:41.043367 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-583a3160\x2da337\x2d4c01\x2d915b\x2db4291262cbc6.mount: Succeeded. Nov 16 13:31:41.051610 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-583a3160\x2da337\x2d4c01\x2d915b\x2db4291262cbc6.mount: Succeeded. Nov 16 13:31:41.192693 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:41.192529041Z" level=info msg="Stopped pod sandbox: f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4" id=68391101-19e8-4fa0-af7c-20d1de95a115 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:31:41.298538 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-4573baadcd73ccdd902ef88edeb92ba64a19cea0b1c55ee4c68d6c15fc4c2b23-merged.mount: Succeeded. Nov 16 13:31:41.298571 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-583a3160\x2da337\x2d4c01\x2d915b\x2db4291262cbc6.mount: Succeeded. Nov 16 13:31:41.298580 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4-userdata-shm.mount: Succeeded. Nov 16 13:31:41.386693 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:41.386579 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fgth\" (UniqueName: \"kubernetes.io/projected/f4a98b32-1c98-4039-9bb6-6cf0748e3c43-kube-api-access-7fgth\") pod \"f4a98b32-1c98-4039-9bb6-6cf0748e3c43\" (UID: \"f4a98b32-1c98-4039-9bb6-6cf0748e3c43\") " Nov 16 13:31:41.392405 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:41.392377 1563 generic.go:296] "Generic (PLEG): container finished" podID=f4a98b32-1c98-4039-9bb6-6cf0748e3c43 containerID="3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29" exitCode=0 Nov 16 13:31:41.392761 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:41.392431 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-m24cr" event=&{ID:f4a98b32-1c98-4039-9bb6-6cf0748e3c43 Type:ContainerDied Data:3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29} Nov 16 13:31:41.392761 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:41.392462 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-m24cr" event=&{ID:f4a98b32-1c98-4039-9bb6-6cf0748e3c43 Type:ContainerDied Data:f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4} Nov 16 13:31:41.392761 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:41.392485 1563 scope.go:110] "RemoveContainer" containerID="3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29" Nov 16 13:31:41.393427 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:41.393385011Z" level=info msg="Removing container: 3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29" id=baaea75a-94d5-49d8-9eb2-17b342e0ba87 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 13:31:41.395787 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-f4a98b32\x2d1c98\x2d4039\x2d9bb6\x2d6cf0748e3c43-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d7fgth.mount: Succeeded. Nov 16 13:31:41.400781 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:41.400740 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4a98b32-1c98-4039-9bb6-6cf0748e3c43-kube-api-access-7fgth" (OuterVolumeSpecName: "kube-api-access-7fgth") pod "f4a98b32-1c98-4039-9bb6-6cf0748e3c43" (UID: "f4a98b32-1c98-4039-9bb6-6cf0748e3c43"). InnerVolumeSpecName "kube-api-access-7fgth". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 13:31:41.419957 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:41.419846031Z" level=info msg="Removed container 3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29: openshift-marketplace/qe-app-registry-m24cr/registry-server" id=baaea75a-94d5-49d8-9eb2-17b342e0ba87 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 13:31:41.420234 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:41.420124 1563 scope.go:110] "RemoveContainer" containerID="3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29" Nov 16 13:31:41.420620 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 13:31:41.420593 1563 remote_runtime.go:597] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29\": container with ID starting with 3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29 not found: ID does not exist" containerID="3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29" Nov 16 13:31:41.420748 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:41.420643 1563 pod_container_deletor.go:52] "DeleteContainer returned error" containerID={Type:cri-o ID:3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29} err="failed to get container status \"3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29\": rpc error: code = NotFound desc = could not find container \"3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29\": container with ID starting with 3effbbdca377a05b3b3bde569d47e7c8c2db184ca2885b5cab5b9f612f7a7e29 not found: ID does not exist" Nov 16 13:31:41.487253 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:41.487033 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-7fgth\" (UniqueName: \"kubernetes.io/projected/f4a98b32-1c98-4039-9bb6-6cf0748e3c43-kube-api-access-7fgth\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 13:31:41.700890 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-podf4a98b32_1c98_4039_9bb6_6cf0748e3c43.slice. Nov 16 13:31:41.700957 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-podf4a98b32_1c98_4039_9bb6_6cf0748e3c43.slice: Consumed 710ms CPU time Nov 16 13:31:41.723358 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:41.723319 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-m24cr] Nov 16 13:31:41.731052 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:41.730992 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-m24cr] Nov 16 13:31:42.592762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:42.592698480Z" level=info msg="Stopping pod sandbox: f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4" id=316f818b-b276-4335-a905-5d51c1ed580c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:31:42.592762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:31:42.592760480Z" level=info msg="Stopped pod sandbox (already stopped): f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4" id=316f818b-b276-4335-a905-5d51c1ed580c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:31:42.594626 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:31:42.594414 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=f4a98b32-1c98-4039-9bb6-6cf0748e3c43 path="/var/lib/kubelet/pods/f4a98b32-1c98-4039-9bb6-6cf0748e3c43/volumes" Nov 16 13:32:21.283292 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:32:21.283150042Z" level=info msg="Stopping pod sandbox: f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4" id=be3c94a3-8d6e-4450-bc75-ef4a6dd3b69c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:32:21.283292 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:32:21.283206842Z" level=info msg="Stopped pod sandbox (already stopped): f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4" id=be3c94a3-8d6e-4450-bc75-ef4a6dd3b69c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:32:21.283857 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:32:21.283636046Z" level=info msg="Removing pod sandbox: f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4" id=616d943c-eccb-4455-a257-913546f17622 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 13:32:21.302149 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:32:21.302051790Z" level=info msg="Removed pod sandbox: f774f9eba0a34be5d95c8eca9cb0a2a2853958ba299469b49f7ab03ba333cca4" id=616d943c-eccb-4455-a257-913546f17622 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 13:33:20.414564 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:33:20.414392352Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=3b07b240-e8db-4fa5-b0e9-38df19eb51f9 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:33:20.434204 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:33:20.414770355Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=3b07b240-e8db-4fa5-b0e9-38df19eb51f9 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:33:46.978436 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.yxVJw0.mount: Succeeded. Nov 16 13:35:25.311938 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.8VId7Y.mount: Succeeded. Nov 16 13:36:36.979864 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.xpEbGq.mount: Succeeded. Nov 16 13:38:16.979560 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.fLdqfx.mount: Succeeded. Nov 16 13:38:20.419680 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:38:20.419447132Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=fdb10dd1-eece-41ad-9d2b-3cfb77cae63d name=/runtime.v1.ImageService/ImageStatus Nov 16 13:38:20.475036 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:38:20.419825635Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=fdb10dd1-eece-41ad-9d2b-3cfb77cae63d name=/runtime.v1.ImageService/ImageStatus Nov 16 13:39:06.978788 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.A74C1C.mount: Succeeded. Nov 16 13:39:16.980318 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.RaYehE.mount: Succeeded. Nov 16 13:43:20.425537 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:43:20.425368822Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=a1ebef75-ec16-4417-90ab-ffc1ce205ba7 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:43:20.444601 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:43:20.425656324Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=a1ebef75-ec16-4417-90ab-ffc1ce205ba7 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:47:04.693676 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:04.693633 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-x4pft] Nov 16 13:47:04.693676 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:04.693687 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 13:47:04.694461 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 13:47:04.693792 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="f4a98b32-1c98-4039-9bb6-6cf0748e3c43" containerName="registry-server" Nov 16 13:47:04.694461 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:04.693806 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4a98b32-1c98-4039-9bb6-6cf0748e3c43" containerName="registry-server" Nov 16 13:47:04.694461 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:04.693855 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="f4a98b32-1c98-4039-9bb6-6cf0748e3c43" containerName="registry-server" Nov 16 13:47:04.704421 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod1c8a06d4_10be_421f_a8eb_fdbe01f522c3.slice. Nov 16 13:47:04.870430 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:04.870320 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnhll\" (UniqueName: \"kubernetes.io/projected/1c8a06d4-10be-421f-a8eb-fdbe01f522c3-kube-api-access-xnhll\") pod \"qe-app-registry-x4pft\" (UID: \"1c8a06d4-10be-421f-a8eb-fdbe01f522c3\") " pod="openshift-marketplace/qe-app-registry-x4pft" Nov 16 13:47:04.971702 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:04.971557 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-xnhll\" (UniqueName: \"kubernetes.io/projected/1c8a06d4-10be-421f-a8eb-fdbe01f522c3-kube-api-access-xnhll\") pod \"qe-app-registry-x4pft\" (UID: \"1c8a06d4-10be-421f-a8eb-fdbe01f522c3\") " pod="openshift-marketplace/qe-app-registry-x4pft" Nov 16 13:47:04.999544 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:04.999472 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnhll\" (UniqueName: \"kubernetes.io/projected/1c8a06d4-10be-421f-a8eb-fdbe01f522c3-kube-api-access-xnhll\") pod \"qe-app-registry-x4pft\" (UID: \"1c8a06d4-10be-421f-a8eb-fdbe01f522c3\") " pod="openshift-marketplace/qe-app-registry-x4pft" Nov 16 13:47:05.014916 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:05.014874 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-x4pft" Nov 16 13:47:05.015587 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:05.015494057Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-x4pft/POD" id=8930c437-53dd-4f93-a89e-b195de50a1b1 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 13:47:05.015975 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:05.015607558Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 13:47:05.100496 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:05.100401858Z" level=info msg="Got pod network &{Name:qe-app-registry-x4pft Namespace:openshift-marketplace ID:39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef UID:1c8a06d4-10be-421f-a8eb-fdbe01f522c3 NetNS:/var/run/netns/8939b469-c9c1-423a-b800-a80a4c2f3bdc Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 13:47:05.100496 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:05.100504559Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-x4pft to CNI network \"multus-cni-network\" (type=multus)" Nov 16 13:47:05.143200 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 13:47:05.170156 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth3319ed58: link is not ready Nov 16 13:47:05.170251 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth3319ed58: link becomes ready Nov 16 13:47:05.170285 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 13:47:05.175433 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[110146]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 13:47:05.175574 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[110146]: Could not generate persistent MAC address for veth3319ed58: No such file or directory Nov 16 13:47:05.175793 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668606425.1757] device (veth3319ed58): carrier: link connected Nov 16 13:47:05.176181 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668606425.1761] manager: (veth3319ed58): new Veth device (/org/freedesktop/NetworkManager/Devices/42) Nov 16 13:47:05.185642 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668606425.1856] manager: (veth3319ed58): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/43) Nov 16 13:47:05.191378 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00129|bridge|INFO|bridge br0: added interface veth3319ed58 on port 19 Nov 16 13:47:05.192167 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth3319ed58 entered promiscuous mode Nov 16 13:47:05.223032 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00130|connmgr|INFO|br0<->unix#262: 5 flow_mods in the last 0 s (5 adds) Nov 16 13:47:05.268375 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00131|connmgr|INFO|br0<->unix#265: 2 flow_mods in the last 0 s (2 deletes) Nov 16 13:47:06.780724 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T13:47:06Z [verbose] Add: openshift-marketplace:qe-app-registry-x4pft:1c8a06d4-10be-421f-a8eb-fdbe01f522c3:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/8939b469-c9c1-423a-b800-a80a4c2f3bdc"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.18/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 13:47:06.780724 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 13:47:06.758703 110129 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-x4pft", UID:"1c8a06d4-10be-421f-a8eb-fdbe01f522c3", APIVersion:"v1", ResourceVersion:"65844", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.18/23] from openshift-sdn Nov 16 13:47:06.781453 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:06.781411430Z" level=info msg="Got pod network &{Name:qe-app-registry-x4pft Namespace:openshift-marketplace ID:39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef UID:1c8a06d4-10be-421f-a8eb-fdbe01f522c3 NetNS:/var/run/netns/8939b469-c9c1-423a-b800-a80a4c2f3bdc Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 13:47:06.781554 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:06.781505 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-marketplace/qe-app-registry-x4pft] Nov 16 13:47:06.781823 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:06.781573031Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-x4pft for CNI network multus-cni-network (type=multus)" Nov 16 13:47:06.784150 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 13:47:06.784105 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c8a06d4_10be_421f_a8eb_fdbe01f522c3.slice/crio-39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef.scope WatchSource:0}: Error finding container 39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef: Status 404 returned error &{%!s(*http.body=&{0xc003837a40 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 13:47:06.799535 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:06.799452879Z" level=info msg="Ran pod sandbox 39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef with infra container: openshift-marketplace/qe-app-registry-x4pft/POD" id=8930c437-53dd-4f93-a89e-b195de50a1b1 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 13:47:06.800594 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:06.800541788Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=6666ee28-313f-45e4-ac1c-8b569c541750 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:47:06.800832 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:06.800796290Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=6666ee28-313f-45e4-ac1c-8b569c541750 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:47:06.801358 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:06.801315 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 13:47:06.801956 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:06.801903999Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=4e0637bb-32d6-4a8b-a4d0-6edc0e8e9b1b name=/runtime.v1.ImageService/PullImage Nov 16 13:47:06.813887 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:06.813849598Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 13:47:06.933174 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:06.933093 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-x4pft" event=&{ID:1c8a06d4-10be-421f-a8eb-fdbe01f522c3 Type:ContainerStarted Data:39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef} Nov 16 13:47:08.454768 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:08.454516337Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=4e0637bb-32d6-4a8b-a4d0-6edc0e8e9b1b name=/runtime.v1.ImageService/PullImage Nov 16 13:47:08.455884 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:08.455831248Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=60fd862e-4812-4ab7-b025-a47ad00a3c97 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:47:08.456104 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:08.456067349Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=60fd862e-4812-4ab7-b025-a47ad00a3c97 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:47:08.457366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:08.457326960Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-x4pft/registry-server" id=ed82eef4-95a8-4fd5-a36c-a1f001462030 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 13:47:08.457451 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:08.457436861Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 13:47:08.515986 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6.scope. Nov 16 13:47:08.542583 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6. Nov 16 13:47:08.572565 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 13:47:08.628594 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:08.628508572Z" level=info msg="Created container 6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6: openshift-marketplace/qe-app-registry-x4pft/registry-server" id=ed82eef4-95a8-4fd5-a36c-a1f001462030 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 13:47:08.628987 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:08.628952876Z" level=info msg="Starting container: 6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6" id=4743fa05-db1a-4375-9a98-9da9f07116fc name=/runtime.v1.RuntimeService/StartContainer Nov 16 13:47:08.640373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:08.640324570Z" level=info msg="Started container" PID=110275 containerID=6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6 description=openshift-marketplace/qe-app-registry-x4pft/registry-server id=4743fa05-db1a-4375-9a98-9da9f07116fc name=/runtime.v1.RuntimeService/StartContainer sandboxID=39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef Nov 16 13:47:08.940573 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:08.940490 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-x4pft" event=&{ID:1c8a06d4-10be-421f-a8eb-fdbe01f522c3 Type:ContainerStarted Data:6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6} Nov 16 13:47:15.015868 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:15.015718 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/qe-app-registry-x4pft" Nov 16 13:47:15.015868 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:15.015792 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/qe-app-registry-x4pft" Nov 16 13:47:15.072287 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:15.072249 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/qe-app-registry-x4pft" Nov 16 13:47:16.013173 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:16.013032 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/qe-app-registry-x4pft" Nov 16 13:47:16.063543 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:16.063473 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-x4pft] Nov 16 13:47:17.967274 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:17.967236 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-x4pft" podUID=1c8a06d4-10be-421f-a8eb-fdbe01f522c3 containerName="registry-server" containerID="cri-o://6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6" gracePeriod=1 Nov 16 13:47:17.968045 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:17.967528603Z" level=info msg="Stopping container: 6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6 (timeout: 1s)" id=255aabda-9bd5-4b36-987d-a53730ea704d name=/runtime.v1.RuntimeService/StopContainer Nov 16 13:47:17.994668 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6.scope: Succeeded. Nov 16 13:47:17.995497 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6.scope: Consumed 688ms CPU time Nov 16 13:47:18.003639 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6.scope: Succeeded. Nov 16 13:47:18.004310 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6.scope: Consumed 40ms CPU time Nov 16 13:47:18.121942 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-bb849bc7791eb1b13ce76698d26d0ad7f5f6d10d18ae9ae76a223d6bbfe5b591-merged.mount: Succeeded. Nov 16 13:47:18.491880 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:18.491757742Z" level=info msg="Stopped container 6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6: openshift-marketplace/qe-app-registry-x4pft/registry-server" id=255aabda-9bd5-4b36-987d-a53730ea704d name=/runtime.v1.RuntimeService/StopContainer Nov 16 13:47:18.492577 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:18.492499448Z" level=info msg="Stopping pod sandbox: 39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef" id=1517fee0-e649-4c10-93bf-20a4a028f8ef name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:47:18.492833 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:18.492799951Z" level=info msg="Got pod network &{Name:qe-app-registry-x4pft Namespace:openshift-marketplace ID:39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef UID:1c8a06d4-10be-421f-a8eb-fdbe01f522c3 NetNS:/var/run/netns/8939b469-c9c1-423a-b800-a80a4c2f3bdc Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 13:47:18.493008 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:18.492984852Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-x4pft from CNI network \"multus-cni-network\" (type=multus)" Nov 16 13:47:18.565938 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00132|connmgr|INFO|br0<->unix#268: 2 flow_mods in the last 0 s (2 deletes) Nov 16 13:47:18.597967 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00133|connmgr|INFO|br0<->unix#271: 4 flow_mods in the last 0 s (4 deletes) Nov 16 13:47:18.627187 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00134|bridge|INFO|bridge br0: deleted interface veth3319ed58 on port 19 Nov 16 13:47:18.633160 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth3319ed58 left promiscuous mode Nov 16 13:47:18.641187 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T13:47:18Z [verbose] Del: openshift-marketplace:qe-app-registry-x4pft:1c8a06d4-10be-421f-a8eb-fdbe01f522c3:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 13:47:18.647795 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-8939b469\x2dc9c1\x2d423a\x2db800\x2da80a4c2f3bdc.mount: Succeeded. Nov 16 13:47:18.653784 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-8939b469\x2dc9c1\x2d423a\x2db800\x2da80a4c2f3bdc.mount: Succeeded. Nov 16 13:47:18.674811 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-8939b469\x2dc9c1\x2d423a\x2db800\x2da80a4c2f3bdc.mount: Succeeded. Nov 16 13:47:18.688982 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef-userdata-shm.mount: Succeeded. Nov 16 13:47:18.842330 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:18.842272077Z" level=info msg="Stopped pod sandbox: 39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef" id=1517fee0-e649-4c10-93bf-20a4a028f8ef name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:47:18.971175 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:18.971137 1563 generic.go:296] "Generic (PLEG): container finished" podID=1c8a06d4-10be-421f-a8eb-fdbe01f522c3 containerID="6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6" exitCode=0 Nov 16 13:47:18.971790 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:18.971196 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-x4pft" event=&{ID:1c8a06d4-10be-421f-a8eb-fdbe01f522c3 Type:ContainerDied Data:6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6} Nov 16 13:47:18.971790 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:18.971237 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-x4pft" event=&{ID:1c8a06d4-10be-421f-a8eb-fdbe01f522c3 Type:ContainerDied Data:39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef} Nov 16 13:47:18.971790 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:18.971261 1563 scope.go:110] "RemoveContainer" containerID="6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6" Nov 16 13:47:18.972602 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:18.972516230Z" level=info msg="Removing container: 6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6" id=b3a7b158-a64e-4ae3-b9cd-57187cf8a567 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 13:47:18.986942 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:18.986847 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnhll\" (UniqueName: \"kubernetes.io/projected/1c8a06d4-10be-421f-a8eb-fdbe01f522c3-kube-api-access-xnhll\") pod \"1c8a06d4-10be-421f-a8eb-fdbe01f522c3\" (UID: \"1c8a06d4-10be-421f-a8eb-fdbe01f522c3\") " Nov 16 13:47:18.995754 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:18.995683 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c8a06d4-10be-421f-a8eb-fdbe01f522c3-kube-api-access-xnhll" (OuterVolumeSpecName: "kube-api-access-xnhll") pod "1c8a06d4-10be-421f-a8eb-fdbe01f522c3" (UID: "1c8a06d4-10be-421f-a8eb-fdbe01f522c3"). InnerVolumeSpecName "kube-api-access-xnhll". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 13:47:19.012087 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:19.012029050Z" level=info msg="Removed container 6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6: openshift-marketplace/qe-app-registry-x4pft/registry-server" id=b3a7b158-a64e-4ae3-b9cd-57187cf8a567 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 13:47:19.012394 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:19.012366 1563 scope.go:110] "RemoveContainer" containerID="6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6" Nov 16 13:47:19.012786 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 13:47:19.012755 1563 remote_runtime.go:597] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6\": container with ID starting with 6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6 not found: ID does not exist" containerID="6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6" Nov 16 13:47:19.012891 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:19.012809 1563 pod_container_deletor.go:52] "DeleteContainer returned error" containerID={Type:cri-o ID:6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6} err="failed to get container status \"6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6\": rpc error: code = NotFound desc = could not find container \"6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6\": container with ID starting with 6cc744f533c832d7e823dd38993e2ef65be00a632c76cb3a70e98f587624fbb6 not found: ID does not exist" Nov 16 13:47:19.088336 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:19.088257 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-xnhll\" (UniqueName: \"kubernetes.io/projected/1c8a06d4-10be-421f-a8eb-fdbe01f522c3-kube-api-access-xnhll\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 13:47:19.122840 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-d35daedc017a1d22849f1a45ad8510de1a1ea0a66771c881c277b0024960e225-merged.mount: Succeeded. Nov 16 13:47:19.122871 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-1c8a06d4\x2d10be\x2d421f\x2da8eb\x2dfdbe01f522c3-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dxnhll.mount: Succeeded. Nov 16 13:47:19.279340 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod1c8a06d4_10be_421f_a8eb_fdbe01f522c3.slice. Nov 16 13:47:19.279454 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod1c8a06d4_10be_421f_a8eb_fdbe01f522c3.slice: Consumed 729ms CPU time Nov 16 13:47:19.300815 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:19.300733 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-x4pft] Nov 16 13:47:19.316142 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:19.316064 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-x4pft] Nov 16 13:47:20.595954 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 13:47:20.595876 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=1c8a06d4-10be-421f-a8eb-fdbe01f522c3 path="/var/lib/kubelet/pods/1c8a06d4-10be-421f-a8eb-fdbe01f522c3/volumes" Nov 16 13:47:21.358452 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:21.354725808Z" level=info msg="Stopping pod sandbox: 39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef" id=661f4e2c-b40f-4d20-9730-6dad2e66d629 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:47:21.358452 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:21.354776408Z" level=info msg="Stopped pod sandbox (already stopped): 39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef" id=661f4e2c-b40f-4d20-9730-6dad2e66d629 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 13:47:21.358952 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:21.358761134Z" level=info msg="Removing pod sandbox: 39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef" id=d1ebd248-eacd-4950-836c-d067f38b9786 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 13:47:21.391005 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:47:21.390888642Z" level=info msg="Removed pod sandbox: 39c64bdad60f66ac0f488fe6fac9398af40be89aa25ae2fa014667d3629732ef" id=d1ebd248-eacd-4950-836c-d067f38b9786 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 13:48:06.981470 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.jUMTY9.mount: Succeeded. Nov 16 13:48:16.978604 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.p9pclW.mount: Succeeded. Nov 16 13:48:20.429596 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:48:20.429405946Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=39a5cbba-04f1-4bf3-98c9-5c1961e0f573 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:48:20.595405 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:48:20.429692447Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=39a5cbba-04f1-4bf3-98c9-5c1961e0f573 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:53:20.434329 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:53:20.434145392Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=da519160-246b-44e4-88bb-249b6a172801 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:53:20.451371 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:53:20.434420395Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=da519160-246b-44e4-88bb-249b6a172801 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:56:16.978482 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.DuRAvC.mount: Succeeded. Nov 16 13:56:26.979303 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.tOwfGC.mount: Succeeded. Nov 16 13:58:20.438164 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:58:20.437994376Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=7ad18936-dae4-4cbd-94bb-dd073df007c5 name=/runtime.v1.ImageService/ImageStatus Nov 16 13:58:20.455563 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 13:58:20.438270278Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=7ad18936-dae4-4cbd-94bb-dd073df007c5 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:02:42.374166 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:42.373976 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-6hmgk] Nov 16 14:02:42.374166 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:42.374048 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:02:42.374166 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:02:42.374143 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="1c8a06d4-10be-421f-a8eb-fdbe01f522c3" containerName="registry-server" Nov 16 14:02:42.374166 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:42.374167 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c8a06d4-10be-421f-a8eb-fdbe01f522c3" containerName="registry-server" Nov 16 14:02:42.374819 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:42.374229 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="1c8a06d4-10be-421f-a8eb-fdbe01f522c3" containerName="registry-server" Nov 16 14:02:42.384672 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod53f2a19f_d395_4b99_8a4a_a6017f71053c.slice. Nov 16 14:02:42.547039 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:42.546908 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9wv6\" (UniqueName: \"kubernetes.io/projected/53f2a19f-d395-4b99-8a4a-a6017f71053c-kube-api-access-r9wv6\") pod \"qe-app-registry-6hmgk\" (UID: \"53f2a19f-d395-4b99-8a4a-a6017f71053c\") " pod="openshift-marketplace/qe-app-registry-6hmgk" Nov 16 14:02:42.650628 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:42.649011 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-r9wv6\" (UniqueName: \"kubernetes.io/projected/53f2a19f-d395-4b99-8a4a-a6017f71053c-kube-api-access-r9wv6\") pod \"qe-app-registry-6hmgk\" (UID: \"53f2a19f-d395-4b99-8a4a-a6017f71053c\") " pod="openshift-marketplace/qe-app-registry-6hmgk" Nov 16 14:02:42.676647 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:42.676578 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9wv6\" (UniqueName: \"kubernetes.io/projected/53f2a19f-d395-4b99-8a4a-a6017f71053c-kube-api-access-r9wv6\") pod \"qe-app-registry-6hmgk\" (UID: \"53f2a19f-d395-4b99-8a4a-a6017f71053c\") " pod="openshift-marketplace/qe-app-registry-6hmgk" Nov 16 14:02:42.705153 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:42.705089 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-6hmgk" Nov 16 14:02:42.705988 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:42.705898939Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-6hmgk/POD" id=f69e0b32-5391-43f5-ba4c-848c2bdcbf8f name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:02:42.706435 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:42.706008340Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:02:42.777507 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:42.777462189Z" level=info msg="Got pod network &{Name:qe-app-registry-6hmgk Namespace:openshift-marketplace ID:0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e UID:53f2a19f-d395-4b99-8a4a-a6017f71053c NetNS:/var/run/netns/a0ae0016-67fe-4978-b6e6-a122ad24deb7 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:02:42.777507 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:42.777505589Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-6hmgk to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:02:42.821181 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:02:42.851205 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth3afa679b: link is not ready Nov 16 14:02:42.851317 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth3afa679b: link becomes ready Nov 16 14:02:42.851351 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:02:42.857478 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[125298]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:02:42.857683 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[125298]: Could not generate persistent MAC address for veth3afa679b: No such file or directory Nov 16 14:02:42.857797 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607362.8577] manager: (veth3afa679b): new Veth device (/org/freedesktop/NetworkManager/Devices/44) Nov 16 14:02:42.858615 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607362.8585] device (veth3afa679b): carrier: link connected Nov 16 14:02:42.871689 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607362.8716] manager: (veth3afa679b): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/45) Nov 16 14:02:42.877703 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00135|bridge|INFO|bridge br0: added interface veth3afa679b on port 20 Nov 16 14:02:42.878163 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth3afa679b entered promiscuous mode Nov 16 14:02:42.909390 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00136|connmgr|INFO|br0<->unix#283: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:02:42.952565 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00137|connmgr|INFO|br0<->unix#286: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:02:44.566044 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:02:44Z [verbose] Add: openshift-marketplace:qe-app-registry-6hmgk:53f2a19f-d395-4b99-8a4a-a6017f71053c:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/a0ae0016-67fe-4978-b6e6-a122ad24deb7"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.19/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:02:44.566044 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:02:44.543634 125280 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-6hmgk", UID:"53f2a19f-d395-4b99-8a4a-a6017f71053c", APIVersion:"v1", ResourceVersion:"71182", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.19/23] from openshift-sdn Nov 16 14:02:44.566567 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:44.566244 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-marketplace/qe-app-registry-6hmgk] Nov 16 14:02:44.566825 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:44.566626331Z" level=info msg="Got pod network &{Name:qe-app-registry-6hmgk Namespace:openshift-marketplace ID:0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e UID:53f2a19f-d395-4b99-8a4a-a6017f71053c NetNS:/var/run/netns/a0ae0016-67fe-4978-b6e6-a122ad24deb7 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:02:44.566825 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:44.566760433Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-6hmgk for CNI network multus-cni-network (type=multus)" Nov 16 14:02:44.569012 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:02:44.568965 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53f2a19f_d395_4b99_8a4a_a6017f71053c.slice/crio-0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e.scope WatchSource:0}: Error finding container 0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e: Status 404 returned error &{%!s(*http.body=&{0xc003083b90 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:02:44.577500 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:44.577430829Z" level=info msg="Ran pod sandbox 0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e with infra container: openshift-marketplace/qe-app-registry-6hmgk/POD" id=f69e0b32-5391-43f5-ba4c-848c2bdcbf8f name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:02:44.578349 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:44.578312237Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=b6b8c44d-fc64-4bc3-8489-137bea24e7ec name=/runtime.v1.ImageService/ImageStatus Nov 16 14:02:44.578590 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:44.578536640Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=b6b8c44d-fc64-4bc3-8489-137bea24e7ec name=/runtime.v1.ImageService/ImageStatus Nov 16 14:02:44.579246 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:44.579114 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 14:02:44.581659 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:44.581622468Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=714d472d-49a0-4326-b8f1-83c610dbca6a name=/runtime.v1.ImageService/PullImage Nov 16 14:02:44.585184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:44.585105999Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 14:02:45.573995 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:45.573950 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-6hmgk" event=&{ID:53f2a19f-d395-4b99-8a4a-a6017f71053c Type:ContainerStarted Data:0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e} Nov 16 14:02:46.202675 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:46.202543683Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=714d472d-49a0-4326-b8f1-83c610dbca6a name=/runtime.v1.ImageService/PullImage Nov 16 14:02:46.203795 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:46.203721194Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=cd04bbd9-4a34-430f-a947-8c61a1f2a865 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:02:46.203976 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:46.203937196Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=cd04bbd9-4a34-430f-a947-8c61a1f2a865 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:02:46.205032 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:46.204959505Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-6hmgk/registry-server" id=bbb4d0ed-193d-4eb1-bec0-d3193a5b8c02 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:02:46.205120 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:46.205062406Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:02:46.269830 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b.scope. Nov 16 14:02:46.298524 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b. Nov 16 14:02:46.332978 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:02:46.396003 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:46.395861538Z" level=info msg="Created container 0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b: openshift-marketplace/qe-app-registry-6hmgk/registry-server" id=bbb4d0ed-193d-4eb1-bec0-d3193a5b8c02 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:02:46.396698 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:46.396652445Z" level=info msg="Starting container: 0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b" id=4f1d9f87-d37c-4e6f-81ce-6c0affa59136 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:02:46.408882 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:46.408824056Z" level=info msg="Started container" PID=125377 containerID=0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b description=openshift-marketplace/qe-app-registry-6hmgk/registry-server id=4f1d9f87-d37c-4e6f-81ce-6c0affa59136 name=/runtime.v1.RuntimeService/StartContainer sandboxID=0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e Nov 16 14:02:46.578322 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:46.578155 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-6hmgk" event=&{ID:53f2a19f-d395-4b99-8a4a-a6017f71053c Type:ContainerStarted Data:0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b} Nov 16 14:02:52.705723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:52.705618 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/qe-app-registry-6hmgk" Nov 16 14:02:52.705723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:52.705705 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/qe-app-registry-6hmgk" Nov 16 14:02:52.761699 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:52.761646 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/qe-app-registry-6hmgk" Nov 16 14:02:53.646247 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:53.646212 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/qe-app-registry-6hmgk" Nov 16 14:02:53.686585 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:53.686506 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-6hmgk] Nov 16 14:02:55.603927 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:55.603784 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-6hmgk" podUID=53f2a19f-d395-4b99-8a4a-a6017f71053c containerName="registry-server" containerID="cri-o://0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b" gracePeriod=1 Nov 16 14:02:55.604457 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:55.604156430Z" level=info msg="Stopping container: 0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b (timeout: 1s)" id=eafceb41-d448-403c-bc9c-2d365ad9b7d0 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:02:55.631387 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b.scope: Succeeded. Nov 16 14:02:55.632040 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b.scope: Consumed 676ms CPU time Nov 16 14:02:55.644055 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b.scope: Succeeded. Nov 16 14:02:55.644694 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b.scope: Consumed 40ms CPU time Nov 16 14:02:55.760572 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-5a400733c95d8960f06df8313a37ae08e19d305f43a9024a8a512eab6f3bf2eb-merged.mount: Succeeded. Nov 16 14:02:56.123526 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:56.123465503Z" level=info msg="Stopped container 0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b: openshift-marketplace/qe-app-registry-6hmgk/registry-server" id=eafceb41-d448-403c-bc9c-2d365ad9b7d0 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:02:56.124328 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:56.124289410Z" level=info msg="Stopping pod sandbox: 0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e" id=46624e20-20b2-4207-8770-2bc5909b7c00 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:02:56.124572 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:56.124539312Z" level=info msg="Got pod network &{Name:qe-app-registry-6hmgk Namespace:openshift-marketplace ID:0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e UID:53f2a19f-d395-4b99-8a4a-a6017f71053c NetNS:/var/run/netns/a0ae0016-67fe-4978-b6e6-a122ad24deb7 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:02:56.124774 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:56.124734013Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-6hmgk from CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:02:56.197082 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00138|connmgr|INFO|br0<->unix#289: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:02:56.228076 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00139|connmgr|INFO|br0<->unix#292: 4 flow_mods in the last 0 s (4 deletes) Nov 16 14:02:56.257182 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00140|bridge|INFO|bridge br0: deleted interface veth3afa679b on port 20 Nov 16 14:02:56.264163 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth3afa679b left promiscuous mode Nov 16 14:02:56.272322 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:02:56Z [verbose] Del: openshift-marketplace:qe-app-registry-6hmgk:53f2a19f-d395-4b99-8a4a-a6017f71053c:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 14:02:56.279327 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-a0ae0016\x2d67fe\x2d4978\x2db6e6\x2da122ad24deb7.mount: Succeeded. Nov 16 14:02:56.285987 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-a0ae0016\x2d67fe\x2d4978\x2db6e6\x2da122ad24deb7.mount: Succeeded. Nov 16 14:02:56.308386 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-a0ae0016\x2d67fe\x2d4978\x2db6e6\x2da122ad24deb7.mount: Succeeded. Nov 16 14:02:56.322272 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e-userdata-shm.mount: Succeeded. Nov 16 14:02:56.329374 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-47963a3c520a76f89ec055ad0a687e14cb8c4a24f177e2bd4b38f2c039aedb3b-merged.mount: Succeeded. Nov 16 14:02:56.538414 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:56.538158635Z" level=info msg="Stopped pod sandbox: 0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e" id=46624e20-20b2-4207-8770-2bc5909b7c00 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:02:56.606847 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:56.606765 1563 generic.go:296] "Generic (PLEG): container finished" podID=53f2a19f-d395-4b99-8a4a-a6017f71053c containerID="0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b" exitCode=0 Nov 16 14:02:56.606847 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:56.606831 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-6hmgk" event=&{ID:53f2a19f-d395-4b99-8a4a-a6017f71053c Type:ContainerDied Data:0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b} Nov 16 14:02:56.606847 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:56.606857 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-6hmgk" event=&{ID:53f2a19f-d395-4b99-8a4a-a6017f71053c Type:ContainerDied Data:0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e} Nov 16 14:02:56.607438 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:56.606879 1563 scope.go:110] "RemoveContainer" containerID="0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b" Nov 16 14:02:56.607969 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:56.607925896Z" level=info msg="Removing container: 0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b" id=1a35ab34-e37b-4602-a2c6-dff3cd5923ca name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:02:56.638221 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:02:56.638108539Z" level=info msg="Removed container 0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b: openshift-marketplace/qe-app-registry-6hmgk/registry-server" id=1a35ab34-e37b-4602-a2c6-dff3cd5923ca name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:02:56.638498 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:56.638466 1563 scope.go:110] "RemoveContainer" containerID="0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b" Nov 16 14:02:56.638934 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:02:56.638896 1563 remote_runtime.go:597] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b\": container with ID starting with 0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b not found: ID does not exist" containerID="0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b" Nov 16 14:02:56.639008 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:56.638947 1563 pod_container_deletor.go:52] "DeleteContainer returned error" containerID={Type:cri-o ID:0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b} err="failed to get container status \"0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b\": rpc error: code = NotFound desc = could not find container \"0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b\": container with ID starting with 0dcaf95ac557dcca809b188a535cf8b8f479ecaa5b995f1d27d6ea9bfd002b6b not found: ID does not exist" Nov 16 14:02:56.660704 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:56.660634 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9wv6\" (UniqueName: \"kubernetes.io/projected/53f2a19f-d395-4b99-8a4a-a6017f71053c-kube-api-access-r9wv6\") pod \"53f2a19f-d395-4b99-8a4a-a6017f71053c\" (UID: \"53f2a19f-d395-4b99-8a4a-a6017f71053c\") " Nov 16 14:02:56.669266 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-53f2a19f\x2dd395\x2d4b99\x2d8a4a\x2da6017f71053c-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dr9wv6.mount: Succeeded. Nov 16 14:02:56.678704 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:56.678665 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53f2a19f-d395-4b99-8a4a-a6017f71053c-kube-api-access-r9wv6" (OuterVolumeSpecName: "kube-api-access-r9wv6") pod "53f2a19f-d395-4b99-8a4a-a6017f71053c" (UID: "53f2a19f-d395-4b99-8a4a-a6017f71053c"). InnerVolumeSpecName "kube-api-access-r9wv6". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:02:56.761436 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:56.761386 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-r9wv6\" (UniqueName: \"kubernetes.io/projected/53f2a19f-d395-4b99-8a4a-a6017f71053c-kube-api-access-r9wv6\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:02:56.915234 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod53f2a19f_d395_4b99_8a4a_a6017f71053c.slice. Nov 16 14:02:56.915316 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod53f2a19f_d395_4b99_8a4a_a6017f71053c.slice: Consumed 716ms CPU time Nov 16 14:02:56.939050 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:56.938970 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-6hmgk] Nov 16 14:02:56.946528 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:56.946419 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-6hmgk] Nov 16 14:02:58.595713 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:02:58.595581 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=53f2a19f-d395-4b99-8a4a-a6017f71053c path="/var/lib/kubelet/pods/53f2a19f-d395-4b99-8a4a-a6017f71053c/volumes" Nov 16 14:03:20.442116 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:03:20.441954599Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=2435a8eb-3704-4fff-ac4b-5c0fe83d23aa name=/runtime.v1.ImageService/ImageStatus Nov 16 14:03:20.585113 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:03:20.442250101Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=2435a8eb-3704-4fff-ac4b-5c0fe83d23aa name=/runtime.v1.ImageService/ImageStatus Nov 16 14:03:21.453707 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:03:21.453650311Z" level=info msg="Stopping pod sandbox: 0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e" id=3e277111-348d-496e-9227-ac7832ba8924 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:03:21.453707 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:03:21.453704911Z" level=info msg="Stopped pod sandbox (already stopped): 0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e" id=3e277111-348d-496e-9227-ac7832ba8924 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:03:21.454210 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:03:21.453938713Z" level=info msg="Removing pod sandbox: 0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e" id=7095c714-4998-4132-800a-4a9f66b734d4 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:03:21.472559 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:03:21.472473561Z" level=info msg="Removed pod sandbox: 0fe9cde216789443ffd852b3221a190493c7839a8b834989557a000db980c59e" id=7095c714-4998-4132-800a-4a9f66b734d4 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:05:28.237035 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:05:28.236867 1563 logs.go:401] "Finished parsing log file, hit bytes limit" path="/var/log/pods/openshift-sdn_sdn-pgwdx_45cd7283-fcc9-4e61-b2b1-517445b7bd5a/sdn/0.log" limit=65536 Nov 16 14:05:28.427925 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:05:28.427888 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-pgwdx_45cd7283-fcc9-4e61-b2b1-517445b7bd5a/kube-rbac-proxy/0.log" Nov 16 14:05:28.625644 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:05:28.625604 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-pgwdx_45cd7283-fcc9-4e61-b2b1-517445b7bd5a/drop-icmp/0.log" Nov 16 14:06:56.981632 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.uYsdzm.mount: Succeeded. Nov 16 14:08:20.447239 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:08:20.447056163Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=4707a572-d54c-46c7-80e1-9f8adf89c063 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:08:20.465517 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:08:20.447362165Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=4707a572-d54c-46c7-80e1-9f8adf89c063 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:10:06.980590 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.TXKB7s.mount: Succeeded. Nov 16 14:10:16.981713 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.YFvh9Y.mount: Succeeded. Nov 16 14:11:06.978434 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.YU0ebf.mount: Succeeded. Nov 16 14:12:00.235677 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:00.235604 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[migration-upgrade-48669/mydep-48669-79647ff996-g97kh] Nov 16 14:12:00.235677 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:00.235660 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:12:00.236352 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:12:00.235744 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="53f2a19f-d395-4b99-8a4a-a6017f71053c" containerName="registry-server" Nov 16 14:12:00.236352 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:00.235756 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="53f2a19f-d395-4b99-8a4a-a6017f71053c" containerName="registry-server" Nov 16 14:12:00.236352 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:00.235831 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="53f2a19f-d395-4b99-8a4a-a6017f71053c" containerName="registry-server" Nov 16 14:12:00.247450 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-besteffort-pod5df14924_dd8b_4deb_a83b_53ff611d5e47.slice. Nov 16 14:12:00.379437 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:00.379354 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg] Nov 16 14:12:00.379437 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:00.379433 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:12:00.390399 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-besteffort-pod4e58fe69_e225_425e_a6ca_0a7f136dbdca.slice. Nov 16 14:12:00.411482 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:00.411386 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95fd9\" (UniqueName: \"kubernetes.io/projected/5df14924-dd8b-4deb-a83b-53ff611d5e47-kube-api-access-95fd9\") pod \"mydep-48669-79647ff996-g97kh\" (UID: \"5df14924-dd8b-4deb-a83b-53ff611d5e47\") " pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" Nov 16 14:12:00.513101 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:00.512794 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-95fd9\" (UniqueName: \"kubernetes.io/projected/5df14924-dd8b-4deb-a83b-53ff611d5e47-kube-api-access-95fd9\") pod \"mydep-48669-79647ff996-g97kh\" (UID: \"5df14924-dd8b-4deb-a83b-53ff611d5e47\") " pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" Nov 16 14:12:00.513101 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:00.512919 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnznm\" (UniqueName: \"kubernetes.io/projected/4e58fe69-e225-425e-a6ca-0a7f136dbdca-kube-api-access-wnznm\") pod \"mydep-50425-85f87ccbc6-wgxqg\" (UID: \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\") " pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" Nov 16 14:12:00.544864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:00.544774 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-95fd9\" (UniqueName: \"kubernetes.io/projected/5df14924-dd8b-4deb-a83b-53ff611d5e47-kube-api-access-95fd9\") pod \"mydep-48669-79647ff996-g97kh\" (UID: \"5df14924-dd8b-4deb-a83b-53ff611d5e47\") " pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" Nov 16 14:12:00.613391 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:00.613351 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-wnznm\" (UniqueName: \"kubernetes.io/projected/4e58fe69-e225-425e-a6ca-0a7f136dbdca-kube-api-access-wnznm\") pod \"mydep-50425-85f87ccbc6-wgxqg\" (UID: \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\") " pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" Nov 16 14:12:00.642538 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:00.642399 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnznm\" (UniqueName: \"kubernetes.io/projected/4e58fe69-e225-425e-a6ca-0a7f136dbdca-kube-api-access-wnznm\") pod \"mydep-50425-85f87ccbc6-wgxqg\" (UID: \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\") " pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" Nov 16 14:12:01.706544 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:01.706440 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4] Nov 16 14:12:01.706544 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:01.706537 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:12:01.718336 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-besteffort-podf5df478d_fc7b_4fda_b15e_1a9a05fdf611.slice. Nov 16 14:12:01.780005 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:01.779922 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx] Nov 16 14:12:01.780005 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:01.779986 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:12:01.790297 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-besteffort-pode3d0df21_083b_4e97_a997_e9a7e992eda4.slice. Nov 16 14:12:01.822816 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:01.822715 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8r4g\" (UniqueName: \"kubernetes.io/projected/f5df478d-fc7b-4fda-b15e-1a9a05fdf611-kube-api-access-s8r4g\") pod \"mydep-49678-6c64845f74-kvbp4\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" Nov 16 14:12:01.923500 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:01.923439 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-s8r4g\" (UniqueName: \"kubernetes.io/projected/f5df478d-fc7b-4fda-b15e-1a9a05fdf611-kube-api-access-s8r4g\") pod \"mydep-49678-6c64845f74-kvbp4\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" Nov 16 14:12:01.923696 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:01.923594 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k78d2\" (UniqueName: \"kubernetes.io/projected/e3d0df21-083b-4e97-a997-e9a7e992eda4-kube-api-access-k78d2\") pod \"mydep-49496-65f7bb956d-stlkx\" (UID: \"e3d0df21-083b-4e97-a997-e9a7e992eda4\") " pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" Nov 16 14:12:01.954030 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:01.953945 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8r4g\" (UniqueName: \"kubernetes.io/projected/f5df478d-fc7b-4fda-b15e-1a9a05fdf611-kube-api-access-s8r4g\") pod \"mydep-49678-6c64845f74-kvbp4\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" Nov 16 14:12:02.024667 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:02.024424 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-k78d2\" (UniqueName: \"kubernetes.io/projected/e3d0df21-083b-4e97-a997-e9a7e992eda4-kube-api-access-k78d2\") pod \"mydep-49496-65f7bb956d-stlkx\" (UID: \"e3d0df21-083b-4e97-a997-e9a7e992eda4\") " pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" Nov 16 14:12:02.052930 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:02.052858 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-k78d2\" (UniqueName: \"kubernetes.io/projected/e3d0df21-083b-4e97-a997-e9a7e992eda4-kube-api-access-k78d2\") pod \"mydep-49496-65f7bb956d-stlkx\" (UID: \"e3d0df21-083b-4e97-a997-e9a7e992eda4\") " pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" Nov 16 14:12:06.996163 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: scsi 5:0:0:0: Direct-Access Msft Virtual Disk 1.0 PQ: 0 ANSI: 5 Nov 16 14:12:07.009659 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:0: Attached scsi generic sg3 type 0 Nov 16 14:12:07.009998 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:0: [sdc] 4194304 512-byte logical blocks: (2.15 GB/2.00 GiB) Nov 16 14:12:07.019909 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:0: [sdc] 4096-byte physical blocks Nov 16 14:12:07.020306 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:0: [sdc] Write Protect is off Nov 16 14:12:07.025174 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:0: [sdc] Mode Sense: 0f 00 10 00 Nov 16 14:12:07.025475 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:0: [sdc] Write cache: disabled, read cache: enabled, supports DPO and FUA Nov 16 14:12:07.044184 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:0: [sdc] Attached SCSI disk Nov 16 14:12:16.150966 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:16.150903 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\") pod \"mydep-48669-79647ff996-g97kh\" (UID: \"5df14924-dd8b-4deb-a83b-53ff611d5e47\") " pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" Nov 16 14:12:16.157074 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:16.157012 1563 operation_generator.go:1587] "Controller attach succeeded for volume \"pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\") pod \"mydep-48669-79647ff996-g97kh\" (UID: \"5df14924-dd8b-4deb-a83b-53ff611d5e47\") device path: \"0\"" pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" Nov 16 14:12:16.252199 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:16.252065 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\") pod \"mydep-48669-79647ff996-g97kh\" (UID: \"5df14924-dd8b-4deb-a83b-53ff611d5e47\") " pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" Nov 16 14:12:16.252390 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:16.252214 1563 operation_generator.go:1123] "MapVolume.WaitForAttach entering for volume \"pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\") pod \"mydep-48669-79647ff996-g97kh\" (UID: \"5df14924-dd8b-4deb-a83b-53ff611d5e47\") DevicePath \"0\"" pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" Nov 16 14:12:16.411459 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: ata1.01: host indicates ignore ATA devices, ignored Nov 16 14:12:16.413174 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: ata1.00: host indicates ignore ATA devices, ignored Nov 16 14:12:16.856345 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: ata2.01: NODEV after polling detection Nov 16 14:12:17.492299 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:17.492183 1563 operation_generator.go:1133] "MapVolume.WaitForAttach succeeded for volume \"pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\") pod \"mydep-48669-79647ff996-g97kh\" (UID: \"5df14924-dd8b-4deb-a83b-53ff611d5e47\") DevicePath \"/dev/disk/azure/scsi1/lun0\"" pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" Nov 16 14:12:17.502975 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 14:12:17.504213 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:17.504043 1563 mount_linux.go:222] Detected OS with systemd Nov 16 14:12:17.531909 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r86f3a70c80af4aa78a0f4a3a455dc7c6.scope: Succeeded. Nov 16 14:12:17.532542 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r86f3a70c80af4aa78a0f4a3a455dc7c6.scope: Consumed 913us CPU time Nov 16 14:12:17.565148 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: loop: module loaded Nov 16 14:12:17.583817 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:17.583780 1563 operation_generator.go:1247] "MapVolume.MapPodDevice succeeded for volume \"pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\") pod \"mydep-48669-79647ff996-g97kh\" (UID: \"5df14924-dd8b-4deb-a83b-53ff611d5e47\") volumeMapPath \"/var/lib/kubelet/pods/5df14924-dd8b-4deb-a83b-53ff611d5e47/volumeDevices/kubernetes.io~azure-disk\"" pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" Nov 16 14:12:17.663457 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:17.663345 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" Nov 16 14:12:17.664182 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:17.664028990Z" level=info msg="Running pod sandbox: migration-upgrade-48669/mydep-48669-79647ff996-g97kh/POD" id=b6f8309d-bf7f-4542-8891-735e30f0c953 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:12:17.664578 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:17.664203792Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:12:17.720931 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:17.720852303Z" level=info msg="Got pod network &{Name:mydep-48669-79647ff996-g97kh Namespace:migration-upgrade-48669 ID:11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30 UID:5df14924-dd8b-4deb-a83b-53ff611d5e47 NetNS:/var/run/netns/5138656a-9b5c-411e-8cef-d25e11462c24 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:12:17.720931 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:17.720892204Z" level=info msg="Adding pod migration-upgrade-48669_mydep-48669-79647ff996-g97kh to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:12:17.767202 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:12:17.787889 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[134713]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:12:17.794979 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): vetha1373bc6: link is not ready Nov 16 14:12:17.795069 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vetha1373bc6: link becomes ready Nov 16 14:12:17.795100 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:12:17.801103 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[134713]: Could not generate persistent MAC address for vetha1373bc6: No such file or directory Nov 16 14:12:17.801448 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607937.8014] manager: (vetha1373bc6): new Veth device (/org/freedesktop/NetworkManager/Devices/46) Nov 16 14:12:17.802313 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607937.8022] device (vetha1373bc6): carrier: link connected Nov 16 14:12:17.815441 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607937.8154] manager: (vetha1373bc6): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/47) Nov 16 14:12:17.821155 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vetha1373bc6 entered promiscuous mode Nov 16 14:12:17.821210 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00141|bridge|INFO|bridge br0: added interface vetha1373bc6 on port 21 Nov 16 14:12:17.851909 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00142|connmgr|INFO|br0<->unix#300: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:12:17.902736 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00143|connmgr|INFO|br0<->unix#304: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:12:17.904887 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00144|connmgr|INFO|br0<->unix#306: 6 flow_mods in the last 0 s (2 adds, 4 deletes) Nov 16 14:12:19.312855 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:12:19Z [verbose] Add: migration-upgrade-48669:mydep-48669-79647ff996-g97kh:5df14924-dd8b-4deb-a83b-53ff611d5e47:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/5138656a-9b5c-411e-8cef-d25e11462c24"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.20/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:12:19.312855 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:12:19.291760 134696 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"migration-upgrade-48669", Name:"mydep-48669-79647ff996-g97kh", UID:"5df14924-dd8b-4deb-a83b-53ff611d5e47", APIVersion:"v1", ResourceVersion:"75212", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.20/23] from openshift-sdn Nov 16 14:12:19.313435 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:19.313300 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[migration-upgrade-48669/mydep-48669-79647ff996-g97kh] Nov 16 14:12:19.313733 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:19.313459314Z" level=info msg="Got pod network &{Name:mydep-48669-79647ff996-g97kh Namespace:migration-upgrade-48669 ID:11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30 UID:5df14924-dd8b-4deb-a83b-53ff611d5e47 NetNS:/var/run/netns/5138656a-9b5c-411e-8cef-d25e11462c24 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:12:19.313733 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:19.313620815Z" level=info msg="Checking pod migration-upgrade-48669_mydep-48669-79647ff996-g97kh for CNI network multus-cni-network (type=multus)" Nov 16 14:12:19.316149 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:12:19.316074 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5df14924_dd8b_4deb_a83b_53ff611d5e47.slice/crio-11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30.scope WatchSource:0}: Error finding container 11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30: Status 404 returned error &{%!s(*http.body=&{0xc00275b320 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:12:19.325555 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:19.325488895Z" level=info msg="Ran pod sandbox 11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30 with infra container: migration-upgrade-48669/mydep-48669-79647ff996-g97kh/POD" id=b6f8309d-bf7f-4542-8891-735e30f0c953 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:12:19.326642 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:19.326601402Z" level=info msg="Checking image status: quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339" id=0a6e5369-659c-47ec-98bf-d0af8f86d44e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:19.326900 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:19.326872204Z" level=info msg="Image quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339 not found" id=0a6e5369-659c-47ec-98bf-d0af8f86d44e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:19.327440 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:19.327395 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 14:12:19.328148 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:19.328092213Z" level=info msg="Pulling image: quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339" id=bcc103af-a2a2-43e9-894e-9e4ec411a9ef name=/runtime.v1.ImageService/PullImage Nov 16 14:12:19.331357 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:19.331322134Z" level=info msg="Trying to access \"quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339\"" Nov 16 14:12:20.145199 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:20.145141 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" event=&{ID:5df14924-dd8b-4deb-a83b-53ff611d5e47 Type:ContainerStarted Data:11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30} Nov 16 14:12:20.934566 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:20.934517967Z" level=info msg="Trying to access \"quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339\"" Nov 16 14:12:21.726183 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: scsi 5:0:0:1: Direct-Access Msft Virtual Disk 1.0 PQ: 0 ANSI: 5 Nov 16 14:12:21.739273 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:1: Attached scsi generic sg4 type 0 Nov 16 14:12:21.739598 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:1: [sdd] 4194304 512-byte logical blocks: (2.15 GB/2.00 GiB) Nov 16 14:12:21.749232 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:1: [sdd] 4096-byte physical blocks Nov 16 14:12:21.749562 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:1: [sdd] Write Protect is off Nov 16 14:12:21.754225 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:1: [sdd] Mode Sense: 0f 00 10 00 Nov 16 14:12:21.754576 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:1: [sdd] Write cache: disabled, read cache: enabled, supports DPO and FUA Nov 16 14:12:21.772191 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:1: [sdd] Attached SCSI disk Nov 16 14:12:26.745803 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:26.745753 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-4618b52d-ff9e-4972-becd-c0da785f9912\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-4618b52d-ff9e-4972-becd-c0da785f9912\") pod \"mydep-50425-85f87ccbc6-wgxqg\" (UID: \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\") " pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" Nov 16 14:12:26.753625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:26.753594 1563 operation_generator.go:1587] "Controller attach succeeded for volume \"pvc-4618b52d-ff9e-4972-becd-c0da785f9912\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-4618b52d-ff9e-4972-becd-c0da785f9912\") pod \"mydep-50425-85f87ccbc6-wgxqg\" (UID: \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\") device path: \"1\"" pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" Nov 16 14:12:26.846780 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:26.846682 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"pvc-4618b52d-ff9e-4972-becd-c0da785f9912\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-4618b52d-ff9e-4972-becd-c0da785f9912\") pod \"mydep-50425-85f87ccbc6-wgxqg\" (UID: \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\") " pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" Nov 16 14:12:26.846977 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:26.846892 1563 operation_generator.go:631] "MountVolume.WaitForAttach entering for volume \"pvc-4618b52d-ff9e-4972-becd-c0da785f9912\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-4618b52d-ff9e-4972-becd-c0da785f9912\") pod \"mydep-50425-85f87ccbc6-wgxqg\" (UID: \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\") DevicePath \"1\"" pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" Nov 16 14:12:27.012245 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: ata1.01: host indicates ignore ATA devices, ignored Nov 16 14:12:27.013244 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: ata1.00: host indicates ignore ATA devices, ignored Nov 16 14:12:27.441851 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: ata2.01: NODEV after polling detection Nov 16 14:12:27.539856 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:27.539810 1563 operation_generator.go:641] "MountVolume.WaitForAttach succeeded for volume \"pvc-4618b52d-ff9e-4972-becd-c0da785f9912\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-4618b52d-ff9e-4972-becd-c0da785f9912\") pod \"mydep-50425-85f87ccbc6-wgxqg\" (UID: \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\") DevicePath \"/dev/disk/azure/scsi1/lun1\"" pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" Nov 16 14:12:27.584022 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:27.583966 1563 mount_linux.go:449] Disk "/dev/disk/azure/scsi1/lun1" appears to be unformatted, attempting to format as type: "ext4" with options: [-F -m0 /dev/disk/azure/scsi1/lun1] Nov 16 14:12:28.092416 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:28.092359 1563 mount_linux.go:459] Disk successfully formatted (mkfs): ext4 - /dev/disk/azure/scsi1/lun1 /var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/m3413458433 Nov 16 14:12:28.102504 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes transient mount for /var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/m3413458433. Nov 16 14:12:28.116920 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:28.116882 1563 operation_generator.go:674] "MountVolume.MountDevice succeeded for volume \"pvc-4618b52d-ff9e-4972-becd-c0da785f9912\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-4618b52d-ff9e-4972-becd-c0da785f9912\") pod \"mydep-50425-85f87ccbc6-wgxqg\" (UID: \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/m3413458433\"" pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" Nov 16 14:12:28.117177 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: EXT4-fs (sdd): mounted filesystem with ordered data mode. Opts: (null) Nov 16 14:12:28.125506 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:28.125440 1563 azure_mounter.go:170] azureDisk - successfully mounted disk ropatil1611az-gs6zj-dynamic-pvc-4618b52d-ff9e-4972-becd-c0da785f9912 on /var/lib/kubelet/pods/4e58fe69-e225-425e-a6ca-0a7f136dbdca/volumes/kubernetes.io~azure-disk/pvc-4618b52d-ff9e-4972-becd-c0da785f9912 Nov 16 14:12:28.125590 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:28.125534 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"pvc-4618b52d-ff9e-4972-becd-c0da785f9912\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-4618b52d-ff9e-4972-becd-c0da785f9912\") pod \"mydep-50425-85f87ccbc6-wgxqg\" (UID: \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\") " pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" Nov 16 14:12:28.137472 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rb32dc305c5eb4b9c841750bc94f5dcd2.scope: Succeeded. Nov 16 14:12:28.138086 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rb32dc305c5eb4b9c841750bc94f5dcd2.scope: Consumed 10ms CPU time Nov 16 14:12:28.313182 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:28.313033 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" Nov 16 14:12:28.313889 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:28.313760694Z" level=info msg="Running pod sandbox: migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg/POD" id=9a526d80-ab99-424f-bdaa-ff02f84fac5f name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:12:28.313889 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:28.313866195Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:12:29.706942 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:29.706877917Z" level=info msg="Got pod network &{Name:mydep-50425-85f87ccbc6-wgxqg Namespace:migration-upgrade-50425 ID:5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78 UID:4e58fe69-e225-425e-a6ca-0a7f136dbdca NetNS:/var/run/netns/f236704e-396e-4bc2-b833-20144735ffde Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:12:29.706942 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:29.706935617Z" level=info msg="Adding pod migration-upgrade-50425_mydep-50425-85f87ccbc6-wgxqg to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:12:29.751162 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:12:29.766215 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607949.7661] manager: (vethf2aca57a): new Veth device (/org/freedesktop/NetworkManager/Devices/48) Nov 16 14:12:29.767212 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607949.7671] device (vethf2aca57a): carrier: link connected Nov 16 14:12:29.773556 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[135029]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:12:29.773715 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[135029]: Could not generate persistent MAC address for vethf2aca57a: No such file or directory Nov 16 14:12:29.783320 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): vethf2aca57a: link is not ready Nov 16 14:12:29.783416 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethf2aca57a: link becomes ready Nov 16 14:12:29.783449 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:12:29.801733 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607949.8017] manager: (vethf2aca57a): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/49) Nov 16 14:12:29.808032 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00145|bridge|INFO|bridge br0: added interface vethf2aca57a on port 22 Nov 16 14:12:29.808385 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethf2aca57a entered promiscuous mode Nov 16 14:12:29.841587 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00146|connmgr|INFO|br0<->unix#309: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:12:29.893350 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00147|connmgr|INFO|br0<->unix#313: 6 flow_mods in the last 0 s (2 adds, 4 deletes) Nov 16 14:12:29.894911 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00148|connmgr|INFO|br0<->unix#315: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:12:29.911338 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:29.911285102Z" level=info msg="Pulled image: quay.io/openshifttest/hello-openshift@sha256:2b1a20c5faceb72a15b20ae0b5b83e3cf87fd76ee5375941071a0efbc196ea45" id=bcc103af-a2a2-43e9-894e-9e4ec411a9ef name=/runtime.v1.ImageService/PullImage Nov 16 14:12:29.912280 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:29.912235209Z" level=info msg="Checking image status: quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339" id=3d7acf2b-9a85-4cc7-b7aa-af0070fd8cbe name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:29.913690 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:29.913609719Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:e4f3998ce785a7eb20d958a205eebb88b77e5de16bc616e5f1d35b6be80021a9,RepoTags:[],RepoDigests:[quay.io/openshifttest/hello-openshift@sha256:2b1a20c5faceb72a15b20ae0b5b83e3cf87fd76ee5375941071a0efbc196ea45 quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339],Size_:266824987,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=3d7acf2b-9a85-4cc7-b7aa-af0070fd8cbe name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:29.914707 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:29.914665927Z" level=info msg="Creating container: migration-upgrade-48669/mydep-48669-79647ff996-g97kh/mydep-48669" id=2272252c-9521-4ed7-ae72-3e10393b99ce name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:12:29.914803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:29.914775228Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:12:29.970924 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a.scope. Nov 16 14:12:29.998526 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a. Nov 16 14:12:30.033217 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:12:30.101099 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:30.101004581Z" level=info msg="Created container 2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a: migration-upgrade-48669/mydep-48669-79647ff996-g97kh/mydep-48669" id=2272252c-9521-4ed7-ae72-3e10393b99ce name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:12:30.101790 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:30.101742586Z" level=info msg="Starting container: 2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a" id=ba5d8ed9-b288-4af4-89fb-db7a6e695aa6 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:12:30.112860 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:30.112817467Z" level=info msg="Started container" PID=135065 containerID=2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a description=migration-upgrade-48669/mydep-48669-79647ff996-g97kh/mydep-48669 id=ba5d8ed9-b288-4af4-89fb-db7a6e695aa6 name=/runtime.v1.RuntimeService/StartContainer sandboxID=11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30 Nov 16 14:12:30.178590 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:30.178536 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" event=&{ID:5df14924-dd8b-4deb-a83b-53ff611d5e47 Type:ContainerStarted Data:2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a} Nov 16 14:12:31.711378 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:12:31Z [verbose] Add: migration-upgrade-50425:mydep-50425-85f87ccbc6-wgxqg:4e58fe69-e225-425e-a6ca-0a7f136dbdca:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/f236704e-396e-4bc2-b833-20144735ffde"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.21/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:12:31.711378 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:12:31.687962 135013 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"migration-upgrade-50425", Name:"mydep-50425-85f87ccbc6-wgxqg", UID:"4e58fe69-e225-425e-a6ca-0a7f136dbdca", APIVersion:"v1", ResourceVersion:"75220", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.21/23] from openshift-sdn Nov 16 14:12:31.712091 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:31.712021487Z" level=info msg="Got pod network &{Name:mydep-50425-85f87ccbc6-wgxqg Namespace:migration-upgrade-50425 ID:5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78 UID:4e58fe69-e225-425e-a6ca-0a7f136dbdca NetNS:/var/run/netns/f236704e-396e-4bc2-b833-20144735ffde Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:12:31.712197 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:31.711955 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg] Nov 16 14:12:31.712552 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:31.712226789Z" level=info msg="Checking pod migration-upgrade-50425_mydep-50425-85f87ccbc6-wgxqg for CNI network multus-cni-network (type=multus)" Nov 16 14:12:31.714627 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:12:31.714548 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e58fe69_e225_425e_a6ca_0a7f136dbdca.slice/crio-5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78.scope WatchSource:0}: Error finding container 5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78: Status 404 returned error &{%!s(*http.body=&{0xc001bb5650 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:12:31.726843 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:31.726797895Z" level=info msg="Ran pod sandbox 5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78 with infra container: migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg/POD" id=9a526d80-ab99-424f-bdaa-ff02f84fac5f name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:12:31.728010 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:31.727973803Z" level=info msg="Checking image status: quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339" id=095539c8-acae-4d9d-98eb-1c508569bbb8 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:31.728246 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:31.728213205Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:e4f3998ce785a7eb20d958a205eebb88b77e5de16bc616e5f1d35b6be80021a9,RepoTags:[],RepoDigests:[quay.io/openshifttest/hello-openshift@sha256:2b1a20c5faceb72a15b20ae0b5b83e3cf87fd76ee5375941071a0efbc196ea45 quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339],Size_:266824987,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=095539c8-acae-4d9d-98eb-1c508569bbb8 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:31.728955 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:31.728873810Z" level=info msg="Checking image status: quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339" id=d3401561-206a-4ab2-b2a5-3fb09f52e450 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:31.729105 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:31.729072111Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:e4f3998ce785a7eb20d958a205eebb88b77e5de16bc616e5f1d35b6be80021a9,RepoTags:[],RepoDigests:[quay.io/openshifttest/hello-openshift@sha256:2b1a20c5faceb72a15b20ae0b5b83e3cf87fd76ee5375941071a0efbc196ea45 quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339],Size_:266824987,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=d3401561-206a-4ab2-b2a5-3fb09f52e450 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:31.730665 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:31.730620022Z" level=info msg="Creating container: migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg/mydep-50425" id=025e2ab2-8e6e-451c-8824-2472db1dbca2 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:12:31.730752 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:31.730726423Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:12:31.794088 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858.scope. Nov 16 14:12:31.820428 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858. Nov 16 14:12:31.861061 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:12:31.937223 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:31.937167523Z" level=info msg="Created container 7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858: migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg/mydep-50425" id=025e2ab2-8e6e-451c-8824-2472db1dbca2 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:12:31.937821 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:31.937786728Z" level=info msg="Starting container: 7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858" id=e86e33ce-ee6e-4c4e-8092-c79175f4feb1 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:12:31.974173 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:31.973964291Z" level=info msg="Started container" PID=135167 containerID=7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858 description=migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg/mydep-50425 id=e86e33ce-ee6e-4c4e-8092-c79175f4feb1 name=/runtime.v1.RuntimeService/StartContainer sandboxID=5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78 Nov 16 14:12:32.186538 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:32.186491 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" event=&{ID:4e58fe69-e225-425e-a6ca-0a7f136dbdca Type:ContainerStarted Data:7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858} Nov 16 14:12:32.186538 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:32.186554 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" event=&{ID:4e58fe69-e225-425e-a6ca-0a7f136dbdca Type:ContainerStarted Data:5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78} Nov 16 14:12:32.349203 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: scsi 5:0:0:2: Direct-Access Msft Virtual Disk 1.0 PQ: 0 ANSI: 5 Nov 16 14:12:32.362226 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:2: Attached scsi generic sg5 type 0 Nov 16 14:12:32.362543 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:2: [sde] 4194304 512-byte logical blocks: (2.15 GB/2.00 GiB) Nov 16 14:12:32.374294 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:2: [sde] 4096-byte physical blocks Nov 16 14:12:32.374594 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:2: [sde] Write Protect is off Nov 16 14:12:32.379981 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:2: [sde] Mode Sense: 0f 00 10 00 Nov 16 14:12:32.380348 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:2: [sde] Write cache: disabled, read cache: enabled, supports DPO and FUA Nov 16 14:12:32.398214 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:2: [sde] Attached SCSI disk Nov 16 14:12:37.331725 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:37.331663 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"mydep-49678-6c64845f74-kvbp4\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" Nov 16 14:12:37.339474 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:37.339391 1563 operation_generator.go:1587] "Controller attach succeeded for volume \"pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"mydep-49678-6c64845f74-kvbp4\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") device path: \"2\"" pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" Nov 16 14:12:37.433262 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:37.433212 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"mydep-49678-6c64845f74-kvbp4\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" Nov 16 14:12:37.433488 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:37.433364 1563 operation_generator.go:1123] "MapVolume.WaitForAttach entering for volume \"pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"mydep-49678-6c64845f74-kvbp4\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") DevicePath \"2\"" pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" Nov 16 14:12:37.596287 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: ata1.01: host indicates ignore ATA devices, ignored Nov 16 14:12:37.598217 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: ata1.00: host indicates ignore ATA devices, ignored Nov 16 14:12:38.052357 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: ata2.01: NODEV after polling detection Nov 16 14:12:38.150578 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:38.150491 1563 operation_generator.go:1133] "MapVolume.WaitForAttach succeeded for volume \"pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"mydep-49678-6c64845f74-kvbp4\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") DevicePath \"/dev/disk/azure/scsi1/lun2\"" pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" Nov 16 14:12:38.161063 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 14:12:38.162114 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:38.162048 1563 mount_linux.go:222] Detected OS with systemd Nov 16 14:12:38.195588 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r9468154bbe704740bf1fc208bc44c24c.scope: Succeeded. Nov 16 14:12:38.196267 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r9468154bbe704740bf1fc208bc44c24c.scope: Consumed 836us CPU time Nov 16 14:12:38.200838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:38.200760 1563 operation_generator.go:1247] "MapVolume.MapPodDevice succeeded for volume \"pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"mydep-49678-6c64845f74-kvbp4\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") volumeMapPath \"/var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk\"" pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" Nov 16 14:12:38.343399 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:38.343360 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" Nov 16 14:12:38.344209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:38.344111446Z" level=info msg="Running pod sandbox: migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4/POD" id=d57ceea8-7da8-4435-841f-a294dabbdc12 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:12:38.344602 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:38.344228047Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:12:38.400720 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:38.400535778Z" level=info msg="Got pod network &{Name:mydep-49678-6c64845f74-kvbp4 Namespace:migration-upgrade-49678 ID:94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b UID:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 NetNS:/var/run/netns/e11d37ec-82ef-4841-85ed-8c2da018d6b5 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:12:38.400720 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:38.400573978Z" level=info msg="Adding pod migration-upgrade-49678_mydep-49678-6c64845f74-kvbp4 to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:12:38.444163 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:12:38.460443 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607958.4603] manager: (vethbe42f6a4): new Veth device (/org/freedesktop/NetworkManager/Devices/50) Nov 16 14:12:38.461518 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607958.4615] device (vethbe42f6a4): carrier: link connected Nov 16 14:12:38.475095 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): vethbe42f6a4: link is not ready Nov 16 14:12:38.475189 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethbe42f6a4: link becomes ready Nov 16 14:12:38.475221 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:12:38.482086 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[135358]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:12:38.482311 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[135358]: Could not generate persistent MAC address for vethbe42f6a4: No such file or directory Nov 16 14:12:38.496618 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607958.4965] manager: (vethbe42f6a4): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/51) Nov 16 14:12:38.502161 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethbe42f6a4 entered promiscuous mode Nov 16 14:12:38.502298 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00149|bridge|INFO|bridge br0: added interface vethbe42f6a4 on port 23 Nov 16 14:12:38.534175 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00150|connmgr|INFO|br0<->unix#318: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:12:38.583460 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00151|connmgr|INFO|br0<->unix#322: 6 flow_mods in the last 0 s (2 adds, 4 deletes) Nov 16 14:12:38.585065 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00152|connmgr|INFO|br0<->unix#324: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:12:39.642989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:39.642948 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4] Nov 16 14:12:39.644474 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:12:39Z [verbose] Add: migration-upgrade-49678:mydep-49678-6c64845f74-kvbp4:f5df478d-fc7b-4fda-b15e-1a9a05fdf611:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/e11d37ec-82ef-4841-85ed-8c2da018d6b5"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.22/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:12:39.644474 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:12:39.622327 135342 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"migration-upgrade-49678", Name:"mydep-49678-6c64845f74-kvbp4", UID:"f5df478d-fc7b-4fda-b15e-1a9a05fdf611", APIVersion:"v1", ResourceVersion:"75241", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.22/23] from openshift-sdn Nov 16 14:12:39.645056 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:39.645014795Z" level=info msg="Got pod network &{Name:mydep-49678-6c64845f74-kvbp4 Namespace:migration-upgrade-49678 ID:94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b UID:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 NetNS:/var/run/netns/e11d37ec-82ef-4841-85ed-8c2da018d6b5 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:12:39.645217 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:39.645181297Z" level=info msg="Checking pod migration-upgrade-49678_mydep-49678-6c64845f74-kvbp4 for CNI network multus-cni-network (type=multus)" Nov 16 14:12:39.647609 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:12:39.647574 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5df478d_fc7b_4fda_b15e_1a9a05fdf611.slice/crio-94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b.scope WatchSource:0}: Error finding container 94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b: Status 404 returned error &{%!s(*http.body=&{0xc0000350b0 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:12:39.659625 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:39.659530906Z" level=info msg="Ran pod sandbox 94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b with infra container: migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4/POD" id=d57ceea8-7da8-4435-841f-a294dabbdc12 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:12:39.660730 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:39.660678315Z" level=info msg="Checking image status: quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339" id=8f07888a-c219-4df2-a4e4-27640807935d name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:39.660964 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:39.660915417Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:e4f3998ce785a7eb20d958a205eebb88b77e5de16bc616e5f1d35b6be80021a9,RepoTags:[],RepoDigests:[quay.io/openshifttest/hello-openshift@sha256:2b1a20c5faceb72a15b20ae0b5b83e3cf87fd76ee5375941071a0efbc196ea45 quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339],Size_:266824987,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=8f07888a-c219-4df2-a4e4-27640807935d name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:39.662044 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:39.661789124Z" level=info msg="Checking image status: quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339" id=0c363d06-299c-4b78-921c-a3e83701c051 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:39.662257 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:39.662213727Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:e4f3998ce785a7eb20d958a205eebb88b77e5de16bc616e5f1d35b6be80021a9,RepoTags:[],RepoDigests:[quay.io/openshifttest/hello-openshift@sha256:2b1a20c5faceb72a15b20ae0b5b83e3cf87fd76ee5375941071a0efbc196ea45 quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339],Size_:266824987,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=0c363d06-299c-4b78-921c-a3e83701c051 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:39.663147 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:39.663047033Z" level=info msg="Creating container: migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4/mydep-49678" id=879f6b05-99aa-4cb1-8cd5-796e21d66ab7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:12:39.663392 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:39.663359836Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:12:39.726106 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d.scope. Nov 16 14:12:39.753186 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d. Nov 16 14:12:39.787110 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:12:39.863572 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:39.863488266Z" level=info msg="Created container 7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d: migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4/mydep-49678" id=879f6b05-99aa-4cb1-8cd5-796e21d66ab7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:12:39.864199 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:39.864143671Z" level=info msg="Starting container: 7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d" id=14f45582-28c4-4249-88ce-fe4a3d8bda9d name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:12:39.874879 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:39.874837853Z" level=info msg="Started container" PID=135455 containerID=7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d description=migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4/mydep-49678 id=14f45582-28c4-4249-88ce-fe4a3d8bda9d name=/runtime.v1.RuntimeService/StartContainer sandboxID=94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b Nov 16 14:12:40.211332 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:40.211279 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" event=&{ID:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Type:ContainerStarted Data:7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d} Nov 16 14:12:40.211332 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:40.211341 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" event=&{ID:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Type:ContainerStarted Data:94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b} Nov 16 14:12:42.930214 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: scsi 5:0:0:3: Direct-Access Msft Virtual Disk 1.0 PQ: 0 ANSI: 5 Nov 16 14:12:42.941198 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:3: Attached scsi generic sg6 type 0 Nov 16 14:12:42.941523 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:3: [sdf] 4194304 512-byte logical blocks: (2.15 GB/2.00 GiB) Nov 16 14:12:42.954839 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:3: [sdf] 4096-byte physical blocks Nov 16 14:12:42.955219 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:3: [sdf] Write Protect is off Nov 16 14:12:42.969939 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:3: [sdf] Mode Sense: 0f 00 10 00 Nov 16 14:12:42.970318 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:3: [sdf] Write cache: disabled, read cache: enabled, supports DPO and FUA Nov 16 14:12:42.980213 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:3: [sdf] Attached SCSI disk Nov 16 14:12:47.821548 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:47.821404 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-e53f0905-53ec-4da6-829f-022ba78b9629\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-e53f0905-53ec-4da6-829f-022ba78b9629\") pod \"mydep-49496-65f7bb956d-stlkx\" (UID: \"e3d0df21-083b-4e97-a997-e9a7e992eda4\") " pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" Nov 16 14:12:47.830223 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:47.830191 1563 operation_generator.go:1587] "Controller attach succeeded for volume \"pvc-e53f0905-53ec-4da6-829f-022ba78b9629\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-e53f0905-53ec-4da6-829f-022ba78b9629\") pod \"mydep-49496-65f7bb956d-stlkx\" (UID: \"e3d0df21-083b-4e97-a997-e9a7e992eda4\") device path: \"3\"" pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" Nov 16 14:12:47.922776 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:47.922635 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"pvc-e53f0905-53ec-4da6-829f-022ba78b9629\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-e53f0905-53ec-4da6-829f-022ba78b9629\") pod \"mydep-49496-65f7bb956d-stlkx\" (UID: \"e3d0df21-083b-4e97-a997-e9a7e992eda4\") " pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" Nov 16 14:12:47.922962 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:47.922863 1563 operation_generator.go:631] "MountVolume.WaitForAttach entering for volume \"pvc-e53f0905-53ec-4da6-829f-022ba78b9629\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-e53f0905-53ec-4da6-829f-022ba78b9629\") pod \"mydep-49496-65f7bb956d-stlkx\" (UID: \"e3d0df21-083b-4e97-a997-e9a7e992eda4\") DevicePath \"3\"" pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" Nov 16 14:12:48.083554 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: ata1.01: host indicates ignore ATA devices, ignored Nov 16 14:12:48.085391 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: ata1.00: host indicates ignore ATA devices, ignored Nov 16 14:12:48.542673 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: ata2.01: NODEV after polling detection Nov 16 14:12:48.650823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:48.650741 1563 operation_generator.go:641] "MountVolume.WaitForAttach succeeded for volume \"pvc-e53f0905-53ec-4da6-829f-022ba78b9629\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-e53f0905-53ec-4da6-829f-022ba78b9629\") pod \"mydep-49496-65f7bb956d-stlkx\" (UID: \"e3d0df21-083b-4e97-a997-e9a7e992eda4\") DevicePath \"/dev/disk/azure/scsi1/lun3\"" pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" Nov 16 14:12:48.657295 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:48.657258 1563 mount_linux.go:449] Disk "/dev/disk/azure/scsi1/lun3" appears to be unformatted, attempting to format as type: "ext4" with options: [-F -m0 /dev/disk/azure/scsi1/lun3] Nov 16 14:12:49.083843 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:49.083659 1563 mount_linux.go:459] Disk successfully formatted (mkfs): ext4 - /dev/disk/azure/scsi1/lun3 /var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/m2408838648 Nov 16 14:12:49.093977 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes transient mount for /var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/m2408838648. Nov 16 14:12:49.120780 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: [EXT4 FS bs=4096, gc=16, bpg=32768, ipg=8192, mo=e882c818, mo2=0002] Nov 16 14:12:49.120897 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: System zones: 0-1, 257-8480, 32768-32769, 98304-98305, 163840-163841, 229376-229377, 262144-278527, 294912-294913 Nov 16 14:12:49.120928 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: EXT4-fs (sdf): mounted filesystem with ordered data mode. Opts: debug,discard Nov 16 14:12:49.129772 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:49.129738 1563 operation_generator.go:674] "MountVolume.MountDevice succeeded for volume \"pvc-e53f0905-53ec-4da6-829f-022ba78b9629\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-e53f0905-53ec-4da6-829f-022ba78b9629\") pod \"mydep-49496-65f7bb956d-stlkx\" (UID: \"e3d0df21-083b-4e97-a997-e9a7e992eda4\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/m2408838648\"" pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" Nov 16 14:12:49.140167 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r25832f6250c748258a9289212b5024e3.scope: Succeeded. Nov 16 14:12:49.140719 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r25832f6250c748258a9289212b5024e3.scope: Consumed 32ms CPU time Nov 16 14:12:49.149529 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:49.149498 1563 azure_mounter.go:170] azureDisk - successfully mounted disk ropatil1611az-gs6zj-dynamic-pvc-e53f0905-53ec-4da6-829f-022ba78b9629 on /var/lib/kubelet/pods/e3d0df21-083b-4e97-a997-e9a7e992eda4/volumes/kubernetes.io~azure-disk/pvc-e53f0905-53ec-4da6-829f-022ba78b9629 Nov 16 14:12:49.149625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:49.149557 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"pvc-e53f0905-53ec-4da6-829f-022ba78b9629\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-e53f0905-53ec-4da6-829f-022ba78b9629\") pod \"mydep-49496-65f7bb956d-stlkx\" (UID: \"e3d0df21-083b-4e97-a997-e9a7e992eda4\") " pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" Nov 16 14:12:49.211116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:49.211054 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" Nov 16 14:12:49.212044 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:49.211958413Z" level=info msg="Running pod sandbox: migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx/POD" id=df41ea45-c88e-48e1-8de0-89d7a1c65ede name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:12:49.212453 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:49.212050214Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:12:49.268236 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:49.268194924Z" level=info msg="Got pod network &{Name:mydep-49496-65f7bb956d-stlkx Namespace:migration-upgrade-49496 ID:18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4 UID:e3d0df21-083b-4e97-a997-e9a7e992eda4 NetNS:/var/run/netns/d48ac26c-599c-4021-a314-85262e441414 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:12:49.268236 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:49.268235625Z" level=info msg="Adding pod migration-upgrade-49496_mydep-49496-65f7bb956d-stlkx to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:12:49.313210 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:12:49.330228 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607969.3302] manager: (veth694c84dd): new Veth device (/org/freedesktop/NetworkManager/Devices/52) Nov 16 14:12:49.331143 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607969.3311] device (veth694c84dd): carrier: link connected Nov 16 14:12:49.346492 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth694c84dd: link is not ready Nov 16 14:12:49.346581 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth694c84dd: link becomes ready Nov 16 14:12:49.346615 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:12:49.353300 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[135726]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:12:49.353498 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[135726]: Could not generate persistent MAC address for veth694c84dd: No such file or directory Nov 16 14:12:49.370645 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668607969.3705] manager: (veth694c84dd): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/53) Nov 16 14:12:49.376866 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00153|bridge|INFO|bridge br0: added interface veth694c84dd on port 24 Nov 16 14:12:49.377230 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth694c84dd entered promiscuous mode Nov 16 14:12:49.412237 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00154|connmgr|INFO|br0<->unix#327: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:12:49.465910 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00155|connmgr|INFO|br0<->unix#331: 6 flow_mods in the last 0 s (2 adds, 4 deletes) Nov 16 14:12:49.468645 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00156|connmgr|INFO|br0<->unix#333: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:12:50.527889 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:12:50Z [verbose] Add: migration-upgrade-49496:mydep-49496-65f7bb956d-stlkx:e3d0df21-083b-4e97-a997-e9a7e992eda4:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/d48ac26c-599c-4021-a314-85262e441414"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.23/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:12:50.527889 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:12:50.504848 135709 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"migration-upgrade-49496", Name:"mydep-49496-65f7bb956d-stlkx", UID:"e3d0df21-083b-4e97-a997-e9a7e992eda4", APIVersion:"v1", ResourceVersion:"75244", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.23/23] from openshift-sdn Nov 16 14:12:50.528526 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:50.528482245Z" level=info msg="Got pod network &{Name:mydep-49496-65f7bb956d-stlkx Namespace:migration-upgrade-49496 ID:18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4 UID:e3d0df21-083b-4e97-a997-e9a7e992eda4 NetNS:/var/run/netns/d48ac26c-599c-4021-a314-85262e441414 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:12:50.528583 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:50.528535 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx] Nov 16 14:12:50.528884 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:50.528667646Z" level=info msg="Checking pod migration-upgrade-49496_mydep-49496-65f7bb956d-stlkx for CNI network multus-cni-network (type=multus)" Nov 16 14:12:50.531690 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:12:50.531641 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3d0df21_083b_4e97_a997_e9a7e992eda4.slice/crio-18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4.scope WatchSource:0}: Error finding container 18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4: Status 404 returned error &{%!s(*http.body=&{0xc0029a7ab8 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:12:50.543602 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:50.543527855Z" level=info msg="Ran pod sandbox 18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4 with infra container: migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx/POD" id=df41ea45-c88e-48e1-8de0-89d7a1c65ede name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:12:50.544710 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:50.544668764Z" level=info msg="Checking image status: quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339" id=d78a0c9b-b331-49db-ad8c-ec2805842114 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:50.544962 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:50.544925365Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:e4f3998ce785a7eb20d958a205eebb88b77e5de16bc616e5f1d35b6be80021a9,RepoTags:[],RepoDigests:[quay.io/openshifttest/hello-openshift@sha256:2b1a20c5faceb72a15b20ae0b5b83e3cf87fd76ee5375941071a0efbc196ea45 quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339],Size_:266824987,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=d78a0c9b-b331-49db-ad8c-ec2805842114 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:50.545705 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:50.545610470Z" level=info msg="Checking image status: quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339" id=1c7a8097-71c4-4b87-9a5a-849e0f5c526e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:50.545902 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:50.545866572Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:e4f3998ce785a7eb20d958a205eebb88b77e5de16bc616e5f1d35b6be80021a9,RepoTags:[],RepoDigests:[quay.io/openshifttest/hello-openshift@sha256:2b1a20c5faceb72a15b20ae0b5b83e3cf87fd76ee5375941071a0efbc196ea45 quay.io/openshifttest/hello-openshift@sha256:56c354e7885051b6bb4263f9faa58b2c292d44790599b7dde0e49e7c466cf339],Size_:266824987,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=1c7a8097-71c4-4b87-9a5a-849e0f5c526e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:12:50.546962 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:50.546899780Z" level=info msg="Creating container: migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx/mydep-49496" id=b4612550-aec3-4fcc-9ac6-5f29a4f5c9f4 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:12:50.547044 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:50.547013481Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:12:50.610325 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8.scope. Nov 16 14:12:50.631277 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8. Nov 16 14:12:50.675448 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:12:50.742197 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:50.742089808Z" level=info msg="Created container e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8: migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx/mydep-49496" id=b4612550-aec3-4fcc-9ac6-5f29a4f5c9f4 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:12:50.742777 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:50.742735513Z" level=info msg="Starting container: e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8" id=ba12d1db-5c32-4d17-9778-b7dfdb9ca873 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:12:50.754188 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:12:50.754104496Z" level=info msg="Started container" PID=135825 containerID=e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8 description=migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx/mydep-49496 id=ba12d1db-5c32-4d17-9778-b7dfdb9ca873 name=/runtime.v1.RuntimeService/StartContainer sandboxID=18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4 Nov 16 14:12:51.245115 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:51.245073 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" event=&{ID:e3d0df21-083b-4e97-a997-e9a7e992eda4 Type:ContainerStarted Data:e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8} Nov 16 14:12:51.245115 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:12:51.245122 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" event=&{ID:e3d0df21-083b-4e97-a997-e9a7e992eda4 Type:ContainerStarted Data:18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4} Nov 16 14:12:56.925497 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d-runc.SWEHi9.mount: Succeeded. Nov 16 14:12:57.882554 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.bxSkpi.mount: Succeeded. Nov 16 14:12:57.882583 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.iGaBjd.mount: Succeeded. Nov 16 14:13:06.983527 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.iszDB4.mount: Succeeded. Nov 16 14:13:20.451284 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:13:20.451106131Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=3898b88b-89a1-4653-ba39-e635766a91a4 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:13:20.499590 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:13:20.451407033Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=3898b88b-89a1-4653-ba39-e635766a91a4 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:17:36.986309 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.DpItKH.mount: Succeeded. Nov 16 14:17:52.345781 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:17:52.344056 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-8kdsk] Nov 16 14:17:52.345781 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:17:52.344137 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:17:52.361103 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod71e69815_2b9b_490e_9da1_50c41c781920.slice. Nov 16 14:17:52.472837 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:17:52.472759 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mp8j\" (UniqueName: \"kubernetes.io/projected/71e69815-2b9b-490e-9da1-50c41c781920-kube-api-access-9mp8j\") pod \"qe-app-registry-8kdsk\" (UID: \"71e69815-2b9b-490e-9da1-50c41c781920\") " pod="openshift-marketplace/qe-app-registry-8kdsk" Nov 16 14:17:52.573773 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:17:52.573641 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-9mp8j\" (UniqueName: \"kubernetes.io/projected/71e69815-2b9b-490e-9da1-50c41c781920-kube-api-access-9mp8j\") pod \"qe-app-registry-8kdsk\" (UID: \"71e69815-2b9b-490e-9da1-50c41c781920\") " pod="openshift-marketplace/qe-app-registry-8kdsk" Nov 16 14:17:52.605337 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:17:52.605162 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mp8j\" (UniqueName: \"kubernetes.io/projected/71e69815-2b9b-490e-9da1-50c41c781920-kube-api-access-9mp8j\") pod \"qe-app-registry-8kdsk\" (UID: \"71e69815-2b9b-490e-9da1-50c41c781920\") " pod="openshift-marketplace/qe-app-registry-8kdsk" Nov 16 14:17:52.676638 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:17:52.676535 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-8kdsk" Nov 16 14:17:52.677658 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:52.677488156Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-8kdsk/POD" id=dd2afd6a-17a8-49d6-bc45-ec22112e7ef2 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:17:52.677658 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:52.677646857Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:17:52.734621 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:52.734573862Z" level=info msg="Got pod network &{Name:qe-app-registry-8kdsk Namespace:openshift-marketplace ID:48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db UID:71e69815-2b9b-490e-9da1-50c41c781920 NetNS:/var/run/netns/10c689fc-d9a4-431b-93c2-685de3b2381a Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:17:52.734621 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:52.734618462Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-8kdsk to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:17:52.787268 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:17:52.801271 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668608272.8012] manager: (veth17d3ddd9): new Veth device (/org/freedesktop/NetworkManager/Devices/54) Nov 16 14:17:52.802224 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668608272.8022] device (veth17d3ddd9): carrier: link connected Nov 16 14:17:52.807830 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[140901]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:17:52.814842 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth17d3ddd9: link is not ready Nov 16 14:17:52.814916 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth17d3ddd9: link becomes ready Nov 16 14:17:52.814954 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:17:52.820713 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[140901]: Could not generate persistent MAC address for veth17d3ddd9: No such file or directory Nov 16 14:17:52.835531 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668608272.8355] manager: (veth17d3ddd9): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/55) Nov 16 14:17:52.841486 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00157|bridge|INFO|bridge br0: added interface veth17d3ddd9 on port 25 Nov 16 14:17:52.842146 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth17d3ddd9 entered promiscuous mode Nov 16 14:17:52.872555 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00158|connmgr|INFO|br0<->unix#339: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:17:52.917723 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00159|connmgr|INFO|br0<->unix#342: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:17:54.682628 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:17:54.682515 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-marketplace/qe-app-registry-8kdsk] Nov 16 14:17:54.683114 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:17:54Z [verbose] Add: openshift-marketplace:qe-app-registry-8kdsk:71e69815-2b9b-490e-9da1-50c41c781920:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/10c689fc-d9a4-431b-93c2-685de3b2381a"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.24/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:17:54.683114 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:17:54.658919 140884 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-8kdsk", UID:"71e69815-2b9b-490e-9da1-50c41c781920", APIVersion:"v1", ResourceVersion:"78124", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.24/23] from openshift-sdn Nov 16 14:17:54.683458 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:54.683302011Z" level=info msg="Got pod network &{Name:qe-app-registry-8kdsk Namespace:openshift-marketplace ID:48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db UID:71e69815-2b9b-490e-9da1-50c41c781920 NetNS:/var/run/netns/10c689fc-d9a4-431b-93c2-685de3b2381a Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:17:54.683497 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:54.683453612Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-8kdsk for CNI network multus-cni-network (type=multus)" Nov 16 14:17:54.685876 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:17:54.685750 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e69815_2b9b_490e_9da1_50c41c781920.slice/crio-48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db.scope WatchSource:0}: Error finding container 48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db: Status 404 returned error &{%!s(*http.body=&{0xc002d5d188 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:17:54.693553 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:54.693475883Z" level=info msg="Ran pod sandbox 48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db with infra container: openshift-marketplace/qe-app-registry-8kdsk/POD" id=dd2afd6a-17a8-49d6-bc45-ec22112e7ef2 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:17:54.694662 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:54.694582291Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=7f78160c-d5d0-4e21-8d16-46bf12dcb656 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:17:54.694836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:54.694800592Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=7f78160c-d5d0-4e21-8d16-46bf12dcb656 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:17:54.695369 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:17:54.695301 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 14:17:54.695875 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:54.695835100Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=1c2277ea-176d-43b2-8c10-9f4128911ecd name=/runtime.v1.ImageService/PullImage Nov 16 14:17:54.718579 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:54.718550161Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 14:17:55.138703 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:17:55.138664 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-8kdsk" event=&{ID:71e69815-2b9b-490e-9da1-50c41c781920 Type:ContainerStarted Data:48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db} Nov 16 14:17:56.286412 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:56.286299326Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=1c2277ea-176d-43b2-8c10-9f4128911ecd name=/runtime.v1.ImageService/PullImage Nov 16 14:17:56.287424 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:56.287386235Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=22100339-e235-4c55-a678-4bade84419d0 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:17:56.287642 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:56.287607736Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=22100339-e235-4c55-a678-4bade84419d0 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:17:56.288936 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:56.288898046Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-8kdsk/registry-server" id=4e953b4c-2105-4e29-b8cd-86f470e29089 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:17:56.289030 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:56.289002646Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:17:56.362556 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064.scope. Nov 16 14:17:56.390399 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064. Nov 16 14:17:56.424081 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:17:56.486677 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:56.486578305Z" level=info msg="Created container 1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064: openshift-marketplace/qe-app-registry-8kdsk/registry-server" id=4e953b4c-2105-4e29-b8cd-86f470e29089 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:17:56.487298 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:56.487260810Z" level=info msg="Starting container: 1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064" id=0299d153-19ff-479f-a2c6-4241c51564d9 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:17:56.497675 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:17:56.497599186Z" level=info msg="Started container" PID=140980 containerID=1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064 description=openshift-marketplace/qe-app-registry-8kdsk/registry-server id=0299d153-19ff-479f-a2c6-4241c51564d9 name=/runtime.v1.RuntimeService/StartContainer sandboxID=48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db Nov 16 14:17:57.145972 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:17:57.145926 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-8kdsk" event=&{ID:71e69815-2b9b-490e-9da1-50c41c781920 Type:ContainerStarted Data:1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064} Nov 16 14:18:02.676928 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:02.676736 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/qe-app-registry-8kdsk" Nov 16 14:18:02.676928 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:02.676805 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/qe-app-registry-8kdsk" Nov 16 14:18:02.735093 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:02.735054 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/qe-app-registry-8kdsk" Nov 16 14:18:03.216742 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:03.216670 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/qe-app-registry-8kdsk" Nov 16 14:18:03.263371 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:03.263246 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-8kdsk] Nov 16 14:18:05.172343 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:05.172241 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-8kdsk" podUID=71e69815-2b9b-490e-9da1-50c41c781920 containerName="registry-server" containerID="cri-o://1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064" gracePeriod=1 Nov 16 14:18:05.172924 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:05.172576357Z" level=info msg="Stopping container: 1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064 (timeout: 1s)" id=777b6672-7c7a-4b09-8f2e-8ff61e9478e9 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:18:05.200511 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064.scope: Succeeded. Nov 16 14:18:05.201101 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064.scope: Consumed 680ms CPU time Nov 16 14:18:05.209645 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064.scope: Succeeded. Nov 16 14:18:05.210269 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064.scope: Consumed 37ms CPU time Nov 16 14:18:05.328385 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-8280ad238b9e7634b7e27d09d6603df1caa398468774a383872523062c645bbf-merged.mount: Succeeded. Nov 16 14:18:06.099018 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:06.098939044Z" level=info msg="Stopped container 1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064: openshift-marketplace/qe-app-registry-8kdsk/registry-server" id=777b6672-7c7a-4b09-8f2e-8ff61e9478e9 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:18:06.099655 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:06.099614149Z" level=info msg="Stopping pod sandbox: 48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db" id=512f2282-6e7a-4631-95f6-31349b66ae49 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:18:06.099982 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:06.099940152Z" level=info msg="Got pod network &{Name:qe-app-registry-8kdsk Namespace:openshift-marketplace ID:48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db UID:71e69815-2b9b-490e-9da1-50c41c781920 NetNS:/var/run/netns/10c689fc-d9a4-431b-93c2-685de3b2381a Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:18:06.100136 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:06.100091353Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-8kdsk from CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:18:06.168333 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00160|connmgr|INFO|br0<->unix#345: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:18:06.175315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:06.175263 1563 generic.go:296] "Generic (PLEG): container finished" podID=71e69815-2b9b-490e-9da1-50c41c781920 containerID="1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064" exitCode=0 Nov 16 14:18:06.175712 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:06.175336 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-8kdsk" event=&{ID:71e69815-2b9b-490e-9da1-50c41c781920 Type:ContainerDied Data:1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064} Nov 16 14:18:06.202563 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00161|connmgr|INFO|br0<->unix#348: 4 flow_mods in the last 0 s (4 deletes) Nov 16 14:18:06.232200 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00162|bridge|INFO|bridge br0: deleted interface veth17d3ddd9 on port 25 Nov 16 14:18:06.238175 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth17d3ddd9 left promiscuous mode Nov 16 14:18:06.245867 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:18:06Z [verbose] Del: openshift-marketplace:qe-app-registry-8kdsk:71e69815-2b9b-490e-9da1-50c41c781920:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 14:18:06.254085 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-10c689fc\x2dd9a4\x2d431b\x2d93c2\x2d685de3b2381a.mount: Succeeded. Nov 16 14:18:06.262029 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-10c689fc\x2dd9a4\x2d431b\x2d93c2\x2d685de3b2381a.mount: Succeeded. Nov 16 14:18:06.299547 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-10c689fc\x2dd9a4\x2d431b\x2d93c2\x2d685de3b2381a.mount: Succeeded. Nov 16 14:18:06.299592 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db-userdata-shm.mount: Succeeded. Nov 16 14:18:06.309678 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-efe1fcd3df51372ff3df8e537e58ae5933ddf782dd96654db37ab6de37e919a3-merged.mount: Succeeded. Nov 16 14:18:06.453226 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:06.452966400Z" level=info msg="Stopped pod sandbox: 48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db" id=512f2282-6e7a-4631-95f6-31349b66ae49 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:18:06.584194 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:06.584147 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mp8j\" (UniqueName: \"kubernetes.io/projected/71e69815-2b9b-490e-9da1-50c41c781920-kube-api-access-9mp8j\") pod \"71e69815-2b9b-490e-9da1-50c41c781920\" (UID: \"71e69815-2b9b-490e-9da1-50c41c781920\") " Nov 16 14:18:06.594804 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-71e69815\x2d2b9b\x2d490e\x2d9da1\x2d50c41c781920-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d9mp8j.mount: Succeeded. Nov 16 14:18:06.599722 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:06.599648 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e69815-2b9b-490e-9da1-50c41c781920-kube-api-access-9mp8j" (OuterVolumeSpecName: "kube-api-access-9mp8j") pod "71e69815-2b9b-490e-9da1-50c41c781920" (UID: "71e69815-2b9b-490e-9da1-50c41c781920"). InnerVolumeSpecName "kube-api-access-9mp8j". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:18:06.685326 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:06.685244 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-9mp8j\" (UniqueName: \"kubernetes.io/projected/71e69815-2b9b-490e-9da1-50c41c781920-kube-api-access-9mp8j\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:18:07.180005 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:07.179963 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-8kdsk" event=&{ID:71e69815-2b9b-490e-9da1-50c41c781920 Type:ContainerDied Data:48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db} Nov 16 14:18:07.180005 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:07.180023 1563 scope.go:110] "RemoveContainer" containerID="1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064" Nov 16 14:18:07.181201 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:07.181147657Z" level=info msg="Removing container: 1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064" id=71f6897a-f6cb-43ed-8ff5-c9466969cc65 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:18:07.188348 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod71e69815_2b9b_490e_9da1_50c41c781920.slice. Nov 16 14:18:07.188417 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod71e69815_2b9b_490e_9da1_50c41c781920.slice: Consumed 717ms CPU time Nov 16 14:18:07.218340 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:07.218304 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-8kdsk] Nov 16 14:18:07.222459 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:07.222420 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-8kdsk] Nov 16 14:18:07.228899 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:07.228847001Z" level=info msg="Removed container 1089276650e5e161646e41e0841db205e68133206cc8d7819883956ef389c064: openshift-marketplace/qe-app-registry-8kdsk/registry-server" id=71f6897a-f6cb-43ed-8ff5-c9466969cc65 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:18:08.594932 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:18:08.594889 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=71e69815-2b9b-490e-9da1-50c41c781920 path="/var/lib/kubelet/pods/71e69815-2b9b-490e-9da1-50c41c781920/volumes" Nov 16 14:18:16.983943 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.4awifg.mount: Succeeded. Nov 16 14:18:20.456321 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:20.456161859Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=1343a7f5-03bd-44b4-bc34-7bf625e0c42b name=/runtime.v1.ImageService/ImageStatus Nov 16 14:18:20.595933 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:20.456449861Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=1343a7f5-03bd-44b4-bc34-7bf625e0c42b name=/runtime.v1.ImageService/ImageStatus Nov 16 14:18:21.528691 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:21.528638728Z" level=info msg="Stopping pod sandbox: 48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db" id=13cbe40f-1bc3-4df1-8456-da2989f8bd5c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:18:21.528691 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:21.528693029Z" level=info msg="Stopped pod sandbox (already stopped): 48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db" id=13cbe40f-1bc3-4df1-8456-da2989f8bd5c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:18:21.529354 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:21.529092632Z" level=info msg="Removing pod sandbox: 48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db" id=aeebc87e-4f89-4ae3-a009-32144c602cf7 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:18:21.548888 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:18:21.548821175Z" level=info msg="Removed pod sandbox: 48e0f3cab8d8cf4262ddefed76d8077a06e6d6ef4d3b4302ef0a75ed178820db" id=aeebc87e-4f89-4ae3-a009-32144c602cf7 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:18:46.983698 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.oVb9P6.mount: Succeeded. Nov 16 14:19:26.984697 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.HZNp6y.mount: Succeeded. Nov 16 14:19:36.983833 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.AZzBcH.mount: Succeeded. Nov 16 14:20:26.983235 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.7NBw34.mount: Succeeded. Nov 16 14:21:16.982178 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.fF4kiL.mount: Succeeded. Nov 16 14:22:06.984795 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.ql0JUF.mount: Succeeded. Nov 16 14:22:16.984221 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.2LVxrO.mount: Succeeded. Nov 16 14:23:06.982246 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.3L58cK.mount: Succeeded. Nov 16 14:23:20.461334 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:23:20.461172551Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=59f83700-c4be-4a2a-8346-cdfca2cbdc9c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:23:20.511643 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:23:20.461428753Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=59f83700-c4be-4a2a-8346-cdfca2cbdc9c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:28:20.466958 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:28:20.466786750Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=73667086-0918-406d-bd74-3c7b2020117e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:28:20.483741 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:28:20.467077053Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=73667086-0918-406d-bd74-3c7b2020117e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:31:46.983455 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.37tvGN.mount: Succeeded. Nov 16 14:33:20.472777 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:20.472523055Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=e5c90f63-7ead-4065-a0b1-51dc2b84d442 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:33:20.495957 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:20.472873257Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=e5c90f63-7ead-4065-a0b1-51dc2b84d442 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:33:37.327439 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:37.327098 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-qd7gq] Nov 16 14:33:37.327439 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:37.327202 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:33:37.327439 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:33:37.327330 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="71e69815-2b9b-490e-9da1-50c41c781920" containerName="registry-server" Nov 16 14:33:37.327439 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:37.327348 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e69815-2b9b-490e-9da1-50c41c781920" containerName="registry-server" Nov 16 14:33:37.327439 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:37.327418 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="71e69815-2b9b-490e-9da1-50c41c781920" containerName="registry-server" Nov 16 14:33:37.336880 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod56d892fe_c1cd_4943_88d8_5ba219e47fb9.slice. Nov 16 14:33:37.445498 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:37.445451 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf4df\" (UniqueName: \"kubernetes.io/projected/56d892fe-c1cd-4943-88d8-5ba219e47fb9-kube-api-access-tf4df\") pod \"qe-app-registry-qd7gq\" (UID: \"56d892fe-c1cd-4943-88d8-5ba219e47fb9\") " pod="openshift-marketplace/qe-app-registry-qd7gq" Nov 16 14:33:37.547020 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:37.546867 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-tf4df\" (UniqueName: \"kubernetes.io/projected/56d892fe-c1cd-4943-88d8-5ba219e47fb9-kube-api-access-tf4df\") pod \"qe-app-registry-qd7gq\" (UID: \"56d892fe-c1cd-4943-88d8-5ba219e47fb9\") " pod="openshift-marketplace/qe-app-registry-qd7gq" Nov 16 14:33:37.575744 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:37.575664 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf4df\" (UniqueName: \"kubernetes.io/projected/56d892fe-c1cd-4943-88d8-5ba219e47fb9-kube-api-access-tf4df\") pod \"qe-app-registry-qd7gq\" (UID: \"56d892fe-c1cd-4943-88d8-5ba219e47fb9\") " pod="openshift-marketplace/qe-app-registry-qd7gq" Nov 16 14:33:37.655805 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:37.655702 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-qd7gq" Nov 16 14:33:37.656573 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:37.656450188Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-qd7gq/POD" id=bbf9a88d-88aa-4576-aa7a-d9d7e88f2c5a name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:33:37.656573 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:37.656557489Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:33:37.713402 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:37.713199016Z" level=info msg="Got pod network &{Name:qe-app-registry-qd7gq Namespace:openshift-marketplace ID:d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad UID:56d892fe-c1cd-4943-88d8-5ba219e47fb9 NetNS:/var/run/netns/5525d364-b451-46f6-ba81-4dae1fb616fa Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:33:37.713402 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:37.713301217Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-qd7gq to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:33:37.761217 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:33:37.782415 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth58c69080: link is not ready Nov 16 14:33:37.782510 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth58c69080: link becomes ready Nov 16 14:33:37.782546 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:33:37.788915 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[156976]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:33:37.789099 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[156976]: Could not generate persistent MAC address for veth58c69080: No such file or directory Nov 16 14:33:37.789250 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668609217.7892] device (veth58c69080): carrier: link connected Nov 16 14:33:37.789625 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668609217.7896] manager: (veth58c69080): new Veth device (/org/freedesktop/NetworkManager/Devices/56) Nov 16 14:33:37.803066 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668609217.8030] manager: (veth58c69080): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/57) Nov 16 14:33:37.808542 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00163|bridge|INFO|bridge br0: added interface veth58c69080 on port 26 Nov 16 14:33:37.809165 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth58c69080 entered promiscuous mode Nov 16 14:33:37.841348 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00164|connmgr|INFO|br0<->unix#359: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:33:37.887508 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00165|connmgr|INFO|br0<->unix#362: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:33:39.009345 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:33:38Z [verbose] Add: openshift-marketplace:qe-app-registry-qd7gq:56d892fe-c1cd-4943-88d8-5ba219e47fb9:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/5525d364-b451-46f6-ba81-4dae1fb616fa"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.25/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:33:39.009345 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:33:38.980730 156960 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-qd7gq", UID:"56d892fe-c1cd-4943-88d8-5ba219e47fb9", APIVersion:"v1", ResourceVersion:"84851", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.25/23] from openshift-sdn Nov 16 14:33:39.009898 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:39.009856925Z" level=info msg="Got pod network &{Name:qe-app-registry-qd7gq Namespace:openshift-marketplace ID:d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad UID:56d892fe-c1cd-4943-88d8-5ba219e47fb9 NetNS:/var/run/netns/5525d364-b451-46f6-ba81-4dae1fb616fa Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:33:39.010027 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:39.010001626Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-qd7gq for CNI network multus-cni-network (type=multus)" Nov 16 14:33:39.010954 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:39.010923 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-marketplace/qe-app-registry-qd7gq] Nov 16 14:33:39.012855 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:33:39.012815 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d892fe_c1cd_4943_88d8_5ba219e47fb9.slice/crio-d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad.scope WatchSource:0}: Error finding container d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad: Status 404 returned error &{%!s(*http.body=&{0xc003f7fc20 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:33:39.024277 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:39.024237783Z" level=info msg="Ran pod sandbox d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad with infra container: openshift-marketplace/qe-app-registry-qd7gq/POD" id=bbf9a88d-88aa-4576-aa7a-d9d7e88f2c5a name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:33:39.025437 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:39.025395488Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=62a782a1-245a-4855-8447-3dd8fada7d47 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:33:39.025685 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:39.025649489Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=62a782a1-245a-4855-8447-3dd8fada7d47 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:33:39.026279 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:39.026247 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 14:33:39.026996 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:39.026947294Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=0aa4474e-72cd-4d0f-891e-c564705f7ca8 name=/runtime.v1.ImageService/PullImage Nov 16 14:33:39.039056 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:39.039020542Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 14:33:39.833536 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:39.833245 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-qd7gq" event=&{ID:56d892fe-c1cd-4943-88d8-5ba219e47fb9 Type:ContainerStarted Data:d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad} Nov 16 14:33:40.618753 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:40.618654883Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=0aa4474e-72cd-4d0f-891e-c564705f7ca8 name=/runtime.v1.ImageService/PullImage Nov 16 14:33:40.619742 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:40.619704390Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=bb4afab1-d637-4d6b-9381-5a7b270b14c4 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:33:40.619956 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:40.619911992Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=bb4afab1-d637-4d6b-9381-5a7b270b14c4 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:33:40.620912 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:40.620831698Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-qd7gq/registry-server" id=edcc0c6d-1acd-4ec4-a99a-519c4c22d7d3 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:33:40.620998 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:40.620943499Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:33:40.681802 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06.scope. Nov 16 14:33:40.712932 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06. Nov 16 14:33:40.748536 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:33:40.814870 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:40.814745023Z" level=info msg="Created container 99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06: openshift-marketplace/qe-app-registry-qd7gq/registry-server" id=edcc0c6d-1acd-4ec4-a99a-519c4c22d7d3 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:33:40.815488 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:40.815452529Z" level=info msg="Starting container: 99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06" id=49bfd652-5c5b-44c9-bdcf-370d7b8a81a7 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:33:40.826320 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:40.826276819Z" level=info msg="Started container" PID=157058 containerID=99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06 description=openshift-marketplace/qe-app-registry-qd7gq/registry-server id=49bfd652-5c5b-44c9-bdcf-370d7b8a81a7 name=/runtime.v1.RuntimeService/StartContainer sandboxID=d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad Nov 16 14:33:41.841934 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:41.841893 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-qd7gq" event=&{ID:56d892fe-c1cd-4943-88d8-5ba219e47fb9 Type:ContainerStarted Data:99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06} Nov 16 14:33:47.657066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:47.656950 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/qe-app-registry-qd7gq" Nov 16 14:33:47.657066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:47.657040 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/qe-app-registry-qd7gq" Nov 16 14:33:47.714553 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:47.714516 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/qe-app-registry-qd7gq" Nov 16 14:33:47.920391 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:47.920246 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/qe-app-registry-qd7gq" Nov 16 14:33:47.961866 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:47.961774 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-qd7gq] Nov 16 14:33:49.865606 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:49.865514 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-qd7gq" podUID=56d892fe-c1cd-4943-88d8-5ba219e47fb9 containerName="registry-server" containerID="cri-o://99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06" gracePeriod=1 Nov 16 14:33:49.866103 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:49.865746550Z" level=info msg="Stopping container: 99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06 (timeout: 1s)" id=6bf2995e-3458-4b0e-9167-8ed10084f396 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:33:49.892098 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06.scope: Succeeded. Nov 16 14:33:49.892823 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06.scope: Consumed 687ms CPU time Nov 16 14:33:49.903318 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06.scope: Succeeded. Nov 16 14:33:49.904049 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06.scope: Consumed 41ms CPU time Nov 16 14:33:50.019248 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-435b2b08f536e23949a947e07b0672a861af1aeafca93994fa00edae5d4b701f-merged.mount: Succeeded. Nov 16 14:33:50.369345 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.RrW6h4.mount: Succeeded. Nov 16 14:33:50.372989 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:50.372871831Z" level=info msg="Stopped container 99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06: openshift-marketplace/qe-app-registry-qd7gq/registry-server" id=6bf2995e-3458-4b0e-9167-8ed10084f396 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:33:50.373565 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:50.373524741Z" level=info msg="Stopping pod sandbox: d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad" id=ce34ed11-9f2f-4043-9b20-f486ad907efb name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:33:50.373845 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:50.373807245Z" level=info msg="Got pod network &{Name:qe-app-registry-qd7gq Namespace:openshift-marketplace ID:d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad UID:56d892fe-c1cd-4943-88d8-5ba219e47fb9 NetNS:/var/run/netns/5525d364-b451-46f6-ba81-4dae1fb616fa Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:33:50.373984 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:50.373958147Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-qd7gq from CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:33:50.453934 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00166|connmgr|INFO|br0<->unix#365: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:33:50.486214 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00167|connmgr|INFO|br0<->unix#368: 4 flow_mods in the last 0 s (4 deletes) Nov 16 14:33:50.517255 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00168|bridge|INFO|bridge br0: deleted interface veth58c69080 on port 26 Nov 16 14:33:50.524160 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth58c69080 left promiscuous mode Nov 16 14:33:50.533062 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:33:50Z [verbose] Del: openshift-marketplace:qe-app-registry-qd7gq:56d892fe-c1cd-4943-88d8-5ba219e47fb9:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 14:33:50.541237 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-5525d364\x2db451\x2d46f6\x2dba81\x2d4dae1fb616fa.mount: Succeeded. Nov 16 14:33:50.548934 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-5525d364\x2db451\x2d46f6\x2dba81\x2d4dae1fb616fa.mount: Succeeded. Nov 16 14:33:50.749672 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:50.749507713Z" level=info msg="Stopped pod sandbox: d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad" id=ce34ed11-9f2f-4043-9b20-f486ad907efb name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:33:50.861304 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:50.861247 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf4df\" (UniqueName: \"kubernetes.io/projected/56d892fe-c1cd-4943-88d8-5ba219e47fb9-kube-api-access-tf4df\") pod \"56d892fe-c1cd-4943-88d8-5ba219e47fb9\" (UID: \"56d892fe-c1cd-4943-88d8-5ba219e47fb9\") " Nov 16 14:33:50.869964 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:50.869935 1563 generic.go:296] "Generic (PLEG): container finished" podID=56d892fe-c1cd-4943-88d8-5ba219e47fb9 containerID="99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06" exitCode=0 Nov 16 14:33:50.870380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:50.869987 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-qd7gq" event=&{ID:56d892fe-c1cd-4943-88d8-5ba219e47fb9 Type:ContainerDied Data:99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06} Nov 16 14:33:50.870380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:50.870024 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-qd7gq" event=&{ID:56d892fe-c1cd-4943-88d8-5ba219e47fb9 Type:ContainerDied Data:d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad} Nov 16 14:33:50.870380 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:50.870048 1563 scope.go:110] "RemoveContainer" containerID="99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06" Nov 16 14:33:50.871121 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:50.871038382Z" level=info msg="Removing container: 99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06" id=5173f346-d285-4ca2-8185-afdca55ccd6a name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:33:50.876828 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:50.876755 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56d892fe-c1cd-4943-88d8-5ba219e47fb9-kube-api-access-tf4df" (OuterVolumeSpecName: "kube-api-access-tf4df") pod "56d892fe-c1cd-4943-88d8-5ba219e47fb9" (UID: "56d892fe-c1cd-4943-88d8-5ba219e47fb9"). InnerVolumeSpecName "kube-api-access-tf4df". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:33:50.903245 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:33:50.903105348Z" level=info msg="Removed container 99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06: openshift-marketplace/qe-app-registry-qd7gq/registry-server" id=5173f346-d285-4ca2-8185-afdca55ccd6a name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:33:50.903493 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:50.903458 1563 scope.go:110] "RemoveContainer" containerID="99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06" Nov 16 14:33:50.903876 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:33:50.903849 1563 remote_runtime.go:597] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06\": container with ID starting with 99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06 not found: ID does not exist" containerID="99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06" Nov 16 14:33:50.903946 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:50.903902 1563 pod_container_deletor.go:52] "DeleteContainer returned error" containerID={Type:cri-o ID:99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06} err="failed to get container status \"99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06\": rpc error: code = NotFound desc = could not find container \"99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06\": container with ID starting with 99f888b9c2c9f93c6ccb2de24c6b2eab909fccf064abee125a954829e6997d06 not found: ID does not exist" Nov 16 14:33:50.962444 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:50.962344 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-tf4df\" (UniqueName: \"kubernetes.io/projected/56d892fe-c1cd-4943-88d8-5ba219e47fb9-kube-api-access-tf4df\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:33:51.018568 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-bd149eb3aa1c6f8535d5929f79657a39a69e50d3ba310f6bdf8004313aa3a8e5-merged.mount: Succeeded. Nov 16 14:33:51.018598 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-5525d364\x2db451\x2d46f6\x2dba81\x2d4dae1fb616fa.mount: Succeeded. Nov 16 14:33:51.018606 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad-userdata-shm.mount: Succeeded. Nov 16 14:33:51.018613 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-56d892fe\x2dc1cd\x2d4943\x2d88d8\x2d5ba219e47fb9-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dtf4df.mount: Succeeded. Nov 16 14:33:51.177689 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod56d892fe_c1cd_4943_88d8_5ba219e47fb9.slice. Nov 16 14:33:51.177757 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod56d892fe_c1cd_4943_88d8_5ba219e47fb9.slice: Consumed 729ms CPU time Nov 16 14:33:51.197990 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:51.197918 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-qd7gq] Nov 16 14:33:51.203966 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:51.203937 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-qd7gq] Nov 16 14:33:52.596844 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:33:52.596702 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=56d892fe-c1cd-4943-88d8-5ba219e47fb9 path="/var/lib/kubelet/pods/56d892fe-c1cd-4943-88d8-5ba219e47fb9/volumes" Nov 16 14:34:16.983222 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.9za3R2.mount: Succeeded. Nov 16 14:34:20.310033 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.D08rJZ.mount: Succeeded. Nov 16 14:34:21.616913 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:34:21.616814346Z" level=info msg="Stopping pod sandbox: d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad" id=2eb569c8-2e95-4457-a93b-2c4512354236 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:34:21.616913 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:34:21.616909547Z" level=info msg="Stopped pod sandbox (already stopped): d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad" id=2eb569c8-2e95-4457-a93b-2c4512354236 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:34:21.617493 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:34:21.617208949Z" level=info msg="Removing pod sandbox: d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad" id=914355f6-41f5-46a6-a1d8-c9e1995189ce name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:34:21.636060 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:34:21.636013076Z" level=info msg="Removed pod sandbox: d594fd4eb5626221f0fb00cb9c8f7e5c996a982034373c2a5e22346834b369ad" id=914355f6-41f5-46a6-a1d8-c9e1995189ce name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:35:06.982374 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.o7uBBk.mount: Succeeded. Nov 16 14:37:25.310733 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.YXxYk9.mount: Succeeded. Nov 16 14:38:20.476647 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:38:20.476457266Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=ebb568b5-60f6-453e-9914-38f7a0d371b7 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:38:20.497217 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:38:20.476755268Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=ebb568b5-60f6-453e-9914-38f7a0d371b7 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:42:16.985342 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.wPNWSg.mount: Succeeded. Nov 16 14:42:25.309734 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.Ie5oBj.mount: Succeeded. Nov 16 14:43:20.482478 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:43:20.482312398Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=b729de5c-000b-4025-ab68-2045c4721ef8 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:43:20.500575 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:43:20.482582501Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=b729de5c-000b-4025-ab68-2045c4721ef8 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:43:26.983029 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.YqnMrO.mount: Succeeded. Nov 16 14:46:25.309189 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.wYu2Oh.mount: Succeeded. Nov 16 14:48:20.487387 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:48:20.487218156Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=4e5a8f5c-ec5d-4830-a4cf-db9bcd554d9a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:48:20.505912 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:48:20.487512559Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=4e5a8f5c-ec5d-4830-a4cf-db9bcd554d9a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:49:26.982919 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259-runc.0ccsxA.mount: Succeeded. Nov 16 14:49:51.297604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:51.297562 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj] Nov 16 14:49:51.297604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:51.297630 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:49:51.298147 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:49:51.297714 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="56d892fe-c1cd-4943-88d8-5ba219e47fb9" containerName="registry-server" Nov 16 14:49:51.298147 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:51.297730 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="56d892fe-c1cd-4943-88d8-5ba219e47fb9" containerName="registry-server" Nov 16 14:49:51.298147 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:51.297791 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="56d892fe-c1cd-4943-88d8-5ba219e47fb9" containerName="registry-server" Nov 16 14:49:51.308406 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod35f6c830_e480_446f_a5cc_a6aaf007ea9a.slice. Nov 16 14:49:51.401819 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:51.401773 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/35f6c830-e480-446f-a5cc-a6aaf007ea9a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c69dfd44-hxqbj\" (UID: \"35f6c830-e480-446f-a5cc-a6aaf007ea9a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj" Nov 16 14:49:51.503059 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:51.502967 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/35f6c830-e480-446f-a5cc-a6aaf007ea9a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c69dfd44-hxqbj\" (UID: \"35f6c830-e480-446f-a5cc-a6aaf007ea9a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj" Nov 16 14:49:51.506212 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:51.506075 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/35f6c830-e480-446f-a5cc-a6aaf007ea9a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c69dfd44-hxqbj\" (UID: \"35f6c830-e480-446f-a5cc-a6aaf007ea9a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj" Nov 16 14:49:51.621644 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:51.621564 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj" Nov 16 14:49:51.622356 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:51.622275851Z" level=info msg="Running pod sandbox: openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj/POD" id=4d3e9fd8-b55d-44ba-a928-abc356748666 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:49:51.622816 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:51.622377052Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:49:51.679090 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:51.678992712Z" level=info msg="Got pod network &{Name:prometheus-operator-admission-webhook-6c69dfd44-hxqbj Namespace:openshift-monitoring ID:c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844 UID:35f6c830-e480-446f-a5cc-a6aaf007ea9a NetNS:/var/run/netns/b34f6c7c-5574-4659-bd55-848e4e74be11 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:49:51.679090 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:51.679035712Z" level=info msg="Adding pod openshift-monitoring_prometheus-operator-admission-webhook-6c69dfd44-hxqbj to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:49:51.725232 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:49:51.749884 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth93dc054e: link is not ready Nov 16 14:49:51.749980 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth93dc054e: link becomes ready Nov 16 14:49:51.750015 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:49:51.756297 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[174429]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:49:51.756485 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[174429]: Could not generate persistent MAC address for veth93dc054e: No such file or directory Nov 16 14:49:51.756603 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610191.7565] device (veth93dc054e): carrier: link connected Nov 16 14:49:51.757034 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610191.7570] manager: (veth93dc054e): new Veth device (/org/freedesktop/NetworkManager/Devices/58) Nov 16 14:49:51.779413 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610191.7793] manager: (veth93dc054e): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/59) Nov 16 14:49:51.784864 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00169|bridge|INFO|bridge br0: added interface veth93dc054e on port 27 Nov 16 14:49:51.785297 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth93dc054e entered promiscuous mode Nov 16 14:49:51.816602 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00170|connmgr|INFO|br0<->unix#379: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:49:51.862607 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00171|connmgr|INFO|br0<->unix#382: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:49:52.313920 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:52.313830 1563 logs.go:401] "Finished parsing log file, hit bytes limit" path="/var/log/pods/openshift-sdn_sdn-pgwdx_45cd7283-fcc9-4e61-b2b1-517445b7bd5a/sdn/0.log" limit=65536 Nov 16 14:49:52.325249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:52.325223 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-pgwdx_45cd7283-fcc9-4e61-b2b1-517445b7bd5a/kube-rbac-proxy/0.log" Nov 16 14:49:52.494300 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:52.494245 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-pgwdx_45cd7283-fcc9-4e61-b2b1-517445b7bd5a/drop-icmp/0.log" Nov 16 14:49:53.030938 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:53.030879 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-ingress-canary/ingress-canary-c77h2] Nov 16 14:49:53.031332 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:53.031298 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-ingress-canary/ingress-canary-c77h2" podUID=b0793e26-e08f-46cd-bee3-f0ec4509d53a containerName="serve-healthcheck-canary" containerID="cri-o://7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4" gracePeriod=30 Nov 16 14:49:53.031778 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:53.031719799Z" level=info msg="Stopping container: 7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4 (timeout: 30s)" id=62d7be12-a7b4-40af-baf9-963208ab2acd name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:49:53.058301 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4.scope: Succeeded. Nov 16 14:49:53.058958 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4.scope: Consumed 1.323s CPU time Nov 16 14:49:53.064258 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4.scope: Succeeded. Nov 16 14:49:53.064863 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4.scope: Consumed 42ms CPU time Nov 16 14:49:53.183890 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-eed6da2bbd2fcab99d7a60e47da0ec0c80da41f80f1e645b7037a5be5fdc8dc2-merged.mount: Succeeded. Nov 16 14:49:53.184741 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-eed6da2bbd2fcab99d7a60e47da0ec0c80da41f80f1e645b7037a5be5fdc8dc2-merged.mount: Consumed 0 CPU time Nov 16 14:49:53.299100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:53.298938 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj] Nov 16 14:49:53.309435 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:49:53Z [verbose] Add: openshift-monitoring:prometheus-operator-admission-webhook-6c69dfd44-hxqbj:35f6c830-e480-446f-a5cc-a6aaf007ea9a:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/b34f6c7c-5574-4659-bd55-848e4e74be11"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.26/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:49:53.309435 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:49:53.251190 174412 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-monitoring", Name:"prometheus-operator-admission-webhook-6c69dfd44-hxqbj", UID:"35f6c830-e480-446f-a5cc-a6aaf007ea9a", APIVersion:"v1", ResourceVersion:"93020", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.26/23] from openshift-sdn Nov 16 14:49:53.310184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:53.310068459Z" level=info msg="Got pod network &{Name:prometheus-operator-admission-webhook-6c69dfd44-hxqbj Namespace:openshift-monitoring ID:c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844 UID:35f6c830-e480-446f-a5cc-a6aaf007ea9a NetNS:/var/run/netns/b34f6c7c-5574-4659-bd55-848e4e74be11 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:49:53.310286 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:53.310247161Z" level=info msg="Checking pod openshift-monitoring_prometheus-operator-admission-webhook-6c69dfd44-hxqbj for CNI network multus-cni-network (type=multus)" Nov 16 14:49:53.520399 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:49:53.520340 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35f6c830_e480_446f_a5cc_a6aaf007ea9a.slice/crio-c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844.scope WatchSource:0}: Error finding container c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844: Status 404 returned error &{%!s(*http.body=&{0xc001fba528 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:49:53.535485 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:53.526178915Z" level=info msg="Stopped container 7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4: openshift-ingress-canary/ingress-canary-c77h2/serve-healthcheck-canary" id=62d7be12-a7b4-40af-baf9-963208ab2acd name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:49:53.535485 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:53.526678919Z" level=info msg="Stopping pod sandbox: 4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171" id=81e2bedf-3e48-4e23-b6c0-f01cfd012869 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:49:53.535485 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:53.526957021Z" level=info msg="Got pod network &{Name:ingress-canary-c77h2 Namespace:openshift-ingress-canary ID:4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171 UID:b0793e26-e08f-46cd-bee3-f0ec4509d53a NetNS:/var/run/netns/bf1e2de8-c57d-477e-97f9-032a09eaee73 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:49:53.535485 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:53.527108622Z" level=info msg="Deleting pod openshift-ingress-canary_ingress-canary-c77h2 from CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:49:53.537705 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:53.537630608Z" level=info msg="Ran pod sandbox c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844 with infra container: openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj/POD" id=4d3e9fd8-b55d-44ba-a928-abc356748666 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:49:53.538810 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:53.538769717Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6a55bdcf28b6d2bfde87e747b11fbeb157a17ef0eef927b54bf69230f40f63ef" id=b7f9db26-8c8e-4b20-8e15-c62c811e5389 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:49:53.539087 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:53.539057219Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6a55bdcf28b6d2bfde87e747b11fbeb157a17ef0eef927b54bf69230f40f63ef not found" id=b7f9db26-8c8e-4b20-8e15-c62c811e5389 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:49:53.539614 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:53.539585 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 14:49:53.540295 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:53.540255529Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6a55bdcf28b6d2bfde87e747b11fbeb157a17ef0eef927b54bf69230f40f63ef" id=87368bef-6cc6-4922-8031-19e11b0fe41e name=/runtime.v1.ImageService/PullImage Nov 16 14:49:53.545264 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:53.545226269Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6a55bdcf28b6d2bfde87e747b11fbeb157a17ef0eef927b54bf69230f40f63ef\"" Nov 16 14:49:53.587574 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:53.587546 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c77h2_b0793e26-e08f-46cd-bee3-f0ec4509d53a/serve-healthcheck-canary/0.log" Nov 16 14:49:53.587714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:53.587603 1563 generic.go:296] "Generic (PLEG): container finished" podID=b0793e26-e08f-46cd-bee3-f0ec4509d53a containerID="7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4" exitCode=2 Nov 16 14:49:53.587714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:53.587667 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c77h2" event=&{ID:b0793e26-e08f-46cd-bee3-f0ec4509d53a Type:ContainerDied Data:7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4} Nov 16 14:49:53.588576 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:53.588544 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj" event=&{ID:35f6c830-e480-446f-a5cc-a6aaf007ea9a Type:ContainerStarted Data:c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844} Nov 16 14:49:53.601262 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00172|connmgr|INFO|br0<->unix#385: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:49:53.634229 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00173|connmgr|INFO|br0<->unix#388: 4 flow_mods in the last 0 s (4 deletes) Nov 16 14:49:53.665368 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00174|bridge|INFO|bridge br0: deleted interface vethb0219afb on port 4 Nov 16 14:49:53.671214 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethb0219afb left promiscuous mode Nov 16 14:49:53.679426 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:49:53Z [verbose] Del: openshift-ingress-canary:ingress-canary-c77h2:b0793e26-e08f-46cd-bee3-f0ec4509d53a:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 14:49:53.687535 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-bf1e2de8\x2dc57d\x2d477e\x2d97f9\x2d032a09eaee73.mount: Succeeded. Nov 16 14:49:53.688548 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-bf1e2de8\x2dc57d\x2d477e\x2d97f9\x2d032a09eaee73.mount: Consumed 0 CPU time Nov 16 14:49:53.695996 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-bf1e2de8\x2dc57d\x2d477e\x2d97f9\x2d032a09eaee73.mount: Succeeded. Nov 16 14:49:53.696822 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-bf1e2de8\x2dc57d\x2d477e\x2d97f9\x2d032a09eaee73.mount: Consumed 0 CPU time Nov 16 14:49:53.711060 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-bf1e2de8\x2dc57d\x2d477e\x2d97f9\x2d032a09eaee73.mount: Succeeded. Nov 16 14:49:53.711968 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-bf1e2de8\x2dc57d\x2d477e\x2d97f9\x2d032a09eaee73.mount: Consumed 0 CPU time Nov 16 14:49:53.911309 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:53.911162641Z" level=info msg="Stopped pod sandbox: 4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171" id=81e2bedf-3e48-4e23-b6c0-f01cfd012869 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:49:53.918160 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:53.918103 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c77h2_b0793e26-e08f-46cd-bee3-f0ec4509d53a/serve-healthcheck-canary/0.log" Nov 16 14:49:54.022383 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.022336 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4dwq\" (UniqueName: \"kubernetes.io/projected/b0793e26-e08f-46cd-bee3-f0ec4509d53a-kube-api-access-k4dwq\") pod \"b0793e26-e08f-46cd-bee3-f0ec4509d53a\" (UID: \"b0793e26-e08f-46cd-bee3-f0ec4509d53a\") " Nov 16 14:49:54.036797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.036708 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0793e26-e08f-46cd-bee3-f0ec4509d53a-kube-api-access-k4dwq" (OuterVolumeSpecName: "kube-api-access-k4dwq") pod "b0793e26-e08f-46cd-bee3-f0ec4509d53a" (UID: "b0793e26-e08f-46cd-bee3-f0ec4509d53a"). InnerVolumeSpecName "kube-api-access-k4dwq". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:49:54.123583 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.123489 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-k4dwq\" (UniqueName: \"kubernetes.io/projected/b0793e26-e08f-46cd-bee3-f0ec4509d53a-kube-api-access-k4dwq\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:49:54.183269 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-38dbe0e8b16f94c2918611549178d4edc4ad46169e45549bdaa870d560ff231c-merged.mount: Succeeded. Nov 16 14:49:54.185029 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-38dbe0e8b16f94c2918611549178d4edc4ad46169e45549bdaa870d560ff231c-merged.mount: Consumed 0 CPU time Nov 16 14:49:54.185088 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171-userdata-shm.mount: Succeeded. Nov 16 14:49:54.186628 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171-userdata-shm.mount: Consumed 0 CPU time Nov 16 14:49:54.186649 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-b0793e26\x2de08f\x2d46cd\x2dbee3\x2df0ec4509d53a-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dk4dwq.mount: Succeeded. Nov 16 14:49:54.187332 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-b0793e26\x2de08f\x2d46cd\x2dbee3\x2df0ec4509d53a-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dk4dwq.mount: Consumed 0 CPU time Nov 16 14:49:54.592307 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.592261 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c77h2_b0793e26-e08f-46cd-bee3-f0ec4509d53a/serve-healthcheck-canary/0.log" Nov 16 14:49:54.595282 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.595256 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c77h2" event=&{ID:b0793e26-e08f-46cd-bee3-f0ec4509d53a Type:ContainerDied Data:4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171} Nov 16 14:49:54.595375 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.595302 1563 scope.go:110] "RemoveContainer" containerID="7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4" Nov 16 14:49:54.596369 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:54.596326106Z" level=info msg="Removing container: 7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4" id=4667bb87-c75d-4738-96e6-10a2f66211ba name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:49:54.600194 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-podb0793e26_e08f_46cd_bee3_f0ec4509d53a.slice. Nov 16 14:49:54.600253 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-podb0793e26_e08f_46cd_bee3_f0ec4509d53a.slice: Consumed 1.366s CPU time Nov 16 14:49:54.633266 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:54.633208406Z" level=info msg="Removed container 7e5bf7ca0278c22ff8c4f5b251ad9b3aff1a1daee120f0f4196a1bd555e141b4: openshift-ingress-canary/ingress-canary-c77h2/serve-healthcheck-canary" id=4667bb87-c75d-4738-96e6-10a2f66211ba name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:49:54.644787 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.644712 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-ingress-canary/ingress-canary-c77h2] Nov 16 14:49:54.659042 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.659014 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-ingress-canary/ingress-canary-c77h2] Nov 16 14:49:54.695272 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.695230 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-ingress-canary/ingress-canary-4zcm6] Nov 16 14:49:54.695272 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.695286 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:49:54.695470 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:49:54.695382 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="b0793e26-e08f-46cd-bee3-f0ec4509d53a" containerName="serve-healthcheck-canary" Nov 16 14:49:54.695470 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.695397 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0793e26-e08f-46cd-bee3-f0ec4509d53a" containerName="serve-healthcheck-canary" Nov 16 14:49:54.695470 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.695454 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="b0793e26-e08f-46cd-bee3-f0ec4509d53a" containerName="serve-healthcheck-canary" Nov 16 14:49:54.705869 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-podb2808f6f_6018_4fb1_8229_65770d542c6d.slice. Nov 16 14:49:54.800303 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:54.800032361Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6a55bdcf28b6d2bfde87e747b11fbeb157a17ef0eef927b54bf69230f40f63ef\"" Nov 16 14:49:54.828747 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.828662 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wmqt\" (UniqueName: \"kubernetes.io/projected/b2808f6f-6018-4fb1-8229-65770d542c6d-kube-api-access-2wmqt\") pod \"ingress-canary-4zcm6\" (UID: \"b2808f6f-6018-4fb1-8229-65770d542c6d\") " pod="openshift-ingress-canary/ingress-canary-4zcm6" Nov 16 14:49:54.930220 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.929975 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-2wmqt\" (UniqueName: \"kubernetes.io/projected/b2808f6f-6018-4fb1-8229-65770d542c6d-kube-api-access-2wmqt\") pod \"ingress-canary-4zcm6\" (UID: \"b2808f6f-6018-4fb1-8229-65770d542c6d\") " pod="openshift-ingress-canary/ingress-canary-4zcm6" Nov 16 14:49:54.956060 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:54.955953 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wmqt\" (UniqueName: \"kubernetes.io/projected/b2808f6f-6018-4fb1-8229-65770d542c6d-kube-api-access-2wmqt\") pod \"ingress-canary-4zcm6\" (UID: \"b2808f6f-6018-4fb1-8229-65770d542c6d\") " pod="openshift-ingress-canary/ingress-canary-4zcm6" Nov 16 14:49:55.015901 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:55.015793 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4zcm6" Nov 16 14:49:55.016663 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:55.016562019Z" level=info msg="Running pod sandbox: openshift-ingress-canary/ingress-canary-4zcm6/POD" id=d4fbb93b-ab78-41a7-812f-abe1f0e615ff name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:49:55.016796 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:55.016670220Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:49:55.076732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:55.076637707Z" level=info msg="Got pod network &{Name:ingress-canary-4zcm6 Namespace:openshift-ingress-canary ID:e34494330a0e2782fede213cc688bbeb57241f90ee1bdd09069d1958581f8538 UID:b2808f6f-6018-4fb1-8229-65770d542c6d NetNS:/var/run/netns/e07dd781-a813-405f-8c4b-30b108c7bb79 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:49:55.076732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:55.076683108Z" level=info msg="Adding pod openshift-ingress-canary_ingress-canary-4zcm6 to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:49:55.123208 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:49:55.138711 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[174661]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:49:55.145034 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth88be9ad1: link is not ready Nov 16 14:49:55.145116 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth88be9ad1: link becomes ready Nov 16 14:49:55.145166 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:49:55.152283 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[174661]: Could not generate persistent MAC address for veth88be9ad1: No such file or directory Nov 16 14:49:55.152767 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610195.1527] device (veth88be9ad1): carrier: link connected Nov 16 14:49:55.153193 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610195.1531] manager: (veth88be9ad1): new Veth device (/org/freedesktop/NetworkManager/Devices/60) Nov 16 14:49:55.161390 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610195.1613] manager: (veth88be9ad1): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/61) Nov 16 14:49:55.167886 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00175|bridge|INFO|bridge br0: added interface veth88be9ad1 on port 28 Nov 16 14:49:55.168332 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth88be9ad1 entered promiscuous mode Nov 16 14:49:55.200758 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00176|connmgr|INFO|br0<->unix#391: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:49:55.245553 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00177|connmgr|INFO|br0<->unix#394: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:49:56.596585 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:56.596498 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=b0793e26-e08f-46cd-bee3-f0ec4509d53a path="/var/lib/kubelet/pods/b0793e26-e08f-46cd-bee3-f0ec4509d53a/volumes" Nov 16 14:49:57.087776 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:57.087736 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-ingress-canary/ingress-canary-4zcm6] Nov 16 14:49:57.092172 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:49:57Z [verbose] Add: openshift-ingress-canary:ingress-canary-4zcm6:b2808f6f-6018-4fb1-8229-65770d542c6d:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/e07dd781-a813-405f-8c4b-30b108c7bb79"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.27/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:49:57.092172 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:49:57.069999 174635 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-ingress-canary", Name:"ingress-canary-4zcm6", UID:"b2808f6f-6018-4fb1-8229-65770d542c6d", APIVersion:"v1", ResourceVersion:"93343", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.27/23] from openshift-sdn Nov 16 14:49:57.092705 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:57.092647481Z" level=info msg="Got pod network &{Name:ingress-canary-4zcm6 Namespace:openshift-ingress-canary ID:e34494330a0e2782fede213cc688bbeb57241f90ee1bdd09069d1958581f8538 UID:b2808f6f-6018-4fb1-8229-65770d542c6d NetNS:/var/run/netns/e07dd781-a813-405f-8c4b-30b108c7bb79 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:49:57.092934 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:57.092807983Z" level=info msg="Checking pod openshift-ingress-canary_ingress-canary-4zcm6 for CNI network multus-cni-network (type=multus)" Nov 16 14:49:57.095805 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:49:57.095766 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2808f6f_6018_4fb1_8229_65770d542c6d.slice/crio-e34494330a0e2782fede213cc688bbeb57241f90ee1bdd09069d1958581f8538.scope WatchSource:0}: Error finding container e34494330a0e2782fede213cc688bbeb57241f90ee1bdd09069d1958581f8538: Status 404 returned error &{%!s(*http.body=&{0xc002cd8360 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:49:57.101495 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:57.101457153Z" level=info msg="Ran pod sandbox e34494330a0e2782fede213cc688bbeb57241f90ee1bdd09069d1958581f8538 with infra container: openshift-ingress-canary/ingress-canary-4zcm6/POD" id=d4fbb93b-ab78-41a7-812f-abe1f0e615ff name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:49:57.102993 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:57.102635862Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3740f6d682f8e1bcb6e8b79363293362ea4125939ced14931480a3f765682a1" id=d4dbce64-5e32-4ae0-9c1d-66e2c6e6b15c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:49:57.102993 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:57.102915565Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3740f6d682f8e1bcb6e8b79363293362ea4125939ced14931480a3f765682a1 not found" id=d4dbce64-5e32-4ae0-9c1d-66e2c6e6b15c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:49:57.104056 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:57.103660771Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3740f6d682f8e1bcb6e8b79363293362ea4125939ced14931480a3f765682a1" id=307d2a63-b6ae-4a50-be51-2047122152ec name=/runtime.v1.ImageService/PullImage Nov 16 14:49:57.110143 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:57.109843521Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3740f6d682f8e1bcb6e8b79363293362ea4125939ced14931480a3f765682a1\"" Nov 16 14:49:57.602320 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:49:57.602288 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4zcm6" event=&{ID:b2808f6f-6018-4fb1-8229-65770d542c6d Type:ContainerStarted Data:e34494330a0e2782fede213cc688bbeb57241f90ee1bdd09069d1958581f8538} Nov 16 14:49:58.350972 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:49:58.350877641Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3740f6d682f8e1bcb6e8b79363293362ea4125939ced14931480a3f765682a1\"" Nov 16 14:50:00.083766 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:00.083713 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-fh97d] Nov 16 14:50:00.083766 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:00.083788 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:50:00.094257 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod590da926_dc2c_4378_a471_69dd415ae65f.slice. Nov 16 14:50:00.174335 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:00.174281 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4srsm\" (UniqueName: \"kubernetes.io/projected/590da926-dc2c-4378-a471-69dd415ae65f-kube-api-access-4srsm\") pod \"qe-app-registry-fh97d\" (UID: \"590da926-dc2c-4378-a471-69dd415ae65f\") " pod="openshift-marketplace/qe-app-registry-fh97d" Nov 16 14:50:00.275746 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:00.275704 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-4srsm\" (UniqueName: \"kubernetes.io/projected/590da926-dc2c-4378-a471-69dd415ae65f-kube-api-access-4srsm\") pod \"qe-app-registry-fh97d\" (UID: \"590da926-dc2c-4378-a471-69dd415ae65f\") " pod="openshift-marketplace/qe-app-registry-fh97d" Nov 16 14:50:00.300357 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:00.300321 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-4srsm\" (UniqueName: \"kubernetes.io/projected/590da926-dc2c-4378-a471-69dd415ae65f-kube-api-access-4srsm\") pod \"qe-app-registry-fh97d\" (UID: \"590da926-dc2c-4378-a471-69dd415ae65f\") " pod="openshift-marketplace/qe-app-registry-fh97d" Nov 16 14:50:00.407937 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:00.407891 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-fh97d" Nov 16 14:50:00.408735 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:00.408610083Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-fh97d/POD" id=44887f7e-f154-4ec2-acbd-0c93b66bc3cc name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:00.408735 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:00.408728484Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:04.660016 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:04.659926473Z" level=info msg="Got pod network &{Name:qe-app-registry-fh97d Namespace:openshift-marketplace ID:eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5 UID:590da926-dc2c-4378-a471-69dd415ae65f NetNS:/var/run/netns/1e5df66f-d601-4db1-9601-458acfac495e Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:04.660016 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:04.659971574Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-fh97d to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:50:04.705167 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:50:04.720445 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610204.7204] manager: (veth6aafb6eb): new Veth device (/org/freedesktop/NetworkManager/Devices/62) Nov 16 14:50:04.721252 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610204.7212] device (veth6aafb6eb): carrier: link connected Nov 16 14:50:04.727510 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[174957]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:50:04.734673 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth6aafb6eb: link is not ready Nov 16 14:50:04.734750 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth6aafb6eb: link becomes ready Nov 16 14:50:04.734785 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:50:04.742418 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[174957]: Could not generate persistent MAC address for veth6aafb6eb: No such file or directory Nov 16 14:50:04.759977 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610204.7599] manager: (veth6aafb6eb): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/63) Nov 16 14:50:04.764791 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00178|bridge|INFO|bridge br0: added interface veth6aafb6eb on port 29 Nov 16 14:50:04.765199 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth6aafb6eb entered promiscuous mode Nov 16 14:50:04.800376 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00179|connmgr|INFO|br0<->unix#397: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:50:04.850272 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00180|connmgr|INFO|br0<->unix#400: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:50:05.308955 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.3XgehT.mount: Succeeded. Nov 16 14:50:05.512093 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.512028185Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6a55bdcf28b6d2bfde87e747b11fbeb157a17ef0eef927b54bf69230f40f63ef" id=87368bef-6cc6-4922-8031-19e11b0fe41e name=/runtime.v1.ImageService/PullImage Nov 16 14:50:05.513097 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.513048192Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6a55bdcf28b6d2bfde87e747b11fbeb157a17ef0eef927b54bf69230f40f63ef" id=7dd36939-9e84-4328-884b-e7d998d61204 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:05.514745 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.514653004Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:fbf33148eb435c1f3d579a533fba721faecf3d2e1c739b13111fb0a0f280c1f9,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6a55bdcf28b6d2bfde87e747b11fbeb157a17ef0eef927b54bf69230f40f63ef],Size_:372809554,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=7dd36939-9e84-4328-884b-e7d998d61204 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:05.515830 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.515787813Z" level=info msg="Creating container: openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj/prometheus-operator-admission-webhook" id=a8634577-39ff-43d1-aab6-5bb7c8880efc name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:05.515922 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.515905113Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:05.516107 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.516067515Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3740f6d682f8e1bcb6e8b79363293362ea4125939ced14931480a3f765682a1" id=307d2a63-b6ae-4a50-be51-2047122152ec name=/runtime.v1.ImageService/PullImage Nov 16 14:50:05.516774 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.516738920Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3740f6d682f8e1bcb6e8b79363293362ea4125939ced14931480a3f765682a1" id=4d45fe73-5382-4a34-9da6-6e3e57f07e31 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:05.518778 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.518702034Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5ec3d62c0ae86da5ddb6ba8c0c9d83bb438afa7c92623daad942c578e9fb9512,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3740f6d682f8e1bcb6e8b79363293362ea4125939ced14931480a3f765682a1],Size_:415346122,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=4d45fe73-5382-4a34-9da6-6e3e57f07e31 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:05.519650 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.519578941Z" level=info msg="Creating container: openshift-ingress-canary/ingress-canary-4zcm6/serve-healthcheck-canary" id=610fbcec-81af-4156-8028-002ec9386d34 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:05.519740 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.519669441Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:05.577258 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778.scope. Nov 16 14:50:05.619974 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-64bed0c179a04023d40529cb2776101cbe84d828926a82d36a0dfa28e6017e17.scope. Nov 16 14:50:05.622175 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778. Nov 16 14:50:05.642676 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 64bed0c179a04023d40529cb2776101cbe84d828926a82d36a0dfa28e6017e17. Nov 16 14:50:05.655532 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:50:05.681283 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:50:05.755874 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.755661889Z" level=info msg="Created container cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778: openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj/prometheus-operator-admission-webhook" id=a8634577-39ff-43d1-aab6-5bb7c8880efc name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:05.756399 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.756346294Z" level=info msg="Starting container: cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778" id=304f0e69-31bf-4f1c-ac6d-89b07c8dd47d name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:50:05.768821 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.768761286Z" level=info msg="Started container" PID=175052 containerID=cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778 description=openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj/prometheus-operator-admission-webhook id=304f0e69-31bf-4f1c-ac6d-89b07c8dd47d name=/runtime.v1.RuntimeService/StartContainer sandboxID=c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844 Nov 16 14:50:05.791785 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.791661956Z" level=info msg="Created container 64bed0c179a04023d40529cb2776101cbe84d828926a82d36a0dfa28e6017e17: openshift-ingress-canary/ingress-canary-4zcm6/serve-healthcheck-canary" id=610fbcec-81af-4156-8028-002ec9386d34 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:05.792351 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.792312661Z" level=info msg="Starting container: 64bed0c179a04023d40529cb2776101cbe84d828926a82d36a0dfa28e6017e17" id=2fe95a45-1d6c-4ac2-837f-bc2272cfd8c5 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:50:05.804948 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:05.804895854Z" level=info msg="Started container" PID=175063 containerID=64bed0c179a04023d40529cb2776101cbe84d828926a82d36a0dfa28e6017e17 description=openshift-ingress-canary/ingress-canary-4zcm6/serve-healthcheck-canary id=2fe95a45-1d6c-4ac2-837f-bc2272cfd8c5 name=/runtime.v1.RuntimeService/StartContainer sandboxID=e34494330a0e2782fede213cc688bbeb57241f90ee1bdd09069d1958581f8538 Nov 16 14:50:06.625482 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:06.625442 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj" event=&{ID:35f6c830-e480-446f-a5cc-a6aaf007ea9a Type:ContainerStarted Data:cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778} Nov 16 14:50:06.625902 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:06.625695 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj" Nov 16 14:50:06.628564 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:06.628525 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4zcm6" event=&{ID:b2808f6f-6018-4fb1-8229-65770d542c6d Type:ContainerStarted Data:64bed0c179a04023d40529cb2776101cbe84d828926a82d36a0dfa28e6017e17} Nov 16 14:50:06.630834 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:06.630805 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj" Nov 16 14:50:06.726161 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:06.724844 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-marketplace/qe-app-registry-fh97d] Nov 16 14:50:06.727649 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:50:06Z [verbose] Add: openshift-marketplace:qe-app-registry-fh97d:590da926-dc2c-4378-a471-69dd415ae65f:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/1e5df66f-d601-4db1-9601-458acfac495e"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.28/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:50:06.727649 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:50:06.692177 174934 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-fh97d", UID:"590da926-dc2c-4378-a471-69dd415ae65f", APIVersion:"v1", ResourceVersion:"93710", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.28/23] from openshift-sdn Nov 16 14:50:06.728396 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:06.728259385Z" level=info msg="Got pod network &{Name:qe-app-registry-fh97d Namespace:openshift-marketplace ID:eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5 UID:590da926-dc2c-4378-a471-69dd415ae65f NetNS:/var/run/netns/1e5df66f-d601-4db1-9601-458acfac495e Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:06.728515 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:06.728409886Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-fh97d for CNI network multus-cni-network (type=multus)" Nov 16 14:50:06.730759 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:06.730718 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod590da926_dc2c_4378_a471_69dd415ae65f.slice/crio-eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5.scope WatchSource:0}: Error finding container eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5: Status 404 returned error &{%!s(*http.body=&{0xc001330198 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:50:06.738850 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:06.738766162Z" level=info msg="Ran pod sandbox eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5 with infra container: openshift-marketplace/qe-app-registry-fh97d/POD" id=44887f7e-f154-4ec2-acbd-0c93b66bc3cc name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:06.740197 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:06.740084872Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=36cfc089-3bdb-4bdf-9f78-59c31b912f40 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:06.740395 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:06.740355874Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=36cfc089-3bdb-4bdf-9f78-59c31b912f40 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:06.741310 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:06.741272281Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=5fce63b7-58d7-49f7-9b13-716931851beb name=/runtime.v1.ImageService/PullImage Nov 16 14:50:06.744482 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:06.744450004Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 14:50:07.632116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:07.632044 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-fh97d" event=&{ID:590da926-dc2c-4378-a471-69dd415ae65f Type:ContainerStarted Data:eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5} Nov 16 14:50:08.376278 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:08.376217066Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=5fce63b7-58d7-49f7-9b13-716931851beb name=/runtime.v1.ImageService/PullImage Nov 16 14:50:08.377247 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:08.377207374Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=0483c4e2-9937-4155-9238-9c59886b995a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:08.377457 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:08.377416875Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=0483c4e2-9937-4155-9238-9c59886b995a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:08.378362 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:08.378323182Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-fh97d/registry-server" id=44e2c537-6d88-4b7d-9320-69402b4a33bc name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:08.378448 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:08.378428083Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:08.439465 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f.scope. Nov 16 14:50:08.469084 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f. Nov 16 14:50:08.508381 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:50:09.460914 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:09.460802884Z" level=info msg="Created container 9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f: openshift-marketplace/qe-app-registry-fh97d/registry-server" id=44e2c537-6d88-4b7d-9320-69402b4a33bc name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:09.461522 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:09.461488489Z" level=info msg="Starting container: 9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f" id=7c0527ab-43f9-4dc9-b33d-639a0bb3942b name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:50:09.474090 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:09.474043981Z" level=info msg="Started container" PID=175326 containerID=9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f description=openshift-marketplace/qe-app-registry-fh97d/registry-server id=7c0527ab-43f9-4dc9-b33d-639a0bb3942b name=/runtime.v1.RuntimeService/StartContainer sandboxID=eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5 Nov 16 14:50:09.638847 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:09.638808 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-fh97d" event=&{ID:590da926-dc2c-4378-a471-69dd415ae65f Type:ContainerStarted Data:9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f} Nov 16 14:50:10.310373 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.1ogEJB.mount: Succeeded. Nov 16 14:50:10.408753 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:10.408620 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/qe-app-registry-fh97d" Nov 16 14:50:10.408753 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:10.408709 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/qe-app-registry-fh97d" Nov 16 14:50:10.470055 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:10.470017 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/qe-app-registry-fh97d" Nov 16 14:50:20.471551 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:20.471476 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/qe-app-registry-fh97d" Nov 16 14:50:21.696638 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:21.696581980Z" level=info msg="Stopping pod sandbox: 4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171" id=aabb8385-1777-4933-8d9f-6ab1adf7fabc name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:21.696638 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:21.696639680Z" level=info msg="Stopped pod sandbox (already stopped): 4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171" id=aabb8385-1777-4933-8d9f-6ab1adf7fabc name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:21.697097 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:21.696914382Z" level=info msg="Removing pod sandbox: 4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171" id=c97e983b-821e-4bc9-8114-f7cc820347b2 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:50:21.714005 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:21.713964508Z" level=info msg="Removed pod sandbox: 4cac105b51ec5b8aa9cf5ab95507f697f0bd48508c67ff8b98f97177ff95b171" id=c97e983b-821e-4bc9-8114-f7cc820347b2 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:50:26.992937 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:26.990165 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-monitoring/kube-state-metrics-68667ff848-f92nw] Nov 16 14:50:26.992937 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:26.990237 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:50:26.992937 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:26.990697 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv] Nov 16 14:50:26.992937 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:26.990727 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:50:27.002577 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-podc8013b4f_15b6_4bac_bfc9_a45301362f0c.slice. Nov 16 14:50:27.017719 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod454b3a37_199c_4abc_a79a_8de93eab2423.slice. Nov 16 14:50:27.095403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.095331 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8013b4f-15b6-4bac-bfc9-a45301362f0c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6959bf55f6-ppwrv\" (UID: \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\") " pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" Nov 16 14:50:27.095635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.095506 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/454b3a37-199c-4abc-a79a-8de93eab2423-metrics-client-ca\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.095635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.095542 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/454b3a37-199c-4abc-a79a-8de93eab2423-kube-state-metrics-tls\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.095635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.095576 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8z4g\" (UniqueName: \"kubernetes.io/projected/c8013b4f-15b6-4bac-bfc9-a45301362f0c-kube-api-access-b8z4g\") pod \"openshift-state-metrics-6959bf55f6-ppwrv\" (UID: \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\") " pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" Nov 16 14:50:27.095635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.095615 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/454b3a37-199c-4abc-a79a-8de93eab2423-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.095786 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.095747 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/454b3a37-199c-4abc-a79a-8de93eab2423-volume-directive-shadow\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.095825 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.095806 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8013b4f-15b6-4bac-bfc9-a45301362f0c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6959bf55f6-ppwrv\" (UID: \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\") " pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" Nov 16 14:50:27.095863 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.095853 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcfpr\" (UniqueName: \"kubernetes.io/projected/454b3a37-199c-4abc-a79a-8de93eab2423-kube-api-access-vcfpr\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.172246 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.172140 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-monitoring/telemeter-client-55c5556665-7vs77] Nov 16 14:50:27.172246 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.172248 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:50:27.186070 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-podadd06140_663d_48c0_ac00_643bd58a3f7d.slice. Nov 16 14:50:27.196330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196293 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/454b3a37-199c-4abc-a79a-8de93eab2423-metrics-client-ca\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.196625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196358 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/454b3a37-199c-4abc-a79a-8de93eab2423-kube-state-metrics-tls\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.196625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196398 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-b8z4g\" (UniqueName: \"kubernetes.io/projected/c8013b4f-15b6-4bac-bfc9-a45301362f0c-kube-api-access-b8z4g\") pod \"openshift-state-metrics-6959bf55f6-ppwrv\" (UID: \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\") " pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" Nov 16 14:50:27.196625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196431 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.196625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196474 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/454b3a37-199c-4abc-a79a-8de93eab2423-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.196625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196507 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-serving-certs-ca-bundle\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.196625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196533 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-secret-telemeter-client\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.196625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196566 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-metrics-client-ca\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.196625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196594 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-547nc\" (UniqueName: \"kubernetes.io/projected/add06140-663d-48c0-ac00-643bd58a3f7d-kube-api-access-547nc\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.196625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196624 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-telemeter-client-tls\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.196895 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196657 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/454b3a37-199c-4abc-a79a-8de93eab2423-volume-directive-shadow\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.196895 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196692 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.196895 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196755 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8013b4f-15b6-4bac-bfc9-a45301362f0c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6959bf55f6-ppwrv\" (UID: \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\") " pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" Nov 16 14:50:27.196895 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196809 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-vcfpr\" (UniqueName: \"kubernetes.io/projected/454b3a37-199c-4abc-a79a-8de93eab2423-kube-api-access-vcfpr\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.196895 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.196846 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8013b4f-15b6-4bac-bfc9-a45301362f0c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6959bf55f6-ppwrv\" (UID: \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\") " pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" Nov 16 14:50:27.197619 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.197331 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/454b3a37-199c-4abc-a79a-8de93eab2423-volume-directive-shadow\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.197619 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.197575 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/454b3a37-199c-4abc-a79a-8de93eab2423-metrics-client-ca\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.199191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.199166 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/454b3a37-199c-4abc-a79a-8de93eab2423-kube-state-metrics-tls\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.199637 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.199612 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/454b3a37-199c-4abc-a79a-8de93eab2423-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.199840 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.199760 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8013b4f-15b6-4bac-bfc9-a45301362f0c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6959bf55f6-ppwrv\" (UID: \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\") " pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" Nov 16 14:50:27.201356 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.201328 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8013b4f-15b6-4bac-bfc9-a45301362f0c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6959bf55f6-ppwrv\" (UID: \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\") " pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" Nov 16 14:50:27.218332 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.218260 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8z4g\" (UniqueName: \"kubernetes.io/projected/c8013b4f-15b6-4bac-bfc9-a45301362f0c-kube-api-access-b8z4g\") pod \"openshift-state-metrics-6959bf55f6-ppwrv\" (UID: \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\") " pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" Nov 16 14:50:27.219054 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.218994 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcfpr\" (UniqueName: \"kubernetes.io/projected/454b3a37-199c-4abc-a79a-8de93eab2423-kube-api-access-vcfpr\") pod \"kube-state-metrics-68667ff848-f92nw\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.298369 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.298180 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-547nc\" (UniqueName: \"kubernetes.io/projected/add06140-663d-48c0-ac00-643bd58a3f7d-kube-api-access-547nc\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.298369 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.298315 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-telemeter-client-tls\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.298369 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.298371 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.298623 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.298429 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.298623 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.298472 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-serving-certs-ca-bundle\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.298623 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.298505 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-secret-telemeter-client\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.298623 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.298556 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-metrics-client-ca\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.299414 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.299349 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-serving-certs-ca-bundle\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.299414 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.299361 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-metrics-client-ca\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.299821 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.299790 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.301611 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.301543 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-secret-telemeter-client\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.301611 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.301609 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-telemeter-client-tls\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.302621 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.302559 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.315423 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.315389 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" Nov 16 14:50:27.316062 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.315984771Z" level=info msg="Running pod sandbox: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/POD" id=6cf31962-4f8b-4119-9388-b9ed4f211ed1 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:27.316501 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.316076672Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:27.322544 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.322501 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-547nc\" (UniqueName: \"kubernetes.io/projected/add06140-663d-48c0-ac00-643bd58a3f7d-kube-api-access-547nc\") pod \"telemeter-client-55c5556665-7vs77\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.322671 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.322645 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" Nov 16 14:50:27.323197 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.323083423Z" level=info msg="Running pod sandbox: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/POD" id=239f6b6c-1de3-445d-b3ac-fc3b628fc4ff name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:27.323197 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.323199324Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:27.376163 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.376075415Z" level=info msg="Got pod network &{Name:openshift-state-metrics-6959bf55f6-ppwrv Namespace:openshift-monitoring ID:9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c UID:c8013b4f-15b6-4bac-bfc9-a45301362f0c NetNS:/var/run/netns/7ac5f633-b1e2-412c-8e55-6972cfdae313 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:27.376163 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.376180915Z" level=info msg="Adding pod openshift-monitoring_openshift-state-metrics-6959bf55f6-ppwrv to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:50:27.397905 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.397820 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx] Nov 16 14:50:27.397905 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.397895 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:50:27.408737 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod17691c72_c4d7_4e7f_b345_a0447ae7d4c6.slice. Nov 16 14:50:27.433280 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:50:27.446906 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.446858037Z" level=info msg="Got pod network &{Name:kube-state-metrics-68667ff848-f92nw Namespace:openshift-monitoring ID:5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3 UID:454b3a37-199c-4abc-a79a-8de93eab2423 NetNS:/var/run/netns/e936529a-7e5f-4ca3-b47c-4441e6fa4a10 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:27.446906 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.446908638Z" level=info msg="Adding pod openshift-monitoring_kube-state-metrics-68667ff848-f92nw to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:50:27.447210 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610227.4471] manager: (veth36d169af): new Veth device (/org/freedesktop/NetworkManager/Devices/64) Nov 16 14:50:27.448111 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610227.4480] device (veth36d169af): carrier: link connected Nov 16 14:50:27.452752 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[176126]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:50:27.458709 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth36d169af: link is not ready Nov 16 14:50:27.458786 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth36d169af: link becomes ready Nov 16 14:50:27.458814 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:50:27.464536 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[176126]: Could not generate persistent MAC address for veth36d169af: No such file or directory Nov 16 14:50:27.486724 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610227.4866] manager: (veth36d169af): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/65) Nov 16 14:50:27.492204 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:50:27.492290 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth36d169af entered promiscuous mode Nov 16 14:50:27.497637 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.497559 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" Nov 16 14:50:27.498158 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00181|bridge|INFO|bridge br0: added interface veth36d169af on port 30 Nov 16 14:50:27.498645 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.498256917Z" level=info msg="Running pod sandbox: openshift-monitoring/telemeter-client-55c5556665-7vs77/POD" id=91649164-7ab2-4774-a2d9-b35add341481 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:27.498645 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.498435918Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:27.499769 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.499739 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-adapter-prometheus-config\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-prometheus-adapter-prometheus-config\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.499887 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.499807 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-adapter-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-prometheus-adapter-audit-profiles\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.499887 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.499856 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-serving-certs-ca-bundle\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.499971 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.499897 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-config\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.499971 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.499927 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-tls\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.499971 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.499961 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6skm9\" (UniqueName: \"kubernetes.io/projected/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-kube-api-access-6skm9\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.500095 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.500058 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-audit-log\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.500173 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.500120 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmpfs\" (UniqueName: \"kubernetes.io/empty-dir/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-tmpfs\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.508156 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610227.5081] manager: (vethb0280b20): new Veth device (/org/freedesktop/NetworkManager/Devices/66) Nov 16 14:50:27.509224 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610227.5092] device (vethb0280b20): carrier: link connected Nov 16 14:50:27.515366 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[176153]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:50:27.515554 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[176153]: Could not generate persistent MAC address for vethb0280b20: No such file or directory Nov 16 14:50:27.516159 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:50:27.543173 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00182|connmgr|INFO|br0<->unix#403: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:50:27.551709 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.551533510Z" level=info msg="Got pod network &{Name:telemeter-client-55c5556665-7vs77 Namespace:openshift-monitoring ID:a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145 UID:add06140-663d-48c0-ac00-643bd58a3f7d NetNS:/var/run/netns/52f1a65d-b622-4276-b4a0-e16b4f43bd50 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:27.551709 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.551576310Z" level=info msg="Adding pod openshift-monitoring_telemeter-client-55c5556665-7vs77 to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:50:27.593243 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00183|connmgr|INFO|br0<->unix#406: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:50:27.600955 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.600895 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-audit-log\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.601170 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.601031 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"tmpfs\" (UniqueName: \"kubernetes.io/empty-dir/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-tmpfs\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.601170 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.601069 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"prometheus-adapter-prometheus-config\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-prometheus-adapter-prometheus-config\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.601170 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.601146 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"prometheus-adapter-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-prometheus-adapter-audit-profiles\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.601282 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.601188 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-serving-certs-ca-bundle\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.601282 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.601223 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-config\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.601282 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.601251 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-tls\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.601375 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.601299 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-6skm9\" (UniqueName: \"kubernetes.io/projected/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-kube-api-access-6skm9\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.601455 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.601427 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-audit-log\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.601455 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.601447 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"tmpfs\" (UniqueName: \"kubernetes.io/empty-dir/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-tmpfs\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.602116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.602083 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-config\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.602116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.602089 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-serving-certs-ca-bundle\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.602279 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:50:27.602684 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.602591 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"prometheus-adapter-prometheus-config\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-prometheus-adapter-prometheus-config\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.602963 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.602937 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"prometheus-adapter-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-prometheus-adapter-audit-profiles\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.605223 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.605185 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"tls\" (UniqueName: \"kubernetes.io/secret/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-tls\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.613122 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610227.6130] manager: (veth442b6033): new Veth device (/org/freedesktop/NetworkManager/Devices/67) Nov 16 14:50:27.614003 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610227.6139] device (veth442b6033): carrier: link connected Nov 16 14:50:27.619913 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[176192]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:50:27.620304 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:50:27.620093 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[176192]: Could not generate persistent MAC address for veth442b6033: No such file or directory Nov 16 14:50:27.621004 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610227.6209] manager: (vethb0280b20): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/68) Nov 16 14:50:27.629157 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethb0280b20 entered promiscuous mode Nov 16 14:50:27.628726 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00184|bridge|INFO|bridge br0: added interface vethb0280b20 on port 31 Nov 16 14:50:27.632383 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.632335 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-6skm9\" (UniqueName: \"kubernetes.io/projected/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-kube-api-access-6skm9\") pod \"prometheus-adapter-6fb766b4f6-jcwvx\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.663459 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00185|connmgr|INFO|br0<->unix#409: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:50:27.709972 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00186|connmgr|INFO|br0<->unix#412: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:50:27.725495 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:27.725336 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:27.726093 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.725947398Z" level=info msg="Running pod sandbox: openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx/POD" id=a668b082-68dd-4f61-b4ea-99439335f0ce name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:27.726222 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.726142699Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:27.735100 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610227.7350] manager: (veth442b6033): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/69) Nov 16 14:50:27.739421 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00187|bridge|INFO|bridge br0: added interface veth442b6033 on port 32 Nov 16 14:50:27.740212 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth442b6033 entered promiscuous mode Nov 16 14:50:27.772492 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00188|connmgr|INFO|br0<->unix#415: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:50:27.785641 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.785594738Z" level=info msg="Got pod network &{Name:prometheus-adapter-6fb766b4f6-jcwvx Namespace:openshift-monitoring ID:9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486 UID:17691c72-c4d7-4e7f-b345-a0447ae7d4c6 NetNS:/var/run/netns/e966bd8c-2d2d-4176-8a8d-7d70130b4c3b Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:27.785641 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:27.785641839Z" level=info msg="Adding pod openshift-monitoring_prometheus-adapter-6fb766b4f6-jcwvx to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:50:27.819752 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00189|connmgr|INFO|br0<->unix#418: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:50:27.828153 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:50:27.840448 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610227.8404] manager: (veth74ac3f97): new Veth device (/org/freedesktop/NetworkManager/Devices/70) Nov 16 14:50:27.841079 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[176237]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:50:27.841338 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[176237]: Could not generate persistent MAC address for veth74ac3f97: No such file or directory Nov 16 14:50:27.841461 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610227.8414] device (veth74ac3f97): carrier: link connected Nov 16 14:50:27.847163 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:50:27.862838 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610227.8628] manager: (veth74ac3f97): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/71) Nov 16 14:50:27.869002 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00190|bridge|INFO|bridge br0: added interface veth74ac3f97 on port 33 Nov 16 14:50:27.869188 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth74ac3f97 entered promiscuous mode Nov 16 14:50:27.902488 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00191|connmgr|INFO|br0<->unix#421: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:50:27.951175 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00192|connmgr|INFO|br0<->unix#424: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:50:28.825652 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:50:28Z [verbose] Add: openshift-monitoring:kube-state-metrics-68667ff848-f92nw:454b3a37-199c-4abc-a79a-8de93eab2423:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/e936529a-7e5f-4ca3-b47c-4441e6fa4a10"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.30/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:50:28.825652 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:50:28.802583 176127 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-monitoring", Name:"kube-state-metrics-68667ff848-f92nw", UID:"454b3a37-199c-4abc-a79a-8de93eab2423", APIVersion:"v1", ResourceVersion:"95238", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.30/23] from openshift-sdn Nov 16 14:50:28.826354 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.826241422Z" level=info msg="Got pod network &{Name:kube-state-metrics-68667ff848-f92nw Namespace:openshift-monitoring ID:5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3 UID:454b3a37-199c-4abc-a79a-8de93eab2423 NetNS:/var/run/netns/e936529a-7e5f-4ca3-b47c-4441e6fa4a10 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:28.826465 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.826395523Z" level=info msg="Checking pod openshift-monitoring_kube-state-metrics-68667ff848-f92nw for CNI network multus-cni-network (type=multus)" Nov 16 14:50:28.827205 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:28.827122 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-monitoring/kube-state-metrics-68667ff848-f92nw] Nov 16 14:50:28.829076 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:28.829014 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod454b3a37_199c_4abc_a79a_8de93eab2423.slice/crio-5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3.scope WatchSource:0}: Error finding container 5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3: Status 404 returned error &{%!s(*http.body=&{0xc002ce5410 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:50:28.840679 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.840625928Z" level=info msg="Ran pod sandbox 5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3 with infra container: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/POD" id=239f6b6c-1de3-445d-b3ac-fc3b628fc4ff name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:28.841842 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.841798537Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3507212f8cf79da97eb88e4456049226704313f07aa9e14726e7a04691cad554" id=b229706a-b324-4686-b45d-b51c9d967848 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:28.842115 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.842083339Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3507212f8cf79da97eb88e4456049226704313f07aa9e14726e7a04691cad554 not found" id=b229706a-b324-4686-b45d-b51c9d967848 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:28.842956 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.842861244Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3507212f8cf79da97eb88e4456049226704313f07aa9e14726e7a04691cad554" id=2e95c021-d8d0-475c-b33d-8246314e2fd1 name=/runtime.v1.ImageService/PullImage Nov 16 14:50:28.846092 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.846007268Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3507212f8cf79da97eb88e4456049226704313f07aa9e14726e7a04691cad554\"" Nov 16 14:50:28.952613 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:50:28Z [verbose] Add: openshift-monitoring:telemeter-client-55c5556665-7vs77:add06140-663d-48c0-ac00-643bd58a3f7d:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/52f1a65d-b622-4276-b4a0-e16b4f43bd50"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.31/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:50:28.952613 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:50:28.923161 176163 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-monitoring", Name:"telemeter-client-55c5556665-7vs77", UID:"add06140-663d-48c0-ac00-643bd58a3f7d", APIVersion:"v1", ResourceVersion:"95261", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.31/23] from openshift-sdn Nov 16 14:50:28.952823 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.952749756Z" level=info msg="Got pod network &{Name:telemeter-client-55c5556665-7vs77 Namespace:openshift-monitoring ID:a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145 UID:add06140-663d-48c0-ac00-643bd58a3f7d NetNS:/var/run/netns/52f1a65d-b622-4276-b4a0-e16b4f43bd50 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:28.952921 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.952884057Z" level=info msg="Checking pod openshift-monitoring_telemeter-client-55c5556665-7vs77 for CNI network multus-cni-network (type=multus)" Nov 16 14:50:28.954141 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:28.954099 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-monitoring/telemeter-client-55c5556665-7vs77] Nov 16 14:50:28.962249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:28.962194 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd06140_663d_48c0_ac00_643bd58a3f7d.slice/crio-a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145.scope WatchSource:0}: Error finding container a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145: Status 404 returned error &{%!s(*http.body=&{0xc002ce5458 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:50:28.963872 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.963804237Z" level=info msg="Ran pod sandbox a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145 with infra container: openshift-monitoring/telemeter-client-55c5556665-7vs77/POD" id=91649164-7ab2-4774-a2d9-b35add341481 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:28.964955 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.964915846Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:119575c2aa477a5216fc18144f23ff48666e950ad7bca70d13af93cc6f2a54d2" id=d081f3fc-32be-4d87-a8aa-19ebce265307 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:28.965234 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.965203848Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:119575c2aa477a5216fc18144f23ff48666e950ad7bca70d13af93cc6f2a54d2 not found" id=d081f3fc-32be-4d87-a8aa-19ebce265307 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:28.966085 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.966043654Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:119575c2aa477a5216fc18144f23ff48666e950ad7bca70d13af93cc6f2a54d2" id=8e425a7d-8945-4c40-95d9-d6710ca0a0b4 name=/runtime.v1.ImageService/PullImage Nov 16 14:50:28.972583 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:28.972550302Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:119575c2aa477a5216fc18144f23ff48666e950ad7bca70d13af93cc6f2a54d2\"" Nov 16 14:50:29.134373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:50:29Z [verbose] Add: openshift-monitoring:openshift-state-metrics-6959bf55f6-ppwrv:c8013b4f-15b6-4bac-bfc9-a45301362f0c:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/7ac5f633-b1e2-412c-8e55-6972cfdae313"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.29/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:50:29.134373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:50:29.086889 176107 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-monitoring", Name:"openshift-state-metrics-6959bf55f6-ppwrv", UID:"c8013b4f-15b6-4bac-bfc9-a45301362f0c", APIVersion:"v1", ResourceVersion:"95225", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.29/23] from openshift-sdn Nov 16 14:50:29.135107 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.135039002Z" level=info msg="Got pod network &{Name:openshift-state-metrics-6959bf55f6-ppwrv Namespace:openshift-monitoring ID:9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c UID:c8013b4f-15b6-4bac-bfc9-a45301362f0c NetNS:/var/run/netns/7ac5f633-b1e2-412c-8e55-6972cfdae313 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:29.135267 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.135224803Z" level=info msg="Checking pod openshift-monitoring_openshift-state-metrics-6959bf55f6-ppwrv for CNI network multus-cni-network (type=multus)" Nov 16 14:50:29.138058 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:29.138015 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8013b4f_15b6_4bac_bfc9_a45301362f0c.slice/crio-9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c.scope WatchSource:0}: Error finding container 9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c: Status 404 returned error &{%!s(*http.body=&{0xc0039c53f8 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:50:29.140502 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.140471 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv] Nov 16 14:50:29.149799 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.149688610Z" level=info msg="Ran pod sandbox 9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c with infra container: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/POD" id=6cf31962-4f8b-4119-9388-b9ed4f211ed1 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:29.150861 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.150823318Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=118d967c-b27c-45f7-b456-dab09b493f89 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:29.151119 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.151089720Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7 not found" id=118d967c-b27c-45f7-b456-dab09b493f89 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:29.151917 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.151846126Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=12b90678-7298-4127-803d-5d6e7afcee0c name=/runtime.v1.ImageService/PullImage Nov 16 14:50:29.154346 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.154310544Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7\"" Nov 16 14:50:29.179925 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.179875 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx] Nov 16 14:50:29.180443 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:50:29Z [verbose] Add: openshift-monitoring:prometheus-adapter-6fb766b4f6-jcwvx:17691c72-c4d7-4e7f-b345-a0447ae7d4c6:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/e966bd8c-2d2d-4176-8a8d-7d70130b4c3b"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.32/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:50:29.180443 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:50:29.139713 176215 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-monitoring", Name:"prometheus-adapter-6fb766b4f6-jcwvx", UID:"17691c72-c4d7-4e7f-b345-a0447ae7d4c6", APIVersion:"v1", ResourceVersion:"95288", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.32/23] from openshift-sdn Nov 16 14:50:29.181020 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.180967241Z" level=info msg="Got pod network &{Name:prometheus-adapter-6fb766b4f6-jcwvx Namespace:openshift-monitoring ID:9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486 UID:17691c72-c4d7-4e7f-b345-a0447ae7d4c6 NetNS:/var/run/netns/e966bd8c-2d2d-4176-8a8d-7d70130b4c3b Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:29.181144 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.181109942Z" level=info msg="Checking pod openshift-monitoring_prometheus-adapter-6fb766b4f6-jcwvx for CNI network multus-cni-network (type=multus)" Nov 16 14:50:29.189986 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.189919 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm] Nov 16 14:50:29.189986 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.189984 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:50:29.192188 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:29.192152 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17691c72_c4d7_4e7f_b345_a0447ae7d4c6.slice/crio-9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486.scope WatchSource:0}: Error finding container 9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486: Status 404 returned error &{%!s(*http.body=&{0xc002ce5758 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:50:29.193816 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.193776635Z" level=info msg="Ran pod sandbox 9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486 with infra container: openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx/POD" id=a668b082-68dd-4f61-b4ea-99439335f0ce name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:29.195434 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.195389047Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ac0f7d274a041d87df1e418467062f45ec923abe9c50ee8901cda86906ffb84d" id=0cc3d6f3-d779-4c26-b57c-54a1b828edb2 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:29.195713 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.195682850Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ac0f7d274a041d87df1e418467062f45ec923abe9c50ee8901cda86906ffb84d not found" id=0cc3d6f3-d779-4c26-b57c-54a1b828edb2 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:29.196422 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.196386155Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ac0f7d274a041d87df1e418467062f45ec923abe9c50ee8901cda86906ffb84d" id=08c6abb7-d001-4703-94db-2b14e13863e0 name=/runtime.v1.ImageService/PullImage Nov 16 14:50:29.198952 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.198877973Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ac0f7d274a041d87df1e418467062f45ec923abe9c50ee8901cda86906ffb84d\"" Nov 16 14:50:29.203545 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-poda776019a_1ce1_4328_aed5_a5962a4f775d.slice. Nov 16 14:50:29.215229 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.215187 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a776019a-1ce1-4328-aed5-a5962a4f775d-metrics-client-ca\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.215342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.215241 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-tls\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.215342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.215275 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdtt2\" (UniqueName: \"kubernetes.io/projected/a776019a-1ce1-4328-aed5-a5962a4f775d-kube-api-access-pdtt2\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.215342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.215318 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-querier-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a776019a-1ce1-4328-aed5-a5962a4f775d-thanos-querier-trusted-ca-bundle\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.215455 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.215354 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-oauth-cookie\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-oauth-cookie\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.215455 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.215385 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-grpc-tls\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.215455 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.215417 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.215455 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.215447 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.215627 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.215501 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.316900 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.316768 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"thanos-querier-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a776019a-1ce1-4328-aed5-a5962a4f775d-thanos-querier-trusted-ca-bundle\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.316900 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.316874 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-oauth-cookie\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-oauth-cookie\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.317116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.316943 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-grpc-tls\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.317116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.316978 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.317116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.317016 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.317116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.317050 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.317116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.317084 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a776019a-1ce1-4328-aed5-a5962a4f775d-metrics-client-ca\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.317116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.317117 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-tls\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.317320 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.317171 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-pdtt2\" (UniqueName: \"kubernetes.io/projected/a776019a-1ce1-4328-aed5-a5962a4f775d-kube-api-access-pdtt2\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.317893 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.317857 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"thanos-querier-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a776019a-1ce1-4328-aed5-a5962a4f775d-thanos-querier-trusted-ca-bundle\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.317989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.317942 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a776019a-1ce1-4328-aed5-a5962a4f775d-metrics-client-ca\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.319751 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.319717 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-oauth-cookie\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-oauth-cookie\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.320150 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.320100 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.320223 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.320161 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-tls\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.320223 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.320180 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.321857 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.321808 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.321930 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.321874 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-grpc-tls\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.343565 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.343501 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdtt2\" (UniqueName: \"kubernetes.io/projected/a776019a-1ce1-4328-aed5-a5962a4f775d-kube-api-access-pdtt2\") pod \"thanos-querier-5b6b7d7fd9-d95wm\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.509248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.509080 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:50:29.509969 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.509899270Z" level=info msg="Running pod sandbox: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/POD" id=8bfd0063-c9a0-4022-a6f6-45b70cf69f76 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:29.510057 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.509992070Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:29.585733 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.585648529Z" level=info msg="Got pod network &{Name:thanos-querier-5b6b7d7fd9-d95wm Namespace:openshift-monitoring ID:4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c UID:a776019a-1ce1-4328-aed5-a5962a4f775d NetNS:/var/run/netns/7a914dca-3986-42a0-ab4c-ab19848f2047 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:29.585733 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:29.585691229Z" level=info msg="Adding pod openshift-monitoring_thanos-querier-5b6b7d7fd9-d95wm to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:50:29.638218 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:50:29.649431 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610229.6494] manager: (veth7b8f5bbe): new Veth device (/org/freedesktop/NetworkManager/Devices/72) Nov 16 14:50:29.650464 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610229.6504] device (veth7b8f5bbe): carrier: link connected Nov 16 14:50:29.654782 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[176376]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:50:29.661907 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth7b8f5bbe: link is not ready Nov 16 14:50:29.661989 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth7b8f5bbe: link becomes ready Nov 16 14:50:29.662019 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:50:29.668336 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[176376]: Could not generate persistent MAC address for veth7b8f5bbe: No such file or directory Nov 16 14:50:29.694769 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.694732 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" event=&{ID:add06140-663d-48c0-ac00-643bd58a3f7d Type:ContainerStarted Data:a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145} Nov 16 14:50:29.695745 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.695712 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" event=&{ID:17691c72-c4d7-4e7f-b345-a0447ae7d4c6 Type:ContainerStarted Data:9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486} Nov 16 14:50:29.695925 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610229.6958] manager: (veth7b8f5bbe): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/73) Nov 16 14:50:29.700720 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.696573 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" event=&{ID:454b3a37-199c-4abc-a79a-8de93eab2423 Type:ContainerStarted Data:5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3} Nov 16 14:50:29.701153 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth7b8f5bbe entered promiscuous mode Nov 16 14:50:29.701291 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00193|bridge|INFO|bridge br0: added interface veth7b8f5bbe on port 34 Nov 16 14:50:29.701804 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:29.701300 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" event=&{ID:c8013b4f-15b6-4bac-bfc9-a45301362f0c Type:ContainerStarted Data:9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c} Nov 16 14:50:29.735533 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00194|connmgr|INFO|br0<->unix#427: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:50:29.784837 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00195|connmgr|INFO|br0<->unix#430: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:50:30.086866 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:30.086767229Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3507212f8cf79da97eb88e4456049226704313f07aa9e14726e7a04691cad554\"" Nov 16 14:50:30.276821 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:30.276771931Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:119575c2aa477a5216fc18144f23ff48666e950ad7bca70d13af93cc6f2a54d2\"" Nov 16 14:50:30.314270 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.B6SJAF.mount: Succeeded. Nov 16 14:50:30.440742 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:30.440607640Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7\"" Nov 16 14:50:30.471620 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:30.471572869Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ac0f7d274a041d87df1e418467062f45ec923abe9c50ee8901cda86906ffb84d\"" Nov 16 14:50:31.011023 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:50:30Z [verbose] Add: openshift-monitoring:thanos-querier-5b6b7d7fd9-d95wm:a776019a-1ce1-4328-aed5-a5962a4f775d:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/7a914dca-3986-42a0-ab4c-ab19848f2047"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.33/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:50:31.011023 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:50:30.976821 176360 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-monitoring", Name:"thanos-querier-5b6b7d7fd9-d95wm", UID:"a776019a-1ce1-4328-aed5-a5962a4f775d", APIVersion:"v1", ResourceVersion:"95451", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.33/23] from openshift-sdn Nov 16 14:50:31.011752 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:31.011706055Z" level=info msg="Got pod network &{Name:thanos-querier-5b6b7d7fd9-d95wm Namespace:openshift-monitoring ID:4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c UID:a776019a-1ce1-4328-aed5-a5962a4f775d NetNS:/var/run/netns/7a914dca-3986-42a0-ab4c-ab19848f2047 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:31.011893 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:31.011856256Z" level=info msg="Checking pod openshift-monitoring_thanos-querier-5b6b7d7fd9-d95wm for CNI network multus-cni-network (type=multus)" Nov 16 14:50:31.014456 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:31.014404 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda776019a_1ce1_4328_aed5_a5962a4f775d.slice/crio-4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c.scope WatchSource:0}: Error finding container 4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c: Status 404 returned error &{%!s(*http.body=&{0xc0022b48a0 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:50:31.022383 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:31.017536 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm] Nov 16 14:50:31.024441 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:31.024398649Z" level=info msg="Ran pod sandbox 4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c with infra container: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/POD" id=8bfd0063-c9a0-4022-a6f6-45b70cf69f76 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:31.025620 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:31.025581858Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c7327d965050fa0e8fbc48300a8c6ce1bb04fad7e4e6349290dde68b53e9cffd" id=87c1e160-6ea9-470e-89bb-f7c604cb4231 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:31.026006 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:31.025902360Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c7327d965050fa0e8fbc48300a8c6ce1bb04fad7e4e6349290dde68b53e9cffd not found" id=87c1e160-6ea9-470e-89bb-f7c604cb4231 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:31.031984 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:31.031932805Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c7327d965050fa0e8fbc48300a8c6ce1bb04fad7e4e6349290dde68b53e9cffd" id=26d563f6-e925-4739-b356-55b61d30bb24 name=/runtime.v1.ImageService/PullImage Nov 16 14:50:31.034568 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:31.034529524Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c7327d965050fa0e8fbc48300a8c6ce1bb04fad7e4e6349290dde68b53e9cffd\"" Nov 16 14:50:31.147778 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f-runc.KE6ZKw.mount: Succeeded. Nov 16 14:50:31.147802 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f-runc.7H3EEs.mount: Succeeded. Nov 16 14:50:31.708245 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:31.708203 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerStarted Data:4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c} Nov 16 14:50:32.208588 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:32.208531 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-s6mgf] Nov 16 14:50:32.209093 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:32.208790 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-s6mgf" podUID=ba736212-a121-4f49-a957-3f6bdc1c986d containerName="registry-server" containerID="cri-o://c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259" gracePeriod=1 Nov 16 14:50:32.209170 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:32.209115692Z" level=info msg="Stopping container: c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259 (timeout: 1s)" id=4ea4ff97-6076-4d01-9315-2bd7da709b52 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:50:32.235073 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259.scope: Succeeded. Nov 16 14:50:32.235725 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259.scope: Consumed 44.227s CPU time Nov 16 14:50:32.242100 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259.scope: Succeeded. Nov 16 14:50:32.242752 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259.scope: Consumed 34ms CPU time Nov 16 14:50:32.250557 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:32.250482 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-bjxz7] Nov 16 14:50:32.250557 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:32.250549 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:50:32.260273 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-podf27359d7_25d3_44d2_a327_f7cbd09b1815.slice. Nov 16 14:50:32.294449 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:32.294406 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-marketplace/qe-app-registry-fh97d] Nov 16 14:50:32.322726 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:32.322671730Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c7327d965050fa0e8fbc48300a8c6ce1bb04fad7e4e6349290dde68b53e9cffd\"" Nov 16 14:50:32.334464 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:32.334422 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-bjxz7] Nov 16 14:50:32.364801 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-4206cc9e8d2476c2baf456b242f61248bbabc11ce13b352e4fd0fb0f8abc8a8f-merged.mount: Succeeded. Nov 16 14:50:32.365587 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-4206cc9e8d2476c2baf456b242f61248bbabc11ce13b352e4fd0fb0f8abc8a8f-merged.mount: Consumed 0 CPU time Nov 16 14:50:32.446472 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:32.446426 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r28qw\" (UniqueName: \"kubernetes.io/projected/f27359d7-25d3-44d2-a327-f7cbd09b1815-kube-api-access-r28qw\") pod \"qe-app-registry-bjxz7\" (UID: \"f27359d7-25d3-44d2-a327-f7cbd09b1815\") " pod="openshift-marketplace/qe-app-registry-bjxz7" Nov 16 14:50:32.547253 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:32.547027 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-r28qw\" (UniqueName: \"kubernetes.io/projected/f27359d7-25d3-44d2-a327-f7cbd09b1815-kube-api-access-r28qw\") pod \"qe-app-registry-bjxz7\" (UID: \"f27359d7-25d3-44d2-a327-f7cbd09b1815\") " pod="openshift-marketplace/qe-app-registry-bjxz7" Nov 16 14:50:32.571377 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:32.571277 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-r28qw\" (UniqueName: \"kubernetes.io/projected/f27359d7-25d3-44d2-a327-f7cbd09b1815-kube-api-access-r28qw\") pod \"qe-app-registry-bjxz7\" (UID: \"f27359d7-25d3-44d2-a327-f7cbd09b1815\") " pod="openshift-marketplace/qe-app-registry-bjxz7" Nov 16 14:50:32.576968 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:32.576905 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-bjxz7" Nov 16 14:50:32.578166 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:32.578038515Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-bjxz7/POD" id=cf00cf02-7c50-4430-a1ed-b1a11685cb23 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:32.578267 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:32.578173216Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:32.711713 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:32.711636 1563 generic.go:296] "Generic (PLEG): container finished" podID=ba736212-a121-4f49-a957-3f6bdc1c986d containerID="c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259" exitCode=0 Nov 16 14:50:32.711713 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:32.711690 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-s6mgf" event=&{ID:ba736212-a121-4f49-a957-3f6bdc1c986d Type:ContainerDied Data:c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259} Nov 16 14:50:34.464007 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:34.464492 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:34.489592 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:34.489916 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:35.075435 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:35.075966 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:35.102104 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:35.102490 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:35.346458 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.346200 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-image-registry/image-registry-944cf86d8-kb2fb] Nov 16 14:50:35.346458 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.346460 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:50:35.361499 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod0520c563_12a4_4756_b022_6ec7396a5839.slice. Nov 16 14:50:35.485100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.485032 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0520c563-12a4-4756-b022-6ec7396a5839-trusted-ca\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.485100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.485099 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0520c563-12a4-4756-b022-6ec7396a5839-installation-pull-secrets\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.485100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.485146 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfvls\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-kube-api-access-kfvls\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.485373 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.485188 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0520c563-12a4-4756-b022-6ec7396a5839-ca-trust-extracted\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.485373 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.485220 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-registry-tls\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.485373 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.485313 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0520c563-12a4-4756-b022-6ec7396a5839-registry-certificates\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.485373 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.485367 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-bound-sa-token\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.586320 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.586273 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0520c563-12a4-4756-b022-6ec7396a5839-trusted-ca\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.586320 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.586338 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0520c563-12a4-4756-b022-6ec7396a5839-installation-pull-secrets\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.586625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.586409 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-kfvls\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-kube-api-access-kfvls\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.586625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.586480 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0520c563-12a4-4756-b022-6ec7396a5839-ca-trust-extracted\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.586625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.586511 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-registry-tls\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.586625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.586568 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0520c563-12a4-4756-b022-6ec7396a5839-registry-certificates\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.586625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.586604 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-bound-sa-token\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.587068 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.587035 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0520c563-12a4-4756-b022-6ec7396a5839-ca-trust-extracted\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.587405 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.587359 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0520c563-12a4-4756-b022-6ec7396a5839-trusted-ca\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.587576 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.587543 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0520c563-12a4-4756-b022-6ec7396a5839-registry-certificates\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.591635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.591554 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0520c563-12a4-4756-b022-6ec7396a5839-installation-pull-secrets\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.591635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.591631 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-registry-tls\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.609525 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.609388 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-bound-sa-token\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.611191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.611111 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfvls\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-kube-api-access-kfvls\") pod \"image-registry-944cf86d8-kb2fb\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.680140 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:35.679984 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:50:35.681004 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:35.680906015Z" level=info msg="Running pod sandbox: openshift-image-registry/image-registry-944cf86d8-kb2fb/POD" id=c2486f8a-7a9a-4d98-a88e-49e9fd16b1a0 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:35.681383 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:35.681030016Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:35.953591 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:35.954014 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:35.975743 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:35.976197 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:36.694589 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:36.694543 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-cluster-node-tuning-operator/tuned-kt88r] Nov 16 14:50:36.942704 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:50:36.942661 1563 remote_runtime.go:704] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259 is running failed: container process not found" containerID="c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259" cmd=[grpc_health_probe -addr=:50051] Nov 16 14:50:36.943040 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:50:36.943015 1563 remote_runtime.go:704] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259 is running failed: container process not found" containerID="c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259" cmd=[grpc_health_probe -addr=:50051] Nov 16 14:50:36.943372 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:50:36.943346 1563 remote_runtime.go:704] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259 is running failed: container process not found" containerID="c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259" cmd=[grpc_health_probe -addr=:50051] Nov 16 14:50:36.943440 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:50:36.943407 1563 prober.go:118] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259 is running failed: container process not found" probeType="Readiness" pod="openshift-marketplace/qe-app-registry-s6mgf" podUID=ba736212-a121-4f49-a957-3f6bdc1c986d containerName="registry-server" Nov 16 14:50:38.710115 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:38.710562 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:38.730083 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:38.730410 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:39.266973 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:39.267491 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:39.295282 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:39.295775 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:40.150187 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:40.150707 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:40.170339 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:40.170681 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:40.380364 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf-runc.TsuS2N.mount: Succeeded. Nov 16 14:50:40.380417 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.HrJ9il.mount: Succeeded. Nov 16 14:50:40.452547 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:40.452379522Z" level=info msg="Stopped container c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259: openshift-marketplace/qe-app-registry-s6mgf/registry-server" id=4ea4ff97-6076-4d01-9315-2bd7da709b52 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:50:40.455042 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:40.454995841Z" level=info msg="Stopping pod sandbox: 50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc" id=08b6afbb-a9b3-4b06-900d-6d0c3a91e71e name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:40.455371 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:40.455335643Z" level=info msg="Got pod network &{Name:qe-app-registry-s6mgf Namespace:openshift-marketplace ID:50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc UID:ba736212-a121-4f49-a957-3f6bdc1c986d NetNS:/var/run/netns/158d06fe-08de-4558-9e81-9effea3a818d Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:40.455531 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:40.455500845Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-s6mgf from CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:50:40.534054 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00196|connmgr|INFO|br0<->unix#433: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:50:40.566618 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00197|connmgr|INFO|br0<->unix#436: 4 flow_mods in the last 0 s (4 deletes) Nov 16 14:50:40.596853 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00198|bridge|INFO|bridge br0: deleted interface veth910dcc54 on port 10 Nov 16 14:50:40.603344 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth910dcc54 left promiscuous mode Nov 16 14:50:40.613099 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:50:40Z [verbose] Del: openshift-marketplace:qe-app-registry-s6mgf:ba736212-a121-4f49-a957-3f6bdc1c986d:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 14:50:40.622421 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-158d06fe\x2d08de\x2d4558\x2d9e81\x2d9effea3a818d.mount: Succeeded. Nov 16 14:50:40.623487 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-158d06fe\x2d08de\x2d4558\x2d9e81\x2d9effea3a818d.mount: Consumed 0 CPU time Nov 16 14:50:40.648322 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:40.648686 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:40.669870 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:50:40.670233 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:50:40.897883 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf.scope: Succeeded. Nov 16 14:50:40.898604 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf.scope: Consumed 68ms CPU time Nov 16 14:50:40.901354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:40.901316 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-cluster-node-tuning-operator/tuned-kt88r" podUID=37496343-5356-4736-accc-adc8f05b43c5 containerName="tuned" containerID="cri-o://9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf" gracePeriod=30 Nov 16 14:50:40.901864 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:40.901559235Z" level=info msg="Stopping container: 9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf (timeout: 30s)" id=ef5f3f06-cf7a-47bf-8033-38fe995224cf name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:50:40.904865 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf.scope: Succeeded. Nov 16 14:50:40.905790 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf.scope: Consumed 18.841s CPU time Nov 16 14:50:40.948372 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:40.948316180Z" level=info msg="Stopped container 9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf: openshift-cluster-node-tuning-operator/tuned-kt88r/tuned" id=ef5f3f06-cf7a-47bf-8033-38fe995224cf name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:50:40.948844 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:40.948805284Z" level=info msg="Stopping pod sandbox: dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669" id=4728bada-9491-4247-975d-2bf9f0ad44d9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:41.380585 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-158d06fe\x2d08de\x2d4558\x2d9e81\x2d9effea3a818d.mount: Succeeded. Nov 16 14:50:41.381562 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-158d06fe\x2d08de\x2d4558\x2d9e81\x2d9effea3a818d.mount: Consumed 0 CPU time Nov 16 14:50:41.381586 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-158d06fe\x2d08de\x2d4558\x2d9e81\x2d9effea3a818d.mount: Succeeded. Nov 16 14:50:41.382296 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-158d06fe\x2d08de\x2d4558\x2d9e81\x2d9effea3a818d.mount: Consumed 0 CPU time Nov 16 14:50:41.382314 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc-userdata-shm.mount: Succeeded. Nov 16 14:50:41.382975 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc-userdata-shm.mount: Consumed 0 CPU time Nov 16 14:50:41.383016 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6c780b52a7cc21d3c6501863f44c285d6c1b67fec6f47f5c1c71583e4a8be82b-merged.mount: Succeeded. Nov 16 14:50:41.383701 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6c780b52a7cc21d3c6501863f44c285d6c1b67fec6f47f5c1c71583e4a8be82b-merged.mount: Consumed 0 CPU time Nov 16 14:50:41.383718 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-e30bedfb\x2d4fc5\x2d453a\x2d82e2\x2de236ff49111d.mount: Succeeded. Nov 16 14:50:41.384381 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-e30bedfb\x2d4fc5\x2d453a\x2d82e2\x2de236ff49111d.mount: Consumed 0 CPU time Nov 16 14:50:41.384399 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-e30bedfb\x2d4fc5\x2d453a\x2d82e2\x2de236ff49111d.mount: Succeeded. Nov 16 14:50:41.385012 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-e30bedfb\x2d4fc5\x2d453a\x2d82e2\x2de236ff49111d.mount: Consumed 0 CPU time Nov 16 14:50:41.385028 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-e30bedfb\x2d4fc5\x2d453a\x2d82e2\x2de236ff49111d.mount: Succeeded. Nov 16 14:50:41.385689 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-e30bedfb\x2d4fc5\x2d453a\x2d82e2\x2de236ff49111d.mount: Consumed 0 CPU time Nov 16 14:50:41.745581 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:41.745383 1563 generic.go:296] "Generic (PLEG): container finished" podID=37496343-5356-4736-accc-adc8f05b43c5 containerID="9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf" exitCode=0 Nov 16 14:50:41.745581 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:41.745445 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kt88r" event=&{ID:37496343-5356-4736-accc-adc8f05b43c5 Type:ContainerDied Data:9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf} Nov 16 14:50:42.797912 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:42.797793723Z" level=info msg="Stopped pod sandbox: dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669" id=4728bada-9491-4247-975d-2bf9f0ad44d9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:42.866304 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866023 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-sys\") pod \"37496343-5356-4736-accc-adc8f05b43c5\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " Nov 16 14:50:42.866823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866344 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"var-lib-tuned-profiles-data\" (UniqueName: \"kubernetes.io/configmap/37496343-5356-4736-accc-adc8f05b43c5-var-lib-tuned-profiles-data\") pod \"37496343-5356-4736-accc-adc8f05b43c5\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " Nov 16 14:50:42.866823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866180 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-sys" (OuterVolumeSpecName: "sys") pod "37496343-5356-4736-accc-adc8f05b43c5" (UID: "37496343-5356-4736-accc-adc8f05b43c5"). InnerVolumeSpecName "sys". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:50:42.866823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866390 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pw2z\" (UniqueName: \"kubernetes.io/projected/37496343-5356-4736-accc-adc8f05b43c5-kube-api-access-8pw2z\") pod \"37496343-5356-4736-accc-adc8f05b43c5\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " Nov 16 14:50:42.866823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866491 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"etc\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-etc\") pod \"37496343-5356-4736-accc-adc8f05b43c5\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " Nov 16 14:50:42.866823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866532 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"run-systemd-system\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-run-systemd-system\") pod \"37496343-5356-4736-accc-adc8f05b43c5\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " Nov 16 14:50:42.866823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866560 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-lib-modules\") pod \"37496343-5356-4736-accc-adc8f05b43c5\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " Nov 16 14:50:42.866823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866597 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-var-run-dbus\") pod \"37496343-5356-4736-accc-adc8f05b43c5\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " Nov 16 14:50:42.866823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866629 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-host\") pod \"37496343-5356-4736-accc-adc8f05b43c5\" (UID: \"37496343-5356-4736-accc-adc8f05b43c5\") " Nov 16 14:50:42.866823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866625 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-run-systemd-system" (OuterVolumeSpecName: "run-systemd-system") pod "37496343-5356-4736-accc-adc8f05b43c5" (UID: "37496343-5356-4736-accc-adc8f05b43c5"). InnerVolumeSpecName "run-systemd-system". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:50:42.866823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866654 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-var-run-dbus" (OuterVolumeSpecName: "var-run-dbus") pod "37496343-5356-4736-accc-adc8f05b43c5" (UID: "37496343-5356-4736-accc-adc8f05b43c5"). InnerVolumeSpecName "var-run-dbus". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:50:42.866823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866644 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-lib-modules" (OuterVolumeSpecName: "lib-modules") pod "37496343-5356-4736-accc-adc8f05b43c5" (UID: "37496343-5356-4736-accc-adc8f05b43c5"). InnerVolumeSpecName "lib-modules". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:50:42.866823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:42.866627 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/37496343-5356-4736-accc-adc8f05b43c5/volumes/kubernetes.io~configmap/var-lib-tuned-profiles-data: clearQuota called, but quotas disabled Nov 16 14:50:42.866823 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866696 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-host" (OuterVolumeSpecName: "host") pod "37496343-5356-4736-accc-adc8f05b43c5" (UID: "37496343-5356-4736-accc-adc8f05b43c5"). InnerVolumeSpecName "host". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:50:42.867248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866856 1563 reconciler.go:300] "Volume detached for volume \"var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-var-run-dbus\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:50:42.867248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866876 1563 reconciler.go:300] "Volume detached for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-host\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:50:42.867248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866884 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37496343-5356-4736-accc-adc8f05b43c5-var-lib-tuned-profiles-data" (OuterVolumeSpecName: "var-lib-tuned-profiles-data") pod "37496343-5356-4736-accc-adc8f05b43c5" (UID: "37496343-5356-4736-accc-adc8f05b43c5"). InnerVolumeSpecName "var-lib-tuned-profiles-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 14:50:42.867248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866891 1563 reconciler.go:300] "Volume detached for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-sys\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:50:42.867248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866932 1563 reconciler.go:300] "Volume detached for volume \"run-systemd-system\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-run-systemd-system\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:50:42.867248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.866951 1563 reconciler.go:300] "Volume detached for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-lib-modules\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:50:42.880915 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-37496343\x2d5356\x2d4736\x2daccc\x2dadc8f05b43c5-volume\x2dsubpaths-etc-tuned-1.mount: Succeeded. Nov 16 14:50:42.881512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:42.881480 1563 mount_helper_common.go:133] Warning: "/var/lib/kubelet/pods/37496343-5356-4736-accc-adc8f05b43c5/volume-subpaths/etc/tuned/1" is not a mountpoint, deleting Nov 16 14:50:42.881809 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-37496343\x2d5356\x2d4736\x2daccc\x2dadc8f05b43c5-volume\x2dsubpaths-etc-tuned-1.mount: Consumed 0 CPU time Nov 16 14:50:42.881848 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-37496343\x2d5356\x2d4736\x2daccc\x2dadc8f05b43c5-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d8pw2z.mount: Succeeded. Nov 16 14:50:42.882610 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-37496343\x2d5356\x2d4736\x2daccc\x2dadc8f05b43c5-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d8pw2z.mount: Consumed 0 CPU time Nov 16 14:50:42.894437 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.894378 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37496343-5356-4736-accc-adc8f05b43c5-kube-api-access-8pw2z" (OuterVolumeSpecName: "kube-api-access-8pw2z") pod "37496343-5356-4736-accc-adc8f05b43c5" (UID: "37496343-5356-4736-accc-adc8f05b43c5"). InnerVolumeSpecName "kube-api-access-8pw2z". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:50:42.911962 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-37496343\x2d5356\x2d4736\x2daccc\x2dadc8f05b43c5-volume\x2dsubpaths-etc-tuned-2.mount: Succeeded. Nov 16 14:50:42.912943 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-37496343\x2d5356\x2d4736\x2daccc\x2dadc8f05b43c5-volume\x2dsubpaths-etc-tuned-2.mount: Consumed 0 CPU time Nov 16 14:50:42.915543 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:42.915473 1563 mount_helper_common.go:133] Warning: "/var/lib/kubelet/pods/37496343-5356-4736-accc-adc8f05b43c5/volume-subpaths/etc/tuned/2" is not a mountpoint, deleting Nov 16 14:50:42.936905 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-37496343\x2d5356\x2d4736\x2daccc\x2dadc8f05b43c5-volume\x2dsubpaths-etc-tuned-3.mount: Succeeded. Nov 16 14:50:42.937800 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-37496343\x2d5356\x2d4736\x2daccc\x2dadc8f05b43c5-volume\x2dsubpaths-etc-tuned-3.mount: Consumed 0 CPU time Nov 16 14:50:42.948030 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:42.947994 1563 mount_helper_common.go:133] Warning: "/var/lib/kubelet/pods/37496343-5356-4736-accc-adc8f05b43c5/volume-subpaths/etc/tuned/3" is not a mountpoint, deleting Nov 16 14:50:42.968116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.968035 1563 reconciler.go:300] "Volume detached for volume \"var-lib-tuned-profiles-data\" (UniqueName: \"kubernetes.io/configmap/37496343-5356-4736-accc-adc8f05b43c5-var-lib-tuned-profiles-data\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:50:42.968116 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:42.968083 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-8pw2z\" (UniqueName: \"kubernetes.io/projected/37496343-5356-4736-accc-adc8f05b43c5-kube-api-access-8pw2z\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:50:42.976766 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-37496343\x2d5356\x2d4736\x2daccc\x2dadc8f05b43c5-volume\x2dsubpaths-etc-tuned-4.mount: Succeeded. Nov 16 14:50:42.977747 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-37496343\x2d5356\x2d4736\x2daccc\x2dadc8f05b43c5-volume\x2dsubpaths-etc-tuned-4.mount: Consumed 0 CPU time Nov 16 14:50:42.980466 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:42.980432 1563 mount_helper_common.go:133] Warning: "/var/lib/kubelet/pods/37496343-5356-4736-accc-adc8f05b43c5/volume-subpaths/etc/tuned/4" is not a mountpoint, deleting Nov 16 14:50:43.009857 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:43.009819 1563 mount_helper_common.go:133] Warning: "/var/lib/kubelet/pods/37496343-5356-4736-accc-adc8f05b43c5/volume-subpaths/etc/tuned/5" is not a mountpoint, deleting Nov 16 14:50:43.022406 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.022357 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-etc" (OuterVolumeSpecName: "etc") pod "37496343-5356-4736-accc-adc8f05b43c5" (UID: "37496343-5356-4736-accc-adc8f05b43c5"). InnerVolumeSpecName "etc". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:50:43.068567 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.068387 1563 reconciler.go:300] "Volume detached for volume \"etc\" (UniqueName: \"kubernetes.io/host-path/37496343-5356-4736-accc-adc8f05b43c5-etc\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:50:43.753621 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.753583 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kt88r" event=&{ID:37496343-5356-4736-accc-adc8f05b43c5 Type:ContainerDied Data:dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669} Nov 16 14:50:43.753621 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.753641 1563 scope.go:110] "RemoveContainer" containerID="9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf" Nov 16 14:50:43.755043 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:43.754988285Z" level=info msg="Removing container: 9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf" id=80eba0fb-86c5-465b-85b8-e46d96dea837 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:50:43.761687 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod37496343_5356_4736_accc_adc8f05b43c5.slice. Nov 16 14:50:43.761773 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod37496343_5356_4736_accc_adc8f05b43c5.slice: Consumed 18.910s CPU time Nov 16 14:50:43.791565 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.791461 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-cluster-node-tuning-operator/tuned-kt88r] Nov 16 14:50:43.804214 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.804180 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-cluster-node-tuning-operator/tuned-kt88r] Nov 16 14:50:43.846069 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.845998 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-cluster-node-tuning-operator/tuned-cvdsv] Nov 16 14:50:43.846069 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.846063 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:50:43.846273 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:50:43.846199 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="37496343-5356-4736-accc-adc8f05b43c5" containerName="tuned" Nov 16 14:50:43.846273 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.846214 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="37496343-5356-4736-accc-adc8f05b43c5" containerName="tuned" Nov 16 14:50:43.846341 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.846305 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="37496343-5356-4736-accc-adc8f05b43c5" containerName="tuned" Nov 16 14:50:43.856345 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod279f55b2_5534_42ae_ba2d_588e2c3a3ffe.slice. Nov 16 14:50:43.874695 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.874659 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2qmh\" (UniqueName: \"kubernetes.io/projected/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-kube-api-access-n2qmh\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.875284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.874714 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-lib-modules\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.875284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.874751 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-tuned-profiles-data\" (UniqueName: \"kubernetes.io/configmap/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-var-lib-tuned-profiles-data\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.875284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.874787 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-host\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.875284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.874917 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-etc\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.875284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.874955 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-sys\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.875284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.875030 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-var-run-dbus\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.875284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.875079 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd-system\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-run-systemd-system\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.879414 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0a38a5a43f6dc4300545098cef5f838232791e34d7392222dc9ed08ddef40f60-merged.mount: Succeeded. Nov 16 14:50:43.880296 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0a38a5a43f6dc4300545098cef5f838232791e34d7392222dc9ed08ddef40f60-merged.mount: Consumed 0 CPU time Nov 16 14:50:43.880332 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-37496343\x2d5356\x2d4736\x2daccc\x2dadc8f05b43c5-volume\x2dsubpaths-etc-tuned-5.mount: Succeeded. Nov 16 14:50:43.881006 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-37496343\x2d5356\x2d4736\x2daccc\x2dadc8f05b43c5-volume\x2dsubpaths-etc-tuned-5.mount: Consumed 0 CPU time Nov 16 14:50:43.976569 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.976527 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-var-run-dbus\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.976569 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.976592 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"run-systemd-system\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-run-systemd-system\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.976797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.976632 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-lib-modules\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.976797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.976665 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-n2qmh\" (UniqueName: \"kubernetes.io/projected/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-kube-api-access-n2qmh\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.976797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.976665 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-var-run-dbus\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.976797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.976702 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"var-lib-tuned-profiles-data\" (UniqueName: \"kubernetes.io/configmap/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-var-lib-tuned-profiles-data\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.976797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.976744 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-host\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.976797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.976701 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"run-systemd-system\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-run-systemd-system\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.976797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.976774 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"etc\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-etc\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.977173 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.976820 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-lib-modules\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.977173 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.976833 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-sys\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.977173 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.976973 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-sys\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.977173 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.976972 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"etc\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-etc\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.977173 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.977069 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-host\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:43.977358 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:43.977292 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"var-lib-tuned-profiles-data\" (UniqueName: \"kubernetes.io/configmap/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-var-lib-tuned-profiles-data\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:44.001336 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:44.001299 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2qmh\" (UniqueName: \"kubernetes.io/projected/279f55b2-5534-42ae-ba2d-588e2c3a3ffe-kube-api-access-n2qmh\") pod \"tuned-cvdsv\" (UID: \"279f55b2-5534-42ae-ba2d-588e2c3a3ffe\") " pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:44.167999 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:44.167823 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" Nov 16 14:50:44.168759 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:44.168670336Z" level=info msg="Running pod sandbox: openshift-cluster-node-tuning-operator/tuned-cvdsv/POD" id=f8c17d7e-3ebd-48e0-8fc5-88ef49069e52 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:44.169178 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:44.168776537Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:44.594105 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:44.594050574Z" level=info msg="Stopping pod sandbox: dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669" id=c75c62dc-8ca9-49e9-bb9b-19c505fea415 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:44.594290 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:44.594111975Z" level=info msg="Stopped pod sandbox (already stopped): dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669" id=c75c62dc-8ca9-49e9-bb9b-19c505fea415 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:44.596193 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:44.596151 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=37496343-5356-4736-accc-adc8f05b43c5 path="/var/lib/kubelet/pods/37496343-5356-4736-accc-adc8f05b43c5/volumes" Nov 16 14:50:44.853710 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:44.853439688Z" level=info msg="Stopped pod sandbox: 50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc" id=08b6afbb-a9b3-4b06-900d-6d0c3a91e71e name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:44.883673 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:44.883593 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrcl8\" (UniqueName: \"kubernetes.io/projected/ba736212-a121-4f49-a957-3f6bdc1c986d-kube-api-access-qrcl8\") pod \"ba736212-a121-4f49-a957-3f6bdc1c986d\" (UID: \"ba736212-a121-4f49-a957-3f6bdc1c986d\") " Nov 16 14:50:44.893314 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-ba736212\x2da121\x2d4f49\x2da957\x2d3f6bdc1c986d-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dqrcl8.mount: Succeeded. Nov 16 14:50:44.894260 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-ba736212\x2da121\x2d4f49\x2da957\x2d3f6bdc1c986d-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dqrcl8.mount: Consumed 0 CPU time Nov 16 14:50:44.894768 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:44.894656 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba736212-a121-4f49-a957-3f6bdc1c986d-kube-api-access-qrcl8" (OuterVolumeSpecName: "kube-api-access-qrcl8") pod "ba736212-a121-4f49-a957-3f6bdc1c986d" (UID: "ba736212-a121-4f49-a957-3f6bdc1c986d"). InnerVolumeSpecName "kube-api-access-qrcl8". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:50:44.984552 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:44.984461 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-qrcl8\" (UniqueName: \"kubernetes.io/projected/ba736212-a121-4f49-a957-3f6bdc1c986d-kube-api-access-qrcl8\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:50:45.233485 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-a3e50dd062cb56319c638f90bca3f3d27d5c03307ab0f4f9a6c2014398b0e4af-merged.mount: Succeeded. Nov 16 14:50:45.234524 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-a3e50dd062cb56319c638f90bca3f3d27d5c03307ab0f4f9a6c2014398b0e4af-merged.mount: Consumed 0 CPU time Nov 16 14:50:45.477484 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.khA9AT.mount: Succeeded. Nov 16 14:50:45.510070 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:45.509935431Z" level=info msg="Removed container 9a9e7b4fdce444139b5ef7565e26bcf98f8d4670f23bda346d148fdf53cd4edf: openshift-cluster-node-tuning-operator/tuned-kt88r/tuned" id=80eba0fb-86c5-465b-85b8-e46d96dea837 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:50:45.761928 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:45.761776 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-s6mgf" event=&{ID:ba736212-a121-4f49-a957-3f6bdc1c986d Type:ContainerDied Data:50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc} Nov 16 14:50:45.761928 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:45.761837 1563 scope.go:110] "RemoveContainer" containerID="c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259" Nov 16 14:50:45.762905 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:45.762856696Z" level=info msg="Removing container: c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259" id=a4ceb777-2fac-4301-a9fe-96ce9a59382b name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:50:45.769433 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-podba736212_a121_4f49_a957_3f6bdc1c986d.slice. Nov 16 14:50:45.769498 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-podba736212_a121_4f49_a957_3f6bdc1c986d.slice: Consumed 44.261s CPU time Nov 16 14:50:45.795804 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:45.795717 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-s6mgf] Nov 16 14:50:45.804187 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:45.804155 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-s6mgf] Nov 16 14:50:46.176410 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.176340047Z" level=info msg="Got pod network &{Name:qe-app-registry-bjxz7 Namespace:openshift-marketplace ID:56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58 UID:f27359d7-25d3-44d2-a327-f7cbd09b1815 NetNS:/var/run/netns/a40a8c16-8055-4d70-9c90-75c32bd7465c Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:46.176410 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.176384747Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-bjxz7 to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:50:46.212329 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.212244511Z" level=info msg="Got pod network &{Name:image-registry-944cf86d8-kb2fb Namespace:openshift-image-registry ID:4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e UID:0520c563-12a4-4756-b022-6ec7396a5839 NetNS:/var/run/netns/887d142c-2ca2-4563-b8cc-9b0e806e344e Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:46.212329 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.212288112Z" level=info msg="Adding pod openshift-image-registry_image-registry-944cf86d8-kb2fb to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:50:46.220189 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:50:46.239525 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610246.2392] manager: (vethe79c7a8b): new Veth device (/org/freedesktop/NetworkManager/Devices/74) Nov 16 14:50:46.240951 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.240899923Z" level=info msg="Removed container c3b62f10575c17e7dfdacdc07240163cc41df1264d4648ab19dc0ef5c5cfa259: openshift-marketplace/qe-app-registry-s6mgf/registry-server" id=a4ceb777-2fac-4301-a9fe-96ce9a59382b name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:50:46.241416 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610246.2413] device (vethe79c7a8b): carrier: link connected Nov 16 14:50:46.245602 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): vethe79c7a8b: link is not ready Nov 16 14:50:46.245683 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethe79c7a8b: link becomes ready Nov 16 14:50:46.245715 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:50:46.252056 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[178018]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:50:46.252497 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[178018]: Could not generate persistent MAC address for vethe79c7a8b: No such file or directory Nov 16 14:50:46.261933 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610246.2618] manager: (vethe79c7a8b): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/75) Nov 16 14:50:46.267254 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethe79c7a8b entered promiscuous mode Nov 16 14:50:46.267342 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00199|bridge|INFO|bridge br0: added interface vethe79c7a8b on port 35 Nov 16 14:50:46.282108 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.282015626Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3507212f8cf79da97eb88e4456049226704313f07aa9e14726e7a04691cad554" id=2e95c021-d8d0-475c-b33d-8246314e2fd1 name=/runtime.v1.ImageService/PullImage Nov 16 14:50:46.282108 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.282077027Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ac0f7d274a041d87df1e418467062f45ec923abe9c50ee8901cda86906ffb84d" id=08c6abb7-d001-4703-94db-2b14e13863e0 name=/runtime.v1.ImageService/PullImage Nov 16 14:50:46.283161 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:50:46.283411 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.283201535Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3507212f8cf79da97eb88e4456049226704313f07aa9e14726e7a04691cad554" id=ac955435-ca31-4b59-a3bf-ef016a35a58f name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.283411 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.283328136Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ac0f7d274a041d87df1e418467062f45ec923abe9c50ee8901cda86906ffb84d" id=fef38ebf-f8d3-4f20-9df5-6bc1a88fc690 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.292241 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610246.2922] manager: (vethf52cb045): new Veth device (/org/freedesktop/NetworkManager/Devices/76) Nov 16 14:50:46.293225 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610246.2932] device (vethf52cb045): carrier: link connected Nov 16 14:50:46.299851 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[178041]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:50:46.300249 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:50:46.300094 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[178041]: Could not generate persistent MAC address for vethf52cb045: No such file or directory Nov 16 14:50:46.312574 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00200|connmgr|INFO|br0<->unix#439: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:50:46.312722 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.312570551Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=f8c17d7e-3ebd-48e0-8fc5-88ef49069e52 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:46.316085 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:46.316047 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod279f55b2_5534_42ae_ba2d_588e2c3a3ffe.slice/crio-c70800789dc90e22c5a5cb32ae41a6619a6235951bc92cafe30ee53d4b9b92d0.scope WatchSource:0}: Error finding container c70800789dc90e22c5a5cb32ae41a6619a6235951bc92cafe30ee53d4b9b92d0: Status 404 returned error &{%!s(*http.body=&{0xc003621728 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:50:46.323333 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.323294931Z" level=info msg="Ran pod sandbox c70800789dc90e22c5a5cb32ae41a6619a6235951bc92cafe30ee53d4b9b92d0 with infra container: openshift-cluster-node-tuning-operator/tuned-cvdsv/POD" id=f8c17d7e-3ebd-48e0-8fc5-88ef49069e52 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:46.324389 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.324351638Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26611bedb7c02dbea85931612f83490e345de3f75644a914a70cb7fba2caa168" id=33d55222-2e59-4f20-8475-a48d5d4fab36 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.335217 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.335104818Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:119575c2aa477a5216fc18144f23ff48666e950ad7bca70d13af93cc6f2a54d2" id=8e425a7d-8945-4c40-95d9-d6710ca0a0b4 name=/runtime.v1.ImageService/PullImage Nov 16 14:50:46.335217 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.335207618Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=12b90678-7298-4127-803d-5d6e7afcee0c name=/runtime.v1.ImageService/PullImage Nov 16 14:50:46.336101 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.336063925Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:119575c2aa477a5216fc18144f23ff48666e950ad7bca70d13af93cc6f2a54d2" id=73c5026b-a2da-4a08-9d62-fa839f7aa3e9 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.336202 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.336092925Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=9fc3a0b4-1c9e-4ffb-9fbb-433e52777643 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.359175 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00201|connmgr|INFO|br0<->unix#442: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:50:46.371172 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.371110883Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26611bedb7c02dbea85931612f83490e345de3f75644a914a70cb7fba2caa168 not found" id=33d55222-2e59-4f20-8475-a48d5d4fab36 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.372756 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.372708795Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:7fc593bf62a0d2d13f0d9f8c24bc8c90e4ed3c49f6c9933d27adf157c092929c,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:119575c2aa477a5216fc18144f23ff48666e950ad7bca70d13af93cc6f2a54d2],Size_:379650883,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=73c5026b-a2da-4a08-9d62-fa839f7aa3e9 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.372845 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.372711495Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:a37e08f6820b6cdd7e1f18a91b13b3573fc74ed41c03326c7fbacdaaf3025855,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3507212f8cf79da97eb88e4456049226704313f07aa9e14726e7a04691cad554],Size_:369683468,Uid:nil,Username:nobody,Spec:nil,},Info:map[string]string{},}" id=ac955435-ca31-4b59-a3bf-ef016a35a58f name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.373008 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.372972497Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26611bedb7c02dbea85931612f83490e345de3f75644a914a70cb7fba2caa168" id=2e270705-d510-40fd-aa74-62c2f78fd717 name=/runtime.v1.ImageService/PullImage Nov 16 14:50:46.373156 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.373091198Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=9fc3a0b4-1c9e-4ffb-9fbb-433e52777643 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.373227 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.373203099Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:a733c23d1409bc02b70dcb819f5f2f5a76297d77506bbce0cf7dcd4cfaf74aae,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ac0f7d274a041d87df1e418467062f45ec923abe9c50ee8901cda86906ffb84d],Size_:397053507,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=fef38ebf-f8d3-4f20-9df5-6bc1a88fc690 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.375460 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.375419715Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c7327d965050fa0e8fbc48300a8c6ce1bb04fad7e4e6349290dde68b53e9cffd" id=26d563f6-e925-4739-b356-55b61d30bb24 name=/runtime.v1.ImageService/PullImage Nov 16 14:50:46.375734 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.375700817Z" level=info msg="Creating container: openshift-monitoring/telemeter-client-55c5556665-7vs77/telemeter-client" id=657efd31-7762-4063-9fd0-b87fb0314a19 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:46.375825 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.375799218Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:46.375939 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.375907919Z" level=info msg="Creating container: openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx/prometheus-adapter" id=5f42de42-5d53-4eb5-a459-7034b16fcca6 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:46.375939 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.375913919Z" level=info msg="Creating container: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/kube-rbac-proxy-main" id=ee4fff2b-3fc5-4ed6-ba6e-626da9307f7b name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:46.376010 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.375720917Z" level=info msg="Creating container: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-state-metrics" id=4631c9b5-a017-4611-b94e-838439964abb name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:46.376010 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.375991019Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:46.376090 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.376000419Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:46.376090 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.376047720Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26611bedb7c02dbea85931612f83490e345de3f75644a914a70cb7fba2caa168\"" Nov 16 14:50:46.376201 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.376056620Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:46.376801 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.376763425Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c7327d965050fa0e8fbc48300a8c6ce1bb04fad7e4e6349290dde68b53e9cffd" id=2359be8f-df6f-48f4-a34b-003b2d1590c3 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.378617 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.378577338Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:c38c6316f11e9dad3a0b7f9c03fb8c7b1d5bd8dbb625d13bc6774e49cb5fd72c,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c7327d965050fa0e8fbc48300a8c6ce1bb04fad7e4e6349290dde68b53e9cffd],Size_:403086886,Uid:nil,Username:nobody,Spec:nil,},Info:map[string]string{},}" id=2359be8f-df6f-48f4-a34b-003b2d1590c3 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.379723 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.379679246Z" level=info msg="Creating container: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/thanos-query" id=531aa640-f2ba-47fd-aa2c-a8ad80637da4 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:46.379811 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.379766647Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:46.380886 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610246.3808] manager: (vethf52cb045): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/77) Nov 16 14:50:46.386338 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00202|bridge|INFO|bridge br0: added interface vethf52cb045 on port 36 Nov 16 14:50:46.387181 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethf52cb045 entered promiscuous mode Nov 16 14:50:46.421597 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00203|connmgr|INFO|br0<->unix#445: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:50:46.467319 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d.scope. Nov 16 14:50:46.476794 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00204|connmgr|INFO|br0<->unix#449: 6 flow_mods in the last 0 s (2 adds, 4 deletes) Nov 16 14:50:46.478106 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00205|connmgr|INFO|br0<->unix#451: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:50:46.490564 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d. Nov 16 14:50:46.526698 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:50:46.535754 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30.scope. Nov 16 14:50:46.561110 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25.scope. Nov 16 14:50:46.562354 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30. Nov 16 14:50:46.569112 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610.scope. Nov 16 14:50:46.588460 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25. Nov 16 14:50:46.599692 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610. Nov 16 14:50:46.607942 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:50:46.613524 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d.scope. Nov 16 14:50:46.615227 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:46.615182 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=ba736212-a121-4f49-a957-3f6bdc1c986d path="/var/lib/kubelet/pods/ba736212-a121-4f49-a957-3f6bdc1c986d/volumes" Nov 16 14:50:46.631441 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.631390203Z" level=info msg="Created container 3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d: openshift-monitoring/telemeter-client-55c5556665-7vs77/telemeter-client" id=657efd31-7762-4063-9fd0-b87fb0314a19 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:46.632010 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.631968807Z" level=info msg="Starting container: 3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d" id=128c867f-f1ee-4dc9-aaf1-07f80ac3e089 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:50:46.632809 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:50:46.638349 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:50:46.642719 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d. Nov 16 14:50:46.646900 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.646850717Z" level=info msg="Started container" PID=178078 containerID=3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d description=openshift-monitoring/telemeter-client-55c5556665-7vs77/telemeter-client id=128c867f-f1ee-4dc9-aaf1-07f80ac3e089 name=/runtime.v1.RuntimeService/StartContainer sandboxID=a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145 Nov 16 14:50:46.668651 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.668599777Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3" id=18eeddfd-09a3-41cc-8a0e-35d61cdd1c4c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.668909 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.668886680Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3 not found" id=18eeddfd-09a3-41cc-8a0e-35d61cdd1c4c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.669768 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.669728786Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3" id=7dd73ede-26a9-4513-8afc-66271c942802 name=/runtime.v1.ImageService/PullImage Nov 16 14:50:46.672475 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.672373505Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3\"" Nov 16 14:50:46.674901 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:50:46.708227 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.708171069Z" level=info msg="Created container 9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30: openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx/prometheus-adapter" id=5f42de42-5d53-4eb5-a459-7034b16fcca6 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:46.708806 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.708772974Z" level=info msg="Starting container: 9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30" id=86ce98ff-05c6-468f-8676-7e3cd6b28f0d name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:50:46.721645 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.721528968Z" level=info msg="Started container" PID=178107 containerID=9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30 description=openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx/prometheus-adapter id=86ce98ff-05c6-468f-8676-7e3cd6b28f0d name=/runtime.v1.RuntimeService/StartContainer sandboxID=9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486 Nov 16 14:50:46.725098 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.725051194Z" level=info msg="Created container 89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/kube-rbac-proxy-main" id=ee4fff2b-3fc5-4ed6-ba6e-626da9307f7b name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:46.725622 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.725585098Z" level=info msg="Starting container: 89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25" id=50e02628-9b5a-42fa-95fa-3d85cd403e92 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:50:46.734167 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.734109061Z" level=info msg="Created container d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-state-metrics" id=4631c9b5-a017-4611-b94e-838439964abb name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:46.735116 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.735038868Z" level=info msg="Starting container: d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610" id=0fd96255-e120-4d73-8e2f-13aa8b1ddc5c name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:50:46.740142 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.740076305Z" level=info msg="Started container" PID=178131 containerID=89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25 description=openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/kube-rbac-proxy-main id=50e02628-9b5a-42fa-95fa-3d85cd403e92 name=/runtime.v1.RuntimeService/StartContainer sandboxID=9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c Nov 16 14:50:46.751756 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.751705690Z" level=info msg="Started container" PID=178138 containerID=d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610 description=openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-state-metrics id=0fd96255-e120-4d73-8e2f-13aa8b1ddc5c name=/runtime.v1.RuntimeService/StartContainer sandboxID=5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3 Nov 16 14:50:46.763910 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.763852480Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=6be9f0f5-899b-46a4-99c4-2195776db011 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.766153 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.765846695Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=6be9f0f5-899b-46a4-99c4-2195776db011 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.768058 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:46.768020 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" event=&{ID:17691c72-c4d7-4e7f-b345-a0447ae7d4c6 Type:ContainerStarted Data:9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30} Nov 16 14:50:46.768265 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:46.768249 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:50:46.769798 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:46.769748 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" event=&{ID:c8013b4f-15b6-4bac-bfc9-a45301362f0c Type:ContainerStarted Data:89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25} Nov 16 14:50:46.770903 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.770732631Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=27680808-54e9-44d1-864b-1931a0bd1d7d name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.773528 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.772543744Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=27680808-54e9-44d1-864b-1931a0bd1d7d name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.773995 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.773967755Z" level=info msg="Creating container: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/kube-rbac-proxy-self" id=8328aa54-acf1-4427-afb8-64bda489ba5a name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:46.774188 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.774164056Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:46.776726 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:46.776652 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" event=&{ID:add06140-663d-48c0-ac00-643bd58a3f7d Type:ContainerStarted Data:3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d} Nov 16 14:50:46.777872 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:46.777813 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" event=&{ID:279f55b2-5534-42ae-ba2d-588e2c3a3ffe Type:ContainerStarted Data:c70800789dc90e22c5a5cb32ae41a6619a6235951bc92cafe30ee53d4b9b92d0} Nov 16 14:50:46.778786 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.778745090Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=98ad0dab-a8a0-4662-811f-b0f532b197ed name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.795506 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.795459013Z" level=info msg="Created container 94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/thanos-query" id=531aa640-f2ba-47fd-aa2c-a8ad80637da4 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:46.796066 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.796028217Z" level=info msg="Starting container: 94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d" id=e5f3c1e4-fd00-43be-beb5-327c34cd1f9c name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:50:46.813376 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.813306745Z" level=info msg="Started container" PID=178176 containerID=94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d description=openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/thanos-query id=e5f3c1e4-fd00-43be-beb5-327c34cd1f9c name=/runtime.v1.RuntimeService/StartContainer sandboxID=4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c Nov 16 14:50:46.819146 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.818757485Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=98ad0dab-a8a0-4662-811f-b0f532b197ed name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.824263 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.824206225Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=9f5c51e8-944b-4ed1-85c6-5f796f87a70e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.825964 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.825922138Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=9f5c51e8-944b-4ed1-85c6-5f796f87a70e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.827160 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.827110647Z" level=info msg="Creating container: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-rbac-proxy-main" id=752f9457-1f9c-4826-86b0-404e72c1a748 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:46.827250 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.827223547Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:46.828435 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a.scope. Nov 16 14:50:46.856603 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a. Nov 16 14:50:46.858786 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.858712280Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634" id=50ce3fe9-de2f-4e8f-8fbb-1bcbe1602640 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.865760 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.865229528Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634 not found" id=50ce3fe9-de2f-4e8f-8fbb-1bcbe1602640 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:46.866189 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.866108434Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634" id=043e2429-1964-4da5-8773-23a1108d1e0b name=/runtime.v1.ImageService/PullImage Nov 16 14:50:46.869214 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:46.869166557Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634\"" Nov 16 14:50:46.941511 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:50:46.944724 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4.scope. Nov 16 14:50:46.977172 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4. Nov 16 14:50:47.013673 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:50:47.052808 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.052742211Z" level=info msg="Created container 83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/kube-rbac-proxy-self" id=8328aa54-acf1-4427-afb8-64bda489ba5a name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:47.053354 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.053319815Z" level=info msg="Starting container: 83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a" id=fd57e9e0-3b3b-401c-a301-a00f9fa020cb name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:50:47.067115 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.066971016Z" level=info msg="Started container" PID=178328 containerID=83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a description=openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/kube-rbac-proxy-self id=fd57e9e0-3b3b-401c-a301-a00f9fa020cb name=/runtime.v1.RuntimeService/StartContainer sandboxID=9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c Nov 16 14:50:47.090602 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.090552190Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2ca2dedab2746a1596dc2d65af5bec6d52be3f14afbbee3c7d2f654086f6917" id=8d1c3acf-4957-435d-b0c8-b169a04c71a9 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:47.090876 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.090843692Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2ca2dedab2746a1596dc2d65af5bec6d52be3f14afbbee3c7d2f654086f6917 not found" id=8d1c3acf-4957-435d-b0c8-b169a04c71a9 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:47.091723 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.091682898Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2ca2dedab2746a1596dc2d65af5bec6d52be3f14afbbee3c7d2f654086f6917" id=fb602b46-831f-430e-b824-e56807d4ec28 name=/runtime.v1.ImageService/PullImage Nov 16 14:50:47.094116 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.094039216Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2ca2dedab2746a1596dc2d65af5bec6d52be3f14afbbee3c7d2f654086f6917\"" Nov 16 14:50:47.115073 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.114981370Z" level=info msg="Created container 4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-rbac-proxy-main" id=752f9457-1f9c-4826-86b0-404e72c1a748 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:47.115669 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.115628475Z" level=info msg="Starting container: 4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4" id=b67cc0ef-fb49-4dfd-9e56-c4207ac10445 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:50:47.129197 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.129154475Z" level=info msg="Started container" PID=178358 containerID=4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4 description=openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-rbac-proxy-main id=b67cc0ef-fb49-4dfd-9e56-c4207ac10445 name=/runtime.v1.RuntimeService/StartContainer sandboxID=5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3 Nov 16 14:50:47.150729 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.150672333Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=a291795d-bca8-4782-a12d-34edbf00ad33 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:47.152646 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.152575947Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=a291795d-bca8-4782-a12d-34edbf00ad33 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:47.153626 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.153589655Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=15e7cb78-d1fa-4f94-8ed5-1b07c82861ed name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:47.155230 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.155188467Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=15e7cb78-d1fa-4f94-8ed5-1b07c82861ed name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:47.156298 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.156261074Z" level=info msg="Creating container: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-rbac-proxy-self" id=41b66fdf-9bd7-4838-a43c-d68b5737a3f1 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:47.156398 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.156365175Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:47.216405 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b.scope. Nov 16 14:50:47.246544 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b. Nov 16 14:50:47.291879 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:50:47.417473 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.417415501Z" level=info msg="Created container 6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-rbac-proxy-self" id=41b66fdf-9bd7-4838-a43c-d68b5737a3f1 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:47.418016 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.417975905Z" level=info msg="Starting container: 6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b" id=e5db143e-ef38-4f7b-91f2-52769067c887 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:50:47.431114 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.431069101Z" level=info msg="Started container" PID=178443 containerID=6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b description=openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-rbac-proxy-self id=e5db143e-ef38-4f7b-91f2-52769067c887 name=/runtime.v1.RuntimeService/StartContainer sandboxID=5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3 Nov 16 14:50:47.521597 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:47.521373 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-bjxz7] Nov 16 14:50:47.522181 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:50:47Z [verbose] Add: openshift-marketplace:qe-app-registry-bjxz7:f27359d7-25d3-44d2-a327-f7cbd09b1815:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/a40a8c16-8055-4d70-9c90-75c32bd7465c"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.34/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:50:47.522181 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:50:47.501271 177992 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-bjxz7", UID:"f27359d7-25d3-44d2-a327-f7cbd09b1815", APIVersion:"v1", ResourceVersion:"95668", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.34/23] from openshift-sdn Nov 16 14:50:47.522296 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.522268074Z" level=info msg="Got pod network &{Name:qe-app-registry-bjxz7 Namespace:openshift-marketplace ID:56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58 UID:f27359d7-25d3-44d2-a327-f7cbd09b1815 NetNS:/var/run/netns/a40a8c16-8055-4d70-9c90-75c32bd7465c Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:47.522478 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.522450775Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-bjxz7 for CNI network multus-cni-network (type=multus)" Nov 16 14:50:47.525275 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:47.525238 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf27359d7_25d3_44d2_a327_f7cbd09b1815.slice/crio-56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58.scope WatchSource:0}: Error finding container 56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58: Status 404 returned error &{%!s(*http.body=&{0xc002204ab0 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:50:47.533340 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.533303055Z" level=info msg="Ran pod sandbox 56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58 with infra container: openshift-marketplace/qe-app-registry-bjxz7/POD" id=cf00cf02-7c50-4430-a1ed-b1a11685cb23 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:47.534456 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.534416364Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=71b3b64e-3cb8-4c31-92b1-2cd88dfbfefe name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:47.534702 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.534663566Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=71b3b64e-3cb8-4c31-92b1-2cd88dfbfefe name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:47.535456 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.535392171Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=52c54e3c-2c9e-4f70-a87b-ca343076ddab name=/runtime.v1.ImageService/PullImage Nov 16 14:50:47.538190 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.538139991Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 14:50:47.544054 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:50:47Z [verbose] Add: openshift-image-registry:image-registry-944cf86d8-kb2fb:0520c563-12a4-4756-b022-6ec7396a5839:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/887d142c-2ca2-4563-b8cc-9b0e806e344e"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.35/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:50:47.544054 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:50:47.514714 178008 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-image-registry", Name:"image-registry-944cf86d8-kb2fb", UID:"0520c563-12a4-4756-b022-6ec7396a5839", APIVersion:"v1", ResourceVersion:"95884", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.35/23] from openshift-sdn Nov 16 14:50:47.544564 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.544452338Z" level=info msg="Got pod network &{Name:image-registry-944cf86d8-kb2fb Namespace:openshift-image-registry ID:4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e UID:0520c563-12a4-4756-b022-6ec7396a5839 NetNS:/var/run/netns/887d142c-2ca2-4563-b8cc-9b0e806e344e Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:47.544726 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.544683339Z" level=info msg="Checking pod openshift-image-registry_image-registry-944cf86d8-kb2fb for CNI network multus-cni-network (type=multus)" Nov 16 14:50:47.544846 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:47.544720 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-image-registry/image-registry-944cf86d8-kb2fb] Nov 16 14:50:47.547400 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:50:47.547369 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0520c563_12a4_4756_b022_6ec7396a5839.slice/crio-4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e.scope WatchSource:0}: Error finding container 4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e: Status 404 returned error &{%!s(*http.body=&{0xc002ac0258 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:50:47.548469 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.548414367Z" level=info msg="Ran pod sandbox 4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e with infra container: openshift-image-registry/image-registry-944cf86d8-kb2fb/POD" id=c2486f8a-7a9a-4d98-a88e-49e9fd16b1a0 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:47.549277 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.549242073Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d8e2c924572c5086c16e027955e69da349d7a6a75c086adb2d2252b6d87f8e2" id=5d642e2d-7f91-475a-b92b-3436c66f4cdb name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:47.549511 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.549482075Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d8e2c924572c5086c16e027955e69da349d7a6a75c086adb2d2252b6d87f8e2 not found" id=5d642e2d-7f91-475a-b92b-3436c66f4cdb name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:47.550303 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.550266281Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d8e2c924572c5086c16e027955e69da349d7a6a75c086adb2d2252b6d87f8e2" id=3e5a857a-f2f0-4fae-808f-c9d8fa86a6b3 name=/runtime.v1.ImageService/PullImage Nov 16 14:50:47.552829 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.552761199Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d8e2c924572c5086c16e027955e69da349d7a6a75c086adb2d2252b6d87f8e2\"" Nov 16 14:50:47.623691 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.623593921Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26611bedb7c02dbea85931612f83490e345de3f75644a914a70cb7fba2caa168\"" Nov 16 14:50:47.782827 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:47.782641 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" event=&{ID:c8013b4f-15b6-4bac-bfc9-a45301362f0c Type:ContainerStarted Data:83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a} Nov 16 14:50:47.783624 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:47.783592 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" event=&{ID:0520c563-12a4-4756-b022-6ec7396a5839 Type:ContainerStarted Data:4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e} Nov 16 14:50:47.785021 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:47.784962 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerStarted Data:94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d} Nov 16 14:50:47.785860 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:47.785787 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-bjxz7" event=&{ID:f27359d7-25d3-44d2-a327-f7cbd09b1815 Type:ContainerStarted Data:56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58} Nov 16 14:50:47.787541 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:47.787517 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" event=&{ID:454b3a37-199c-4abc-a79a-8de93eab2423 Type:ContainerStarted Data:d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610} Nov 16 14:50:47.787627 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:47.787553 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" event=&{ID:454b3a37-199c-4abc-a79a-8de93eab2423 Type:ContainerStarted Data:6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b} Nov 16 14:50:47.787627 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:47.787570 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" event=&{ID:454b3a37-199c-4abc-a79a-8de93eab2423 Type:ContainerStarted Data:4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4} Nov 16 14:50:47.904540 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:47.904441293Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3\"" Nov 16 14:50:48.112982 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:48.112894430Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634\"" Nov 16 14:50:48.347183 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:48.347028757Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2ca2dedab2746a1596dc2d65af5bec6d52be3f14afbbee3c7d2f654086f6917\"" Nov 16 14:50:48.802854 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:48.802772119Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d8e2c924572c5086c16e027955e69da349d7a6a75c086adb2d2252b6d87f8e2\"" Nov 16 14:50:49.146368 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:49.146276753Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=52c54e3c-2c9e-4f70-a87b-ca343076ddab name=/runtime.v1.ImageService/PullImage Nov 16 14:50:49.147383 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:49.147347861Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=0ce09911-3ff6-4f22-8a57-91e00c41d26a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:49.147626 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:49.147575762Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=0ce09911-3ff6-4f22-8a57-91e00c41d26a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:50:49.148840 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:49.148796171Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-bjxz7/registry-server" id=1cf114e8-63fc-4e76-88a2-a0ad2542c4c3 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:49.148932 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:49.148905272Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:50:49.221616 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c.scope. Nov 16 14:50:49.249073 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c. Nov 16 14:50:49.288618 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:50:49.364033 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:49.363962758Z" level=info msg="Created container 776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c: openshift-marketplace/qe-app-registry-bjxz7/registry-server" id=1cf114e8-63fc-4e76-88a2-a0ad2542c4c3 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:50:49.364779 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:49.364741264Z" level=info msg="Starting container: 776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c" id=68dadc20-4812-447d-8a04-2c94e84080f7 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:50:49.376506 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:49.376421750Z" level=info msg="Started container" PID=178546 containerID=776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c description=openshift-marketplace/qe-app-registry-bjxz7/registry-server id=68dadc20-4812-447d-8a04-2c94e84080f7 name=/runtime.v1.RuntimeService/StartContainer sandboxID=56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58 Nov 16 14:50:49.795731 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:49.795693 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-bjxz7" event=&{ID:f27359d7-25d3-44d2-a327-f7cbd09b1815 Type:ContainerStarted Data:776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c} Nov 16 14:50:50.788924 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:50.788832 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/node-exporter-dz75k] Nov 16 14:50:50.789232 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:50.789191 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/node-exporter-dz75k" podUID=5026bf33-68a7-4497-8c41-20a85e407ef9 containerName="kube-rbac-proxy" containerID="cri-o://cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697" gracePeriod=30 Nov 16 14:50:50.789533 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:50.789503 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/node-exporter-dz75k" podUID=5026bf33-68a7-4497-8c41-20a85e407ef9 containerName="node-exporter" containerID="cri-o://aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28" gracePeriod=30 Nov 16 14:50:50.789743 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:50.789665074Z" level=info msg="Stopping container: cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697 (timeout: 30s)" id=0ad2f9a2-9788-4aeb-b921-91a42ceec551 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:50:50.789743 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:50.789734175Z" level=info msg="Stopping container: aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28 (timeout: 30s)" id=6510b09f-44b8-46ac-9e43-517d07eff010 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:50:50.798414 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:50.798386 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-bjxz7" podUID=f27359d7-25d3-44d2-a327-f7cbd09b1815 containerName="registry-server" containerID="cri-o://776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c" gracePeriod=1 Nov 16 14:50:50.798790 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:50.798562840Z" level=info msg="Stopping container: 776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c (timeout: 1s)" id=ddf8e258-1f3c-4a3d-8600-7c30d85ac680 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:50:50.811144 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28.scope: Succeeded. Nov 16 14:50:50.811763 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28.scope: Consumed 36ms CPU time Nov 16 14:50:50.815500 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28.scope: Succeeded. Nov 16 14:50:50.816072 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28.scope: Consumed 1min 56.223s CPU time Nov 16 14:50:50.823767 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c.scope: Succeeded. Nov 16 14:50:50.824482 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c.scope: Consumed 666ms CPU time Nov 16 14:50:50.968922 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-85ddaf9fca23ad593438e4c55229ea1d04540cc6ba1b82f4eadd8f6adbd22d9a-merged.mount: Succeeded. Nov 16 14:50:50.969791 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-85ddaf9fca23ad593438e4c55229ea1d04540cc6ba1b82f4eadd8f6adbd22d9a-merged.mount: Consumed 0 CPU time Nov 16 14:50:51.714553 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c.scope: Succeeded. Nov 16 14:50:51.715289 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c.scope: Consumed 37ms CPU time Nov 16 14:50:51.802819 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:51.802482 1563 generic.go:296] "Generic (PLEG): container finished" podID=f27359d7-25d3-44d2-a327-f7cbd09b1815 containerID="776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c" exitCode=0 Nov 16 14:50:51.802819 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:51.802564 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-bjxz7" event=&{ID:f27359d7-25d3-44d2-a327-f7cbd09b1815 Type:ContainerDied Data:776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c} Nov 16 14:50:51.804759 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:51.804683 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dz75k_5026bf33-68a7-4497-8c41-20a85e407ef9/node-exporter/0.log" Nov 16 14:50:51.805110 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:51.805087 1563 generic.go:296] "Generic (PLEG): container finished" podID=5026bf33-68a7-4497-8c41-20a85e407ef9 containerID="aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28" exitCode=143 Nov 16 14:50:51.805197 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:51.805137 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dz75k" event=&{ID:5026bf33-68a7-4497-8c41-20a85e407ef9 Type:ContainerDied Data:aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28} Nov 16 14:50:51.899517 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697.scope: Succeeded. Nov 16 14:50:51.900268 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697.scope: Consumed 8.286s CPU time Nov 16 14:50:52.578107 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:52.577971 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/qe-app-registry-bjxz7" Nov 16 14:50:53.209320 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:53.209261 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-image-registry/node-ca-dn4b9] Nov 16 14:50:53.209948 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:53.209521 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-image-registry/node-ca-dn4b9" podUID=39a966fc-6573-4f04-9a9d-8cb766b277ba containerName="node-ca" containerID="cri-o://131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6" gracePeriod=30 Nov 16 14:50:53.210041 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:53.209983826Z" level=info msg="Stopping container: 131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6 (timeout: 30s)" id=2faabf68-6e88-4905-9cc8-bb564986a9f6 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:50:53.730761 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6.scope: Succeeded. Nov 16 14:50:53.731435 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6.scope: Consumed 4.475s CPU time Nov 16 14:50:53.904697 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697.scope: Succeeded. Nov 16 14:50:53.905445 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697.scope: Consumed 36ms CPU time Nov 16 14:50:54.364509 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6.scope: Succeeded. Nov 16 14:50:54.365202 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6.scope: Consumed 101ms CPU time Nov 16 14:50:54.488617 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:54.488486756Z" level=info msg="Stopped container aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28: openshift-monitoring/node-exporter-dz75k/node-exporter" id=6510b09f-44b8-46ac-9e43-517d07eff010 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:50:54.488754 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-8ab9d60b567878d924f2c28baff62061555ea0497e77b6236e0c55f29047458e-merged.mount: Succeeded. Nov 16 14:50:54.623404 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:54.623254350Z" level=info msg="Stopped container 776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c: openshift-marketplace/qe-app-registry-bjxz7/registry-server" id=ddf8e258-1f3c-4a3d-8600-7c30d85ac680 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:50:54.623821 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:54.623785654Z" level=info msg="Stopping pod sandbox: 56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58" id=12b83608-a171-4cab-b6ed-48204b95c612 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:54.624097 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:54.624064056Z" level=info msg="Got pod network &{Name:qe-app-registry-bjxz7 Namespace:openshift-marketplace ID:56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58 UID:f27359d7-25d3-44d2-a327-f7cbd09b1815 NetNS:/var/run/netns/a40a8c16-8055-4d70-9c90-75c32bd7465c Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:50:54.624822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:54.624790761Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-bjxz7 from CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:50:54.693405 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00206|connmgr|INFO|br0<->unix#454: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:50:54.727325 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00207|connmgr|INFO|br0<->unix#457: 4 flow_mods in the last 0 s (4 deletes) Nov 16 14:50:54.760786 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00208|bridge|INFO|bridge br0: deleted interface vethe79c7a8b on port 35 Nov 16 14:50:54.767159 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethe79c7a8b left promiscuous mode Nov 16 14:50:54.775954 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:50:54Z [verbose] Del: openshift-marketplace:qe-app-registry-bjxz7:f27359d7-25d3-44d2-a327-f7cbd09b1815:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 14:50:54.784294 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-a40a8c16\x2d8055\x2d4d70\x2d9c90\x2d75c32bd7465c.mount: Succeeded. Nov 16 14:50:54.791277 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-a40a8c16\x2d8055\x2d4d70\x2d9c90\x2d75c32bd7465c.mount: Succeeded. Nov 16 14:50:54.815484 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-a40a8c16\x2d8055\x2d4d70\x2d9c90\x2d75c32bd7465c.mount: Succeeded. Nov 16 14:50:54.815888 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:54.815858 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dz75k_5026bf33-68a7-4497-8c41-20a85e407ef9/node-exporter/0.log" Nov 16 14:50:54.816269 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:54.816220 1563 generic.go:296] "Generic (PLEG): container finished" podID=5026bf33-68a7-4497-8c41-20a85e407ef9 containerID="cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697" exitCode=0 Nov 16 14:50:54.816314 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:54.816287 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dz75k" event=&{ID:5026bf33-68a7-4497-8c41-20a85e407ef9 Type:ContainerDied Data:cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697} Nov 16 14:50:54.817501 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:54.817424 1563 generic.go:296] "Generic (PLEG): container finished" podID=39a966fc-6573-4f04-9a9d-8cb766b277ba containerID="131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6" exitCode=0 Nov 16 14:50:54.817501 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:54.817460 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dn4b9" event=&{ID:39a966fc-6573-4f04-9a9d-8cb766b277ba Type:ContainerDied Data:131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6} Nov 16 14:50:54.827404 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58-userdata-shm.mount: Succeeded. Nov 16 14:50:55.283154 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:55.283093516Z" level=info msg="Stopped container 131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6: openshift-image-registry/node-ca-dn4b9/node-ca" id=2faabf68-6e88-4905-9cc8-bb564986a9f6 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:50:55.283641 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:55.283607120Z" level=info msg="Stopping pod sandbox: e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e" id=82301d5c-0dc7-4fce-94bd-843b766ae50f name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:55.376073 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:55.376009502Z" level=info msg="Stopped container cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697: openshift-monitoring/node-exporter-dz75k/kube-rbac-proxy" id=0ad2f9a2-9788-4aeb-b921-91a42ceec551 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:50:55.376631 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:55.376597906Z" level=info msg="Stopping pod sandbox: 502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b" id=f4d81bc1-e453-4579-89bc-b8d4a0070e8c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:55.485568 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:55.485514109Z" level=info msg="Stopped pod sandbox: 56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58" id=12b83608-a171-4cab-b6ed-48204b95c612 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:55.488730 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-de42aa56d4b31ffde1ea3c26b0b2ff68169955d56ce16311a84add21d9317e97-merged.mount: Succeeded. Nov 16 14:50:55.488823 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-cb80a03668840b1570674f09617cc418cdff9d7c3fd164a693814ad5e1a40914-merged.mount: Succeeded. Nov 16 14:50:55.489713 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-cb80a03668840b1570674f09617cc418cdff9d7c3fd164a693814ad5e1a40914-merged.mount: Consumed 0 CPU time Nov 16 14:50:55.489732 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0720b1d91e4613804fec9318e077f228d154850dce69a45b9adbafefa65bb0fd-merged.mount: Succeeded. Nov 16 14:50:55.490419 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0720b1d91e4613804fec9318e077f228d154850dce69a45b9adbafefa65bb0fd-merged.mount: Consumed 0 CPU time Nov 16 14:50:55.490441 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-112b620d\x2d9466\x2d4065\x2d86b2\x2d2bcb22eb1383.mount: Succeeded. Nov 16 14:50:55.491088 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-112b620d\x2d9466\x2d4065\x2d86b2\x2d2bcb22eb1383.mount: Consumed 0 CPU time Nov 16 14:50:55.491104 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-9c72922c\x2df661\x2d4558\x2d95e9\x2dd0554bd1bf0a.mount: Succeeded. Nov 16 14:50:55.491771 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-9c72922c\x2df661\x2d4558\x2d95e9\x2dd0554bd1bf0a.mount: Consumed 0 CPU time Nov 16 14:50:55.491790 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-112b620d\x2d9466\x2d4065\x2d86b2\x2d2bcb22eb1383.mount: Succeeded. Nov 16 14:50:55.492453 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-112b620d\x2d9466\x2d4065\x2d86b2\x2d2bcb22eb1383.mount: Consumed 0 CPU time Nov 16 14:50:55.492471 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-9c72922c\x2df661\x2d4558\x2d95e9\x2dd0554bd1bf0a.mount: Succeeded. Nov 16 14:50:55.493107 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-9c72922c\x2df661\x2d4558\x2d95e9\x2dd0554bd1bf0a.mount: Consumed 0 CPU time Nov 16 14:50:55.493135 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-112b620d\x2d9466\x2d4065\x2d86b2\x2d2bcb22eb1383.mount: Succeeded. Nov 16 14:50:55.493754 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-112b620d\x2d9466\x2d4065\x2d86b2\x2d2bcb22eb1383.mount: Consumed 0 CPU time Nov 16 14:50:55.493780 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-9c72922c\x2df661\x2d4558\x2d95e9\x2dd0554bd1bf0a.mount: Succeeded. Nov 16 14:50:55.494466 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-9c72922c\x2df661\x2d4558\x2d95e9\x2dd0554bd1bf0a.mount: Consumed 0 CPU time Nov 16 14:50:55.494484 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b-userdata-shm.mount: Succeeded. Nov 16 14:50:55.495119 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b-userdata-shm.mount: Consumed 0 CPU time Nov 16 14:50:55.495145 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e-userdata-shm.mount: Succeeded. Nov 16 14:50:55.495773 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e-userdata-shm.mount: Consumed 0 CPU time Nov 16 14:50:55.674107 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:55.674018 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r28qw\" (UniqueName: \"kubernetes.io/projected/f27359d7-25d3-44d2-a327-f7cbd09b1815-kube-api-access-r28qw\") pod \"f27359d7-25d3-44d2-a327-f7cbd09b1815\" (UID: \"f27359d7-25d3-44d2-a327-f7cbd09b1815\") " Nov 16 14:50:55.686271 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-f27359d7\x2d25d3\x2d44d2\x2da327\x2df7cbd09b1815-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dr28qw.mount: Succeeded. Nov 16 14:50:55.694759 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:55.694709 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f27359d7-25d3-44d2-a327-f7cbd09b1815-kube-api-access-r28qw" (OuterVolumeSpecName: "kube-api-access-r28qw") pod "f27359d7-25d3-44d2-a327-f7cbd09b1815" (UID: "f27359d7-25d3-44d2-a327-f7cbd09b1815"). InnerVolumeSpecName "kube-api-access-r28qw". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:50:55.775832 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:55.775788 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-r28qw\" (UniqueName: \"kubernetes.io/projected/f27359d7-25d3-44d2-a327-f7cbd09b1815-kube-api-access-r28qw\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:50:55.822012 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:55.821975 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-bjxz7" event=&{ID:f27359d7-25d3-44d2-a327-f7cbd09b1815 Type:ContainerDied Data:56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58} Nov 16 14:50:55.822012 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:55.822032 1563 scope.go:110] "RemoveContainer" containerID="776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c" Nov 16 14:50:55.823182 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:55.823140799Z" level=info msg="Removing container: 776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c" id=3b9b44c3-9fbe-4238-a552-9a479fe7bba0 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:50:55.829675 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-podf27359d7_25d3_44d2_a327_f7cbd09b1815.slice. Nov 16 14:50:55.829743 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-podf27359d7_25d3_44d2_a327_f7cbd09b1815.slice: Consumed 704ms CPU time Nov 16 14:50:55.858079 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:55.858035 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-bjxz7] Nov 16 14:50:55.864137 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:55.864045 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-bjxz7] Nov 16 14:50:56.593150 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:56.593090278Z" level=info msg="Stopping pod sandbox: 56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58" id=a96a94ab-2699-4cf8-ba24-7646563e2240 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:56.593150 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:56.593167078Z" level=info msg="Stopped pod sandbox (already stopped): 56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58" id=a96a94ab-2699-4cf8-ba24-7646563e2240 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:50:56.594932 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:56.594864 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=f27359d7-25d3-44d2-a327-f7cbd09b1815 path="/var/lib/kubelet/pods/f27359d7-25d3-44d2-a327-f7cbd09b1815/volumes" Nov 16 14:50:57.632483 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-5219e98cb6f8649fa33f7bda279b28f6c29450bb2041ffb3c01eaee219b77e9c-merged.mount: Succeeded. Nov 16 14:50:57.633534 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-5219e98cb6f8649fa33f7bda279b28f6c29450bb2041ffb3c01eaee219b77e9c-merged.mount: Consumed 0 CPU time Nov 16 14:50:58.315809 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:58.315686 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-fh97d] Nov 16 14:50:58.316362 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:58.315986 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-fh97d" podUID=590da926-dc2c-4378-a471-69dd415ae65f containerName="registry-server" containerID="cri-o://9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f" gracePeriod=1 Nov 16 14:50:58.316413 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:58.316344487Z" level=info msg="Stopping container: 9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f (timeout: 1s)" id=d380cd3f-7547-46c2-9b6c-f57a5057e523 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:50:58.346465 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f.scope: Succeeded. Nov 16 14:50:58.347177 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f.scope: Consumed 1.781s CPU time Nov 16 14:50:58.349854 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:58.349788 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-marketplace/qe-app-registry-9hthz] Nov 16 14:50:58.349854 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:58.349857 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:50:58.349994 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:50:58.349956 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="ba736212-a121-4f49-a957-3f6bdc1c986d" containerName="registry-server" Nov 16 14:50:58.349994 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:58.349970 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba736212-a121-4f49-a957-3f6bdc1c986d" containerName="registry-server" Nov 16 14:50:58.349994 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:50:58.349982 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="f27359d7-25d3-44d2-a327-f7cbd09b1815" containerName="registry-server" Nov 16 14:50:58.349994 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:58.349990 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f27359d7-25d3-44d2-a327-f7cbd09b1815" containerName="registry-server" Nov 16 14:50:58.350115 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:58.350073 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="ba736212-a121-4f49-a957-3f6bdc1c986d" containerName="registry-server" Nov 16 14:50:58.350115 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:58.350087 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="f27359d7-25d3-44d2-a327-f7cbd09b1815" containerName="registry-server" Nov 16 14:50:58.360087 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod5eb5be5f_46bd_425a_a486_e2641bb0ad88.slice. Nov 16 14:50:58.499831 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:58.499743 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxtpb\" (UniqueName: \"kubernetes.io/projected/5eb5be5f-46bd-425a-a486-e2641bb0ad88-kube-api-access-mxtpb\") pod \"qe-app-registry-9hthz\" (UID: \"5eb5be5f-46bd-425a-a486-e2641bb0ad88\") " pod="openshift-marketplace/qe-app-registry-9hthz" Nov 16 14:50:58.600217 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:58.600182 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-mxtpb\" (UniqueName: \"kubernetes.io/projected/5eb5be5f-46bd-425a-a486-e2641bb0ad88-kube-api-access-mxtpb\") pod \"qe-app-registry-9hthz\" (UID: \"5eb5be5f-46bd-425a-a486-e2641bb0ad88\") " pod="openshift-marketplace/qe-app-registry-9hthz" Nov 16 14:50:58.625525 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:58.625421 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxtpb\" (UniqueName: \"kubernetes.io/projected/5eb5be5f-46bd-425a-a486-e2641bb0ad88-kube-api-access-mxtpb\") pod \"qe-app-registry-9hthz\" (UID: \"5eb5be5f-46bd-425a-a486-e2641bb0ad88\") " pod="openshift-marketplace/qe-app-registry-9hthz" Nov 16 14:50:58.675430 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:50:58.675269 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/qe-app-registry-9hthz" Nov 16 14:50:58.676109 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:58.676043740Z" level=info msg="Running pod sandbox: openshift-marketplace/qe-app-registry-9hthz/POD" id=c98924fa-2be1-460f-855d-8ce29c3bdb2a name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:50:58.676232 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:50:58.676143141Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:00.408952 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:00.408910 1563 remote_runtime.go:704] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f is running failed: open /proc/175326/stat: no such file or directory: container process not found" containerID="9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f" cmd=[grpc_health_probe -addr=:50051] Nov 16 14:51:00.409468 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:00.409324 1563 remote_runtime.go:704] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f is running failed: open /proc/175326/stat: no such file or directory: container process not found" containerID="9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f" cmd=[grpc_health_probe -addr=:50051] Nov 16 14:51:00.409729 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:00.409701 1563 remote_runtime.go:704] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f is running failed: open /proc/175326/stat: no such file or directory: container process not found" containerID="9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f" cmd=[grpc_health_probe -addr=:50051] Nov 16 14:51:00.409796 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:00.409752 1563 prober.go:118] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f is running failed: open /proc/175326/stat: no such file or directory: container process not found" probeType="Readiness" pod="openshift-marketplace/qe-app-registry-fh97d" podUID=590da926-dc2c-4378-a471-69dd415ae65f containerName="registry-server" Nov 16 14:51:01.426507 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f.scope: Succeeded. Nov 16 14:51:01.427148 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f.scope: Consumed 43ms CPU time Nov 16 14:51:01.469206 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:01.469153239Z" level=info msg="Stopped pod sandbox: e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e" id=82301d5c-0dc7-4fce-94bd-843b766ae50f name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:01.476575 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.DuImrN.mount: Succeeded. Nov 16 14:51:01.488754 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-3cbdadb794eda9af2981f82b508e110930758dd6bae49618fc5711b6fc24f37e-merged.mount: Succeeded. Nov 16 14:51:01.496968 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-3cbdadb794eda9af2981f82b508e110930758dd6bae49618fc5711b6fc24f37e-merged.mount: Consumed 0 CPU time Nov 16 14:51:01.620961 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:01.620867958Z" level=info msg="Stopped pod sandbox: 502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b" id=f4d81bc1-e453-4579-89bc-b8d4a0070e8c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:01.629314 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.629249 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dz75k_5026bf33-68a7-4497-8c41-20a85e407ef9/node-exporter/0.log" Nov 16 14:51:01.630190 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.630149 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39a966fc-6573-4f04-9a9d-8cb766b277ba-host\") pod \"39a966fc-6573-4f04-9a9d-8cb766b277ba\" (UID: \"39a966fc-6573-4f04-9a9d-8cb766b277ba\") " Nov 16 14:51:01.630282 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.630212 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/39a966fc-6573-4f04-9a9d-8cb766b277ba-serviceca\") pod \"39a966fc-6573-4f04-9a9d-8cb766b277ba\" (UID: \"39a966fc-6573-4f04-9a9d-8cb766b277ba\") " Nov 16 14:51:01.630282 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.630247 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pc6n8\" (UniqueName: \"kubernetes.io/projected/39a966fc-6573-4f04-9a9d-8cb766b277ba-kube-api-access-pc6n8\") pod \"39a966fc-6573-4f04-9a9d-8cb766b277ba\" (UID: \"39a966fc-6573-4f04-9a9d-8cb766b277ba\") " Nov 16 14:51:01.630282 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.630254 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39a966fc-6573-4f04-9a9d-8cb766b277ba-host" (OuterVolumeSpecName: "host") pod "39a966fc-6573-4f04-9a9d-8cb766b277ba" (UID: "39a966fc-6573-4f04-9a9d-8cb766b277ba"). InnerVolumeSpecName "host". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:51:01.630391 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.630360 1563 reconciler.go:300] "Volume detached for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39a966fc-6573-4f04-9a9d-8cb766b277ba-host\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:01.630637 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:51:01.630610 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/39a966fc-6573-4f04-9a9d-8cb766b277ba/volumes/kubernetes.io~configmap/serviceca: clearQuota called, but quotas disabled Nov 16 14:51:01.631064 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.630976 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39a966fc-6573-4f04-9a9d-8cb766b277ba-serviceca" (OuterVolumeSpecName: "serviceca") pod "39a966fc-6573-4f04-9a9d-8cb766b277ba" (UID: "39a966fc-6573-4f04-9a9d-8cb766b277ba"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 14:51:01.640610 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-39a966fc\x2d6573\x2d4f04\x2d9a9d\x2d8cb766b277ba-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dpc6n8.mount: Succeeded. Nov 16 14:51:01.641494 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-39a966fc\x2d6573\x2d4f04\x2d9a9d\x2d8cb766b277ba-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dpc6n8.mount: Consumed 0 CPU time Nov 16 14:51:01.655491 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.655434 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39a966fc-6573-4f04-9a9d-8cb766b277ba-kube-api-access-pc6n8" (OuterVolumeSpecName: "kube-api-access-pc6n8") pod "39a966fc-6573-4f04-9a9d-8cb766b277ba" (UID: "39a966fc-6573-4f04-9a9d-8cb766b277ba"). InnerVolumeSpecName "kube-api-access-pc6n8". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:51:01.731494 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731291 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-textfile\") pod \"5026bf33-68a7-4497-8c41-20a85e407ef9\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " Nov 16 14:51:01.731494 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731364 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-root\") pod \"5026bf33-68a7-4497-8c41-20a85e407ef9\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " Nov 16 14:51:01.731494 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731431 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5026bf33-68a7-4497-8c41-20a85e407ef9-metrics-client-ca\") pod \"5026bf33-68a7-4497-8c41-20a85e407ef9\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " Nov 16 14:51:01.731494 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731458 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-tls\") pod \"5026bf33-68a7-4497-8c41-20a85e407ef9\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " Nov 16 14:51:01.731494 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731500 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-wtmp\") pod \"5026bf33-68a7-4497-8c41-20a85e407ef9\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " Nov 16 14:51:01.731792 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731506 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-root" (OuterVolumeSpecName: "root") pod "5026bf33-68a7-4497-8c41-20a85e407ef9" (UID: "5026bf33-68a7-4497-8c41-20a85e407ef9"). InnerVolumeSpecName "root". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:51:01.731792 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731551 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5b5s\" (UniqueName: \"kubernetes.io/projected/5026bf33-68a7-4497-8c41-20a85e407ef9-kube-api-access-w5b5s\") pod \"5026bf33-68a7-4497-8c41-20a85e407ef9\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " Nov 16 14:51:01.731792 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731578 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-wtmp" (OuterVolumeSpecName: "node-exporter-wtmp") pod "5026bf33-68a7-4497-8c41-20a85e407ef9" (UID: "5026bf33-68a7-4497-8c41-20a85e407ef9"). InnerVolumeSpecName "node-exporter-wtmp". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:51:01.731792 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731582 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-kube-rbac-proxy-config\") pod \"5026bf33-68a7-4497-8c41-20a85e407ef9\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " Nov 16 14:51:01.731792 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:51:01.731631 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/5026bf33-68a7-4497-8c41-20a85e407ef9/volumes/kubernetes.io~empty-dir/node-exporter-textfile: clearQuota called, but quotas disabled Nov 16 14:51:01.731792 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731682 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-sys" (OuterVolumeSpecName: "sys") pod "5026bf33-68a7-4497-8c41-20a85e407ef9" (UID: "5026bf33-68a7-4497-8c41-20a85e407ef9"). InnerVolumeSpecName "sys". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:51:01.731792 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731655 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-sys\") pod \"5026bf33-68a7-4497-8c41-20a85e407ef9\" (UID: \"5026bf33-68a7-4497-8c41-20a85e407ef9\") " Nov 16 14:51:01.731792 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:51:01.731745 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/5026bf33-68a7-4497-8c41-20a85e407ef9/volumes/kubernetes.io~configmap/metrics-client-ca: clearQuota called, but quotas disabled Nov 16 14:51:01.732032 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731803 1563 reconciler.go:300] "Volume detached for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-sys\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:01.732032 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731821 1563 reconciler.go:300] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/39a966fc-6573-4f04-9a9d-8cb766b277ba-serviceca\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:01.732032 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731838 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-pc6n8\" (UniqueName: \"kubernetes.io/projected/39a966fc-6573-4f04-9a9d-8cb766b277ba-kube-api-access-pc6n8\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:01.732032 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731852 1563 reconciler.go:300] "Volume detached for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-root\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:01.732032 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731919 1563 reconciler.go:300] "Volume detached for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-wtmp\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:01.732032 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731960 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5026bf33-68a7-4497-8c41-20a85e407ef9-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "5026bf33-68a7-4497-8c41-20a85e407ef9" (UID: "5026bf33-68a7-4497-8c41-20a85e407ef9"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 14:51:01.732032 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.731982 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-textfile" (OuterVolumeSpecName: "node-exporter-textfile") pod "5026bf33-68a7-4497-8c41-20a85e407ef9" (UID: "5026bf33-68a7-4497-8c41-20a85e407ef9"). InnerVolumeSpecName "node-exporter-textfile". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Nov 16 14:51:01.741487 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-5026bf33\x2d68a7\x2d4497\x2d8c41\x2d20a85e407ef9-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dw5b5s.mount: Succeeded. Nov 16 14:51:01.742554 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-5026bf33\x2d68a7\x2d4497\x2d8c41\x2d20a85e407ef9-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dw5b5s.mount: Consumed 0 CPU time Nov 16 14:51:01.742581 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-5026bf33\x2d68a7\x2d4497\x2d8c41\x2d20a85e407ef9-volumes-kubernetes.io\x7esecret-node\x2dexporter\x2dtls.mount: Succeeded. Nov 16 14:51:01.743677 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-5026bf33\x2d68a7\x2d4497\x2d8c41\x2d20a85e407ef9-volumes-kubernetes.io\x7esecret-node\x2dexporter\x2dtls.mount: Consumed 0 CPU time Nov 16 14:51:01.743701 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-5026bf33\x2d68a7\x2d4497\x2d8c41\x2d20a85e407ef9-volumes-kubernetes.io\x7esecret-node\x2dexporter\x2dkube\x2drbac\x2dproxy\x2dconfig.mount: Succeeded. Nov 16 14:51:01.744531 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-5026bf33\x2d68a7\x2d4497\x2d8c41\x2d20a85e407ef9-volumes-kubernetes.io\x7esecret-node\x2dexporter\x2dkube\x2drbac\x2dproxy\x2dconfig.mount: Consumed 0 CPU time Nov 16 14:51:01.752438 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.752384 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-kube-rbac-proxy-config" (OuterVolumeSpecName: "node-exporter-kube-rbac-proxy-config") pod "5026bf33-68a7-4497-8c41-20a85e407ef9" (UID: "5026bf33-68a7-4497-8c41-20a85e407ef9"). InnerVolumeSpecName "node-exporter-kube-rbac-proxy-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 14:51:01.753708 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.753662 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5026bf33-68a7-4497-8c41-20a85e407ef9-kube-api-access-w5b5s" (OuterVolumeSpecName: "kube-api-access-w5b5s") pod "5026bf33-68a7-4497-8c41-20a85e407ef9" (UID: "5026bf33-68a7-4497-8c41-20a85e407ef9"). InnerVolumeSpecName "kube-api-access-w5b5s". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:51:01.760404 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.760365 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-tls" (OuterVolumeSpecName: "node-exporter-tls") pod "5026bf33-68a7-4497-8c41-20a85e407ef9" (UID: "5026bf33-68a7-4497-8c41-20a85e407ef9"). InnerVolumeSpecName "node-exporter-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 14:51:01.832839 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.832717 1563 reconciler.go:300] "Volume detached for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:01.832839 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.832760 1563 reconciler.go:300] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5026bf33-68a7-4497-8c41-20a85e407ef9-metrics-client-ca\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:01.832839 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.832780 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-w5b5s\" (UniqueName: \"kubernetes.io/projected/5026bf33-68a7-4497-8c41-20a85e407ef9-kube-api-access-w5b5s\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:01.832839 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.832798 1563 reconciler.go:300] "Volume detached for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-kube-rbac-proxy-config\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:01.832839 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.832815 1563 reconciler.go:300] "Volume detached for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5026bf33-68a7-4497-8c41-20a85e407ef9-node-exporter-textfile\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:01.842349 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.842322 1563 generic.go:296] "Generic (PLEG): container finished" podID=590da926-dc2c-4378-a471-69dd415ae65f containerID="9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f" exitCode=0 Nov 16 14:51:01.842436 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.842419 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-fh97d" event=&{ID:590da926-dc2c-4378-a471-69dd415ae65f Type:ContainerDied Data:9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f} Nov 16 14:51:01.844930 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.844870 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dz75k_5026bf33-68a7-4497-8c41-20a85e407ef9/node-exporter/0.log" Nov 16 14:51:01.845226 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.845202 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dz75k" event=&{ID:5026bf33-68a7-4497-8c41-20a85e407ef9 Type:ContainerDied Data:502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b} Nov 16 14:51:01.850511 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.850473 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dn4b9" event=&{ID:39a966fc-6573-4f04-9a9d-8cb766b277ba Type:ContainerDied Data:e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e} Nov 16 14:51:01.854202 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod5026bf33_68a7_4497_8c41_20a85e407ef9.slice. Nov 16 14:51:01.854273 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod5026bf33_68a7_4497_8c41_20a85e407ef9.slice: Consumed 2min 4.787s CPU time Nov 16 14:51:01.858121 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod39a966fc_6573_4f04_9a9d_8cb766b277ba.slice. Nov 16 14:51:01.858189 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod39a966fc_6573_4f04_9a9d_8cb766b277ba.slice: Consumed 4.576s CPU time Nov 16 14:51:01.898005 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.897963 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/node-exporter-dz75k] Nov 16 14:51:01.903011 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.902948 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-monitoring/node-exporter-dz75k] Nov 16 14:51:01.930805 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.930725 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-image-registry/node-ca-dn4b9] Nov 16 14:51:01.937706 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.937629 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-image-registry/node-ca-dn4b9] Nov 16 14:51:01.953092 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.953007 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-monitoring/node-exporter-kxmx6] Nov 16 14:51:01.953092 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.953085 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:51:01.953307 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:01.953233 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="5026bf33-68a7-4497-8c41-20a85e407ef9" containerName="init-textfile" Nov 16 14:51:01.953307 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.953248 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5026bf33-68a7-4497-8c41-20a85e407ef9" containerName="init-textfile" Nov 16 14:51:01.953307 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:01.953264 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="39a966fc-6573-4f04-9a9d-8cb766b277ba" containerName="node-ca" Nov 16 14:51:01.953307 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.953281 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a966fc-6573-4f04-9a9d-8cb766b277ba" containerName="node-ca" Nov 16 14:51:01.953307 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:01.953290 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="5026bf33-68a7-4497-8c41-20a85e407ef9" containerName="node-exporter" Nov 16 14:51:01.953307 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.953298 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5026bf33-68a7-4497-8c41-20a85e407ef9" containerName="node-exporter" Nov 16 14:51:01.953307 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:01.953309 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="5026bf33-68a7-4497-8c41-20a85e407ef9" containerName="kube-rbac-proxy" Nov 16 14:51:01.953529 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.953320 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5026bf33-68a7-4497-8c41-20a85e407ef9" containerName="kube-rbac-proxy" Nov 16 14:51:01.953529 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.953397 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="39a966fc-6573-4f04-9a9d-8cb766b277ba" containerName="node-ca" Nov 16 14:51:01.953529 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.953417 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="5026bf33-68a7-4497-8c41-20a85e407ef9" containerName="node-exporter" Nov 16 14:51:01.953529 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.953429 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="5026bf33-68a7-4497-8c41-20a85e407ef9" containerName="kube-rbac-proxy" Nov 16 14:51:01.964380 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod89cfb802_14c7_4c79_bc9c_015efafee773.slice. Nov 16 14:51:01.980526 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.980485 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-image-registry/node-ca-dg87v] Nov 16 14:51:01.980526 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:01.980559 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:51:01.992954 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-podaacb53e4_6716_4337_a1ae_4ac3a857cbb0.slice. Nov 16 14:51:02.134972 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.134924 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aacb53e4-6716-4337-a1ae-4ac3a857cbb0-serviceca\") pod \"node-ca-dg87v\" (UID: \"aacb53e4-6716-4337-a1ae-4ac3a857cbb0\") " pod="openshift-image-registry/node-ca-dg87v" Nov 16 14:51:02.135189 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.134998 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89cfb802-14c7-4c79-bc9c-015efafee773-metrics-client-ca\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.135189 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.135035 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9784\" (UniqueName: \"kubernetes.io/projected/aacb53e4-6716-4337-a1ae-4ac3a857cbb0-kube-api-access-g9784\") pod \"node-ca-dg87v\" (UID: \"aacb53e4-6716-4337-a1ae-4ac3a857cbb0\") " pod="openshift-image-registry/node-ca-dg87v" Nov 16 14:51:02.135189 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.135064 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/89cfb802-14c7-4c79-bc9c-015efafee773-node-exporter-tls\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.135189 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.135106 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/89cfb802-14c7-4c79-bc9c-015efafee773-node-exporter-wtmp\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.135349 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.135226 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89cfb802-14c7-4c79-bc9c-015efafee773-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.135349 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.135329 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89cfb802-14c7-4c79-bc9c-015efafee773-sys\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.135429 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.135369 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh8rw\" (UniqueName: \"kubernetes.io/projected/89cfb802-14c7-4c79-bc9c-015efafee773-kube-api-access-rh8rw\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.135429 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.135408 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aacb53e4-6716-4337-a1ae-4ac3a857cbb0-host\") pod \"node-ca-dg87v\" (UID: \"aacb53e4-6716-4337-a1ae-4ac3a857cbb0\") " pod="openshift-image-registry/node-ca-dg87v" Nov 16 14:51:02.135497 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.135480 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/89cfb802-14c7-4c79-bc9c-015efafee773-node-exporter-textfile\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.135580 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.135546 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/89cfb802-14c7-4c79-bc9c-015efafee773-root\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.236100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236039 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aacb53e4-6716-4337-a1ae-4ac3a857cbb0-serviceca\") pod \"node-ca-dg87v\" (UID: \"aacb53e4-6716-4337-a1ae-4ac3a857cbb0\") " pod="openshift-image-registry/node-ca-dg87v" Nov 16 14:51:02.236100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236107 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89cfb802-14c7-4c79-bc9c-015efafee773-metrics-client-ca\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.236350 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236180 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-g9784\" (UniqueName: \"kubernetes.io/projected/aacb53e4-6716-4337-a1ae-4ac3a857cbb0-kube-api-access-g9784\") pod \"node-ca-dg87v\" (UID: \"aacb53e4-6716-4337-a1ae-4ac3a857cbb0\") " pod="openshift-image-registry/node-ca-dg87v" Nov 16 14:51:02.236350 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236216 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/89cfb802-14c7-4c79-bc9c-015efafee773-node-exporter-tls\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.236350 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236257 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/89cfb802-14c7-4c79-bc9c-015efafee773-node-exporter-wtmp\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.236350 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236294 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89cfb802-14c7-4c79-bc9c-015efafee773-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.236350 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236333 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89cfb802-14c7-4c79-bc9c-015efafee773-sys\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.236507 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236365 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-rh8rw\" (UniqueName: \"kubernetes.io/projected/89cfb802-14c7-4c79-bc9c-015efafee773-kube-api-access-rh8rw\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.236507 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236393 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aacb53e4-6716-4337-a1ae-4ac3a857cbb0-host\") pod \"node-ca-dg87v\" (UID: \"aacb53e4-6716-4337-a1ae-4ac3a857cbb0\") " pod="openshift-image-registry/node-ca-dg87v" Nov 16 14:51:02.236507 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236424 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/89cfb802-14c7-4c79-bc9c-015efafee773-node-exporter-textfile\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.236507 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236455 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/89cfb802-14c7-4c79-bc9c-015efafee773-root\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.236507 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236480 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/89cfb802-14c7-4c79-bc9c-015efafee773-node-exporter-wtmp\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.236714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236531 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/89cfb802-14c7-4c79-bc9c-015efafee773-root\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.236714 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236578 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aacb53e4-6716-4337-a1ae-4ac3a857cbb0-host\") pod \"node-ca-dg87v\" (UID: \"aacb53e4-6716-4337-a1ae-4ac3a857cbb0\") " pod="openshift-image-registry/node-ca-dg87v" Nov 16 14:51:02.236813 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236800 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89cfb802-14c7-4c79-bc9c-015efafee773-sys\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.236852 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236813 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89cfb802-14c7-4c79-bc9c-015efafee773-metrics-client-ca\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.236957 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236928 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aacb53e4-6716-4337-a1ae-4ac3a857cbb0-serviceca\") pod \"node-ca-dg87v\" (UID: \"aacb53e4-6716-4337-a1ae-4ac3a857cbb0\") " pod="openshift-image-registry/node-ca-dg87v" Nov 16 14:51:02.237008 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.236993 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/89cfb802-14c7-4c79-bc9c-015efafee773-node-exporter-textfile\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.239213 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.239183 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89cfb802-14c7-4c79-bc9c-015efafee773-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.239396 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.239370 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/89cfb802-14c7-4c79-bc9c-015efafee773-node-exporter-tls\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.259471 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.259332 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9784\" (UniqueName: \"kubernetes.io/projected/aacb53e4-6716-4337-a1ae-4ac3a857cbb0-kube-api-access-g9784\") pod \"node-ca-dg87v\" (UID: \"aacb53e4-6716-4337-a1ae-4ac3a857cbb0\") " pod="openshift-image-registry/node-ca-dg87v" Nov 16 14:51:02.259615 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.259473 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh8rw\" (UniqueName: \"kubernetes.io/projected/89cfb802-14c7-4c79-bc9c-015efafee773-kube-api-access-rh8rw\") pod \"node-exporter-kxmx6\" (UID: \"89cfb802-14c7-4c79-bc9c-015efafee773\") " pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.283712 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.283646 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kxmx6" Nov 16 14:51:02.284356 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:02.284281651Z" level=info msg="Running pod sandbox: openshift-monitoring/node-exporter-kxmx6/POD" id=d1511648-ddc8-45b9-a39a-7f7e0716703e name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:51:02.284474 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:02.284376551Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:02.297771 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.297727 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dg87v" Nov 16 14:51:02.298390 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:02.298319054Z" level=info msg="Running pod sandbox: openshift-image-registry/node-ca-dg87v/POD" id=21f0b9cb-7e5c-4e04-8a58-8d0fd73aa531 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:51:02.298390 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:02.298423955Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:02.597098 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.597018 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=39a966fc-6573-4f04-9a9d-8cb766b277ba path="/var/lib/kubelet/pods/39a966fc-6573-4f04-9a9d-8cb766b277ba/volumes" Nov 16 14:51:02.597725 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:02.597669 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=5026bf33-68a7-4497-8c41-20a85e407ef9 path="/var/lib/kubelet/pods/5026bf33-68a7-4497-8c41-20a85e407ef9/volumes" Nov 16 14:51:02.709345 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-045c8408e50f7a9d78d48a1979629a568239636163ccd1a8a4f53e7b2793c135-merged.mount: Succeeded. Nov 16 14:51:04.183923 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.183855752Z" level=info msg="Stopped container 9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f: openshift-marketplace/qe-app-registry-fh97d/registry-server" id=d380cd3f-7547-46c2-9b6c-f57a5057e523 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:51:04.184482 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.184446657Z" level=info msg="Stopping pod sandbox: eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5" id=ee1659ce-9002-49ca-b3bd-1e39a5adf798 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:04.184762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.184729259Z" level=info msg="Got pod network &{Name:qe-app-registry-fh97d Namespace:openshift-marketplace ID:eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5 UID:590da926-dc2c-4378-a471-69dd415ae65f NetNS:/var/run/netns/1e5df66f-d601-4db1-9601-458acfac495e Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:51:04.184909 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.184868960Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-fh97d from CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:51:04.188760 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.188681988Z" level=info msg="Removed container 776bf634969672681bf7dea1f6750bbd4312c23c2a150e395504f639d4bb9f9c: openshift-marketplace/qe-app-registry-bjxz7/registry-server" id=3b9b44c3-9fbe-4238-a552-9a479fe7bba0 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:04.188912 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:04.188882 1563 scope.go:110] "RemoveContainer" containerID="cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697" Nov 16 14:51:04.189968 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.189926997Z" level=info msg="Removing container: cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697" id=ca423d95-ac85-49fc-ac92-e8b117ccbc52 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:04.264980 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00209|connmgr|INFO|br0<->unix#460: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:51:04.301790 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00210|connmgr|INFO|br0<->unix#463: 4 flow_mods in the last 0 s (4 deletes) Nov 16 14:51:04.334574 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00211|bridge|INFO|bridge br0: deleted interface veth6aafb6eb on port 29 Nov 16 14:51:04.341169 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth6aafb6eb left promiscuous mode Nov 16 14:51:04.349571 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:51:04Z [verbose] Del: openshift-marketplace:qe-app-registry-fh97d:590da926-dc2c-4378-a471-69dd415ae65f:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 14:51:04.357344 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-1e5df66f\x2dd601\x2d4db1\x2d9601\x2d458acfac495e.mount: Succeeded. Nov 16 14:51:04.364239 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-1e5df66f\x2dd601\x2d4db1\x2d9601\x2d458acfac495e.mount: Succeeded. Nov 16 14:51:04.387356 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-1e5df66f\x2dd601\x2d4db1\x2d9601\x2d458acfac495e.mount: Succeeded. Nov 16 14:51:04.398780 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5-userdata-shm.mount: Succeeded. Nov 16 14:51:04.544200 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.543939807Z" level=info msg="Removed container cb67891923a59ee8b02ad9f99eeae14f9db22dcd7689356203294c83cd2f3697: openshift-monitoring/node-exporter-dz75k/kube-rbac-proxy" id=ca423d95-ac85-49fc-ac92-e8b117ccbc52 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:04.544362 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:04.544327 1563 scope.go:110] "RemoveContainer" containerID="aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28" Nov 16 14:51:04.545608 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.545538318Z" level=info msg="Removing container: aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28" id=bbea627c-7ea3-45e9-a809-b57880494d24 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:04.871780 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.871734923Z" level=info msg="Stopped pod sandbox: eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5" id=ee1659ce-9002-49ca-b3bd-1e39a5adf798 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:04.902617 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.902570450Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3" id=7dd73ede-26a9-4513-8afc-66271c942802 name=/runtime.v1.ImageService/PullImage Nov 16 14:51:04.903791 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.903724958Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3" id=2cf885af-68ee-4e1b-a2c3-7cf0f66dbda6 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:04.956722 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.956622448Z" level=info msg="Removed container aa5c2025e6eb66c0d5b1083c1d57cd625b30293cce18caa938b91b473b80fb28: openshift-monitoring/node-exporter-dz75k/node-exporter" id=bbea627c-7ea3-45e9-a809-b57880494d24 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:04.957037 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:04.957005 1563 scope.go:110] "RemoveContainer" containerID="9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb" Nov 16 14:51:04.958102 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.958042659Z" level=info msg="Removing container: 9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb" id=84efea37-d309-4fa0-aec1-3e642b4e4052 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:04.959536 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:04.959500 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4srsm\" (UniqueName: \"kubernetes.io/projected/590da926-dc2c-4378-a471-69dd415ae65f-kube-api-access-4srsm\") pod \"590da926-dc2c-4378-a471-69dd415ae65f\" (UID: \"590da926-dc2c-4378-a471-69dd415ae65f\") " Nov 16 14:51:04.974170 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:04.974055 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/590da926-dc2c-4378-a471-69dd415ae65f-kube-api-access-4srsm" (OuterVolumeSpecName: "kube-api-access-4srsm") pod "590da926-dc2c-4378-a471-69dd415ae65f" (UID: "590da926-dc2c-4378-a471-69dd415ae65f"). InnerVolumeSpecName "kube-api-access-4srsm". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:51:04.974951 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.974871183Z" level=info msg="Got pod network &{Name:qe-app-registry-9hthz Namespace:openshift-marketplace ID:9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9 UID:5eb5be5f-46bd-425a-a486-e2641bb0ad88 NetNS:/var/run/netns/29c01d2e-8012-44ee-9594-9dcbd2a39acd Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:51:04.974951 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:04.974925483Z" level=info msg="Adding pod openshift-marketplace_qe-app-registry-9hthz to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:51:05.020199 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:51:05.027599 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.027513971Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634" id=043e2429-1964-4da5-8773-23a1108d1e0b name=/runtime.v1.ImageService/PullImage Nov 16 14:51:05.028620 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.028579479Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634" id=654a26e7-a175-4825-8d00-bd4d66802a34 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.037232 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:51:05.037505 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[179539]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:51:05.038022 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610265.0378] device (veth09f52757): carrier: link connected Nov 16 14:51:05.038023 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[179539]: Could not generate persistent MAC address for veth09f52757: No such file or directory Nov 16 14:51:05.038466 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610265.0384] manager: (veth09f52757): new Veth device (/org/freedesktop/NetworkManager/Devices/78) Nov 16 14:51:05.060117 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.059950 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-4srsm\" (UniqueName: \"kubernetes.io/projected/590da926-dc2c-4378-a471-69dd415ae65f-kube-api-access-4srsm\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:05.062020 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610265.0619] manager: (veth09f52757): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/79) Nov 16 14:51:05.068020 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00212|bridge|INFO|bridge br0: added interface veth09f52757 on port 37 Nov 16 14:51:05.068236 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth09f52757 entered promiscuous mode Nov 16 14:51:05.102994 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00213|connmgr|INFO|br0<->unix#466: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:51:05.150923 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00214|connmgr|INFO|br0<->unix#469: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:51:05.279206 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.279095225Z" level=info msg="Removed container 9bc74a85595976d7a95ad27c6da2b95dd1db4747822715127d7cff88420cafeb: openshift-monitoring/node-exporter-dz75k/init-textfile" id=84efea37-d309-4fa0-aec1-3e642b4e4052 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:05.280026 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.279408 1563 scope.go:110] "RemoveContainer" containerID="131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6" Nov 16 14:51:05.281050 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.280974439Z" level=info msg="Removing container: 131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6" id=791cc5f9-767e-4576-8678-c0bd6daf46c0 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:05.281706 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.281607344Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:f4a6bddcf29f1036d73e15b666897a7d42426b1a799e533d71d356d6f22b6826,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3],Size_:344385412,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=2cf885af-68ee-4e1b-a2c3-7cf0f66dbda6 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.282789 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.282719952Z" level=info msg="Creating container: openshift-monitoring/telemeter-client-55c5556665-7vs77/reload" id=66ccc16f-7edf-4d39-8a5f-f094deec0685 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.282877 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.282840353Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:05.293310 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.293260530Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=21f0b9cb-7e5c-4e04-8a58-8d0fd73aa531 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:51:05.294538 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.294446038Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26611bedb7c02dbea85931612f83490e345de3f75644a914a70cb7fba2caa168" id=2e270705-d510-40fd-aa74-62c2f78fd717 name=/runtime.v1.ImageService/PullImage Nov 16 14:51:05.305378 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.305168418Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=d1511648-ddc8-45b9-a39a-7f7e0716703e name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:51:05.307642 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:51:05.307605 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaacb53e4_6716_4337_a1ae_4ac3a857cbb0.slice/crio-5b949bb6d720fbebd5c7320ff33fce728fa4e25d503805bf0249583a45f2afc0.scope WatchSource:0}: Error finding container 5b949bb6d720fbebd5c7320ff33fce728fa4e25d503805bf0249583a45f2afc0: Status 404 returned error &{%!s(*http.body=&{0xc00303ab28 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:51:05.314921 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:51:05.314881 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89cfb802_14c7_4c79_bc9c_015efafee773.slice/crio-855d812de5e04cf95ba97f58b76bd5839a4b5706b80db63d618286d8e31644e8.scope WatchSource:0}: Error finding container 855d812de5e04cf95ba97f58b76bd5839a4b5706b80db63d618286d8e31644e8: Status 404 returned error &{%!s(*http.body=&{0xc002b5bf98 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:51:05.315366 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.315329392Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26611bedb7c02dbea85931612f83490e345de3f75644a914a70cb7fba2caa168" id=6e322daa-9698-4c5f-bee4-7dfa31bd88ee name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.315637 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.315546094Z" level=info msg="Ran pod sandbox 5b949bb6d720fbebd5c7320ff33fce728fa4e25d503805bf0249583a45f2afc0 with infra container: openshift-image-registry/node-ca-dg87v/POD" id=21f0b9cb-7e5c-4e04-8a58-8d0fd73aa531 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:51:05.316285 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.316098298Z" level=info msg="Ran pod sandbox 855d812de5e04cf95ba97f58b76bd5839a4b5706b80db63d618286d8e31644e8 with infra container: openshift-monitoring/node-exporter-kxmx6/POD" id=d1511648-ddc8-45b9-a39a-7f7e0716703e name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:51:05.316676 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.316626902Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d8e2c924572c5086c16e027955e69da349d7a6a75c086adb2d2252b6d87f8e2" id=0d34d0a4-8cc6-4295-9449-6530a68046f6 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.317163 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.317111506Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0084c60f448bf0b9fb140ca70c10b860fd3c7e930f46f78eff2287d0ebf756d6" id=6db055e8-453a-4ece-a0f3-a261e090c73e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.328237 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.328189087Z" level=info msg="Removed container 131cd9dd4d81459bd3da6628ad575952a404f928f1f07de9e2b64b083b691fc6: openshift-image-registry/node-ca-dn4b9/node-ca" id=791cc5f9-767e-4576-8678-c0bd6daf46c0 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:05.328564 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.328526490Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0084c60f448bf0b9fb140ca70c10b860fd3c7e930f46f78eff2287d0ebf756d6 not found" id=6db055e8-453a-4ece-a0f3-a261e090c73e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.329307 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.329263395Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0084c60f448bf0b9fb140ca70c10b860fd3c7e930f46f78eff2287d0ebf756d6" id=9656aa39-a1d9-4b8a-b8df-dd8c4167a76b name=/runtime.v1.ImageService/PullImage Nov 16 14:51:05.330598 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.330539305Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:2f3a1ac7bb93faa2312d7fca21ca26dc55cbcd57502b470ad0bc40866a6ce73d,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634],Size_:340927490,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=654a26e7-a175-4825-8d00-bd4d66802a34 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.330868 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.330766006Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d8e2c924572c5086c16e027955e69da349d7a6a75c086adb2d2252b6d87f8e2" id=3e5a857a-f2f0-4fae-808f-c9d8fa86a6b3 name=/runtime.v1.ImageService/PullImage Nov 16 14:51:05.330868 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.330797806Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2ca2dedab2746a1596dc2d65af5bec6d52be3f14afbbee3c7d2f654086f6917" id=fb602b46-831f-430e-b824-e56807d4ec28 name=/runtime.v1.ImageService/PullImage Nov 16 14:51:05.331574 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.331496312Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:c02089e8997203170f5dca016eea44ee0fbe060028d433b115bd6db13803c3f8,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d8e2c924572c5086c16e027955e69da349d7a6a75c086adb2d2252b6d87f8e2],Size_:412120847,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=0d34d0a4-8cc6-4295-9449-6530a68046f6 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.331938 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.331895015Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:b1f628503a7313e2f8b55de870f13b1b8cf15eaf14b273adaa7c5de7e72f202c,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26611bedb7c02dbea85931612f83490e345de3f75644a914a70cb7fba2caa168],Size_:583852743,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=6e322daa-9698-4c5f-bee4-7dfa31bd88ee name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.332553 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.332515419Z" level=info msg="Creating container: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/oauth-proxy" id=7c7156ed-788e-4c0a-b444-8b6600c3dcb2 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.332644 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.332615020Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:05.332688 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.332636520Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2ca2dedab2746a1596dc2d65af5bec6d52be3f14afbbee3c7d2f654086f6917" id=6a9148c2-95fb-45d5-80d4-f90460b59f3d name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.332727 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.332672520Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d8e2c924572c5086c16e027955e69da349d7a6a75c086adb2d2252b6d87f8e2" id=80c14a79-0607-4318-a967-e70ef6958d50 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.332727 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.332673120Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d8e2c924572c5086c16e027955e69da349d7a6a75c086adb2d2252b6d87f8e2" id=7073715a-8fbe-4379-8de9-16b314c2a1e0 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.332839 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.332807021Z" level=info msg="Creating container: openshift-cluster-node-tuning-operator/tuned-cvdsv/tuned" id=ea11bbeb-c1e1-48b9-a9fc-7c5ae323fcaf name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.332898 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.332840621Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0084c60f448bf0b9fb140ca70c10b860fd3c7e930f46f78eff2287d0ebf756d6\"" Nov 16 14:51:05.332898 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.332887222Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:05.379194 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.379121263Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:c02089e8997203170f5dca016eea44ee0fbe060028d433b115bd6db13803c3f8,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d8e2c924572c5086c16e027955e69da349d7a6a75c086adb2d2252b6d87f8e2],Size_:412120847,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=80c14a79-0607-4318-a967-e70ef6958d50 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.379632 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.379588666Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:c02089e8997203170f5dca016eea44ee0fbe060028d433b115bd6db13803c3f8,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d8e2c924572c5086c16e027955e69da349d7a6a75c086adb2d2252b6d87f8e2],Size_:412120847,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=7073715a-8fbe-4379-8de9-16b314c2a1e0 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.379778 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.379600866Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:a81a185daaf4bd4547fd6797f6d0313ffd89e0c2f1f0111d4fe17d006ae34858,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2ca2dedab2746a1596dc2d65af5bec6d52be3f14afbbee3c7d2f654086f6917],Size_:363099082,Uid:nil,Username:nobody,Spec:nil,},Info:map[string]string{},}" id=6a9148c2-95fb-45d5-80d4-f90460b59f3d name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.380548 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.380513673Z" level=info msg="Creating container: openshift-image-registry/node-ca-dg87v/node-ca" id=a5ea2a14-9d6e-45b3-86e2-1d3e4e339636 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.380548 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.380541873Z" level=info msg="Creating container: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/openshift-state-metrics" id=399f4169-898c-4087-999b-1c700c1dfdb7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.380656 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.380617574Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:05.380693 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.380523073Z" level=info msg="Creating container: openshift-image-registry/image-registry-944cf86d8-kb2fb/registry" id=9ca25bb4-699b-4d0a-8937-9f886a3e8e99 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.380733 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.380626674Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:05.380787 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.380754875Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:05.399032 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-26402f26c066e9241993921f382abb2a2f178c803c38b42f80000f824a1032d8-merged.mount: Succeeded. Nov 16 14:51:05.399057 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-590da926\x2ddc2c\x2d4378\x2da471\x2d69dd415ae65f-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d4srsm.mount: Succeeded. Nov 16 14:51:05.399394 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-aed7136d07cd46efb639a0acb76d106242c3c3f4263f8c9d6bbf5e3813eaee1b-merged.mount: Succeeded. Nov 16 14:51:05.400219 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-aed7136d07cd46efb639a0acb76d106242c3c3f4263f8c9d6bbf5e3813eaee1b-merged.mount: Consumed 0 CPU time Nov 16 14:51:05.416857 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487.scope. Nov 16 14:51:05.450437 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487. Nov 16 14:51:05.572420 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222.scope. Nov 16 14:51:05.576475 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-5a53dc54be4e56f8a8c3ba5188210d4240ab49d348b65ef4c0163aac68af0ab4.scope. Nov 16 14:51:05.610329 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:05.615517 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-d32d12ec3df5836a124554148d33b54de74915e74f9d080e792dbe29890c6c36.scope. Nov 16 14:51:05.616588 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa.scope. Nov 16 14:51:05.618361 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 5a53dc54be4e56f8a8c3ba5188210d4240ab49d348b65ef4c0163aac68af0ab4. Nov 16 14:51:05.627817 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778.scope. Nov 16 14:51:05.628901 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222. Nov 16 14:51:05.638662 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container d32d12ec3df5836a124554148d33b54de74915e74f9d080e792dbe29890c6c36. Nov 16 14:51:05.644036 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa. Nov 16 14:51:05.652915 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778. Nov 16 14:51:05.670247 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:05.684145 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:05.690412 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:05.713075 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.713020724Z" level=info msg="Created container d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487: openshift-monitoring/telemeter-client-55c5556665-7vs77/reload" id=66ccc16f-7edf-4d39-8a5f-f094deec0685 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.713075 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.713056024Z" level=info msg="Created container 5a53dc54be4e56f8a8c3ba5188210d4240ab49d348b65ef4c0163aac68af0ab4: openshift-cluster-node-tuning-operator/tuned-cvdsv/tuned" id=ea11bbeb-c1e1-48b9-a9fc-7c5ae323fcaf name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.714108 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.714069831Z" level=info msg="Starting container: 5a53dc54be4e56f8a8c3ba5188210d4240ab49d348b65ef4c0163aac68af0ab4" id=fb8b6777-d823-4da9-908a-7c01112e5f91 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:05.714222 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.714124132Z" level=info msg="Starting container: d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487" id=e80a542a-9476-4f2a-8993-03f10d4cf7dd name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:05.726459 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.726410822Z" level=info msg="Started container" PID=179678 containerID=5a53dc54be4e56f8a8c3ba5188210d4240ab49d348b65ef4c0163aac68af0ab4 description=openshift-cluster-node-tuning-operator/tuned-cvdsv/tuned id=fb8b6777-d823-4da9-908a-7c01112e5f91 name=/runtime.v1.RuntimeService/StartContainer sandboxID=c70800789dc90e22c5a5cb32ae41a6619a6235951bc92cafe30ee53d4b9b92d0 Nov 16 14:51:05.727287 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.727251329Z" level=info msg="Started container" PID=179622 containerID=d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487 description=openshift-monitoring/telemeter-client-55c5556665-7vs77/reload id=e80a542a-9476-4f2a-8993-03f10d4cf7dd name=/runtime.v1.RuntimeService/StartContainer sandboxID=a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145 Nov 16 14:51:05.735406 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.735351788Z" level=info msg="Created container d32d12ec3df5836a124554148d33b54de74915e74f9d080e792dbe29890c6c36: openshift-image-registry/node-ca-dg87v/node-ca" id=a5ea2a14-9d6e-45b3-86e2-1d3e4e339636 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.735903 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.735868492Z" level=info msg="Starting container: d32d12ec3df5836a124554148d33b54de74915e74f9d080e792dbe29890c6c36" id=cd7ae6ca-026b-45df-993f-52f6afb27acf name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:05.754838 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.754758031Z" level=info msg="Started container" PID=179705 containerID=d32d12ec3df5836a124554148d33b54de74915e74f9d080e792dbe29890c6c36 description=openshift-image-registry/node-ca-dg87v/node-ca id=cd7ae6ca-026b-45df-993f-52f6afb27acf name=/runtime.v1.RuntimeService/StartContainer sandboxID=5b949bb6d720fbebd5c7320ff33fce728fa4e25d503805bf0249583a45f2afc0 Nov 16 14:51:05.755056 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.755019333Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=2d0b2661-90b7-4ca1-a1ea-9ed0fe357aec name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.755300 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.755267535Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=2d0b2661-90b7-4ca1-a1ea-9ed0fe357aec name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.756189 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.756074341Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=5ba330d6-f69f-4ba2-ae63-a14915068283 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.756329 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.756294343Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=5ba330d6-f69f-4ba2-ae63-a14915068283 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.757328 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.757285850Z" level=info msg="Creating container: openshift-monitoring/telemeter-client-55c5556665-7vs77/kube-rbac-proxy" id=277ed268-2f50-426f-9e4b-219c86c3c886 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.757410 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.757386651Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:05.771362 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.771306953Z" level=info msg="Created container a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/oauth-proxy" id=7c7156ed-788e-4c0a-b444-8b6600c3dcb2 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.772301 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.772265660Z" level=info msg="Starting container: a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222" id=23acc5f9-9be5-4d4f-9051-4228ba5ce6a1 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:05.786899 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.786842268Z" level=info msg="Started container" PID=179681 containerID=a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222 description=openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/oauth-proxy id=23acc5f9-9be5-4d4f-9051-4228ba5ce6a1 name=/runtime.v1.RuntimeService/StartContainer sandboxID=4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c Nov 16 14:51:05.787592 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.787543973Z" level=info msg="Created container d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa: openshift-image-registry/image-registry-944cf86d8-kb2fb/registry" id=9ca25bb4-699b-4d0a-8937-9f886a3e8e99 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.788221 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.788173078Z" level=info msg="Starting container: d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa" id=c248d961-d935-4ec6-944c-dcb6e50c850f name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:05.799661 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.799463661Z" level=info msg="Created container ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/openshift-state-metrics" id=399f4169-898c-4087-999b-1c700c1dfdb7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.800024 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.799988965Z" level=info msg="Starting container: ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778" id=ab02f956-90d9-4a7d-afec-ddf65c99ebc8 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:05.800889 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.800839871Z" level=info msg="Started container" PID=179713 containerID=d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa description=openshift-image-registry/image-registry-944cf86d8-kb2fb/registry id=c248d961-d935-4ec6-944c-dcb6e50c850f name=/runtime.v1.RuntimeService/StartContainer sandboxID=4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e Nov 16 14:51:05.810195 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.810146940Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=89ea60d5-c685-4040-b52a-5d07f54c950b name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.810432 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.810397742Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=89ea60d5-c685-4040-b52a-5d07f54c950b name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.811407 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.811368249Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=be36fbf8-1ea0-400f-8034-ed095e036e85 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.811629 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.811584450Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=be36fbf8-1ea0-400f-8034-ed095e036e85 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:05.812602 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.812561357Z" level=info msg="Creating container: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy" id=9695cc81-39f0-4961-ac49-e8a5d304015d name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.812683 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.812662658Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:05.817085 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e.scope. Nov 16 14:51:05.819456 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.819411508Z" level=info msg="Started container" PID=179720 containerID=ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778 description=openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/openshift-state-metrics id=ab02f956-90d9-4a7d-afec-ddf65c99ebc8 name=/runtime.v1.RuntimeService/StartContainer sandboxID=9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c Nov 16 14:51:05.839784 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e. Nov 16 14:51:05.867935 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.866893 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-fh97d" event=&{ID:590da926-dc2c-4378-a471-69dd415ae65f Type:ContainerDied Data:eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5} Nov 16 14:51:05.867935 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.866957 1563 scope.go:110] "RemoveContainer" containerID="9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f" Nov 16 14:51:05.868167 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.868021466Z" level=info msg="Removing container: 9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f" id=c75461c2-b7ee-4a77-9859-39555d972868 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:05.871945 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf.scope. Nov 16 14:51:05.876221 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.876037 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerStarted Data:a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222} Nov 16 14:51:05.877075 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.877014 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kxmx6" event=&{ID:89cfb802-14c7-4c79-bc9c-015efafee773 Type:ContainerStarted Data:855d812de5e04cf95ba97f58b76bd5839a4b5706b80db63d618286d8e31644e8} Nov 16 14:51:05.878056 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.877991 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" event=&{ID:0520c563-12a4-4756-b022-6ec7396a5839 Type:ContainerStarted Data:d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa} Nov 16 14:51:05.878604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.878575 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:51:05.880057 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:05.881267 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.881084 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" event=&{ID:add06140-663d-48c0-ac00-643bd58a3f7d Type:ContainerStarted Data:d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487} Nov 16 14:51:05.882690 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.882664 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cvdsv" event=&{ID:279f55b2-5534-42ae-ba2d-588e2c3a3ffe Type:ContainerStarted Data:5a53dc54be4e56f8a8c3ba5188210d4240ab49d348b65ef4c0163aac68af0ab4} Nov 16 14:51:05.884113 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.884086 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dg87v" event=&{ID:aacb53e4-6716-4337-a1ae-4ac3a857cbb0 Type:ContainerStarted Data:d32d12ec3df5836a124554148d33b54de74915e74f9d080e792dbe29890c6c36} Nov 16 14:51:05.884113 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.884141 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dg87v" event=&{ID:aacb53e4-6716-4337-a1ae-4ac3a857cbb0 Type:ContainerStarted Data:5b949bb6d720fbebd5c7320ff33fce728fa4e25d503805bf0249583a45f2afc0} Nov 16 14:51:05.884271 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod590da926_dc2c_4378_a471_69dd415ae65f.slice. Nov 16 14:51:05.884342 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod590da926_dc2c_4378_a471_69dd415ae65f.slice: Consumed 1.825s CPU time Nov 16 14:51:05.893914 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.892928 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" event=&{ID:c8013b4f-15b6-4bac-bfc9-a45301362f0c Type:ContainerStarted Data:ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778} Nov 16 14:51:05.900118 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.900087 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-fh97d] Nov 16 14:51:05.911336 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf. Nov 16 14:51:05.913159 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:05.912921 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-fh97d] Nov 16 14:51:05.915413 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.915362615Z" level=info msg="Removed container 9be1735ae0351eca66f4fdb4eb2aef097679f656683361ccf8bb6bd0f4ce356f: openshift-marketplace/qe-app-registry-fh97d/registry-server" id=c75461c2-b7ee-4a77-9859-39555d972868 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:05.948364 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:05.971658 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.971600830Z" level=info msg="Created container c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e: openshift-monitoring/telemeter-client-55c5556665-7vs77/kube-rbac-proxy" id=277ed268-2f50-426f-9e4b-219c86c3c886 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:05.972455 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.972419736Z" level=info msg="Starting container: c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e" id=d4bbf038-c41c-4e98-a455-93f150e5bd43 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:05.986357 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:05.986296538Z" level=info msg="Started container" PID=179900 containerID=c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e description=openshift-monitoring/telemeter-client-55c5556665-7vs77/kube-rbac-proxy id=d4bbf038-c41c-4e98-a455-93f150e5bd43 name=/runtime.v1.RuntimeService/StartContainer sandboxID=a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145 Nov 16 14:51:06.041029 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.040969841Z" level=info msg="Created container fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy" id=9695cc81-39f0-4961-ac49-e8a5d304015d name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:06.041556 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.041518745Z" level=info msg="Starting container: fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf" id=670b4c6c-5b0b-496e-82ef-e1aaaaff054a name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:06.058503 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.058444270Z" level=info msg="Started container" PID=179934 containerID=fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf description=openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy id=670b4c6c-5b0b-496e-82ef-e1aaaaff054a name=/runtime.v1.RuntimeService/StartContainer sandboxID=4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c Nov 16 14:51:06.083227 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.083174952Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:415636a81f07dca43e4ec03e27bea870ab8e35da872754afc4cc19b575728cdb" id=0dd10f4d-1035-48aa-9ebf-e11ad044ee52 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:06.083506 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.083476654Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:415636a81f07dca43e4ec03e27bea870ab8e35da872754afc4cc19b575728cdb not found" id=0dd10f4d-1035-48aa-9ebf-e11ad044ee52 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:06.084394 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.084360961Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:415636a81f07dca43e4ec03e27bea870ab8e35da872754afc4cc19b575728cdb" id=7842386f-86fa-4a57-996a-67aaea91a2f6 name=/runtime.v1.ImageService/PullImage Nov 16 14:51:06.087426 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.087387883Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:415636a81f07dca43e4ec03e27bea870ab8e35da872754afc4cc19b575728cdb\"" Nov 16 14:51:06.140698 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:51:06.141351 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 2:0:0:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:51:06.160257 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:51:06.160680 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 3:0:1:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:51:06.178390 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:0: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:51:06.178813 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:0: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:51:06.196075 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:1: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:51:06.196490 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:1: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:51:06.216802 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:2: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:51:06.217306 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:2: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:51:06.233433 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:3: [storvsc] Sense Key : Illegal Request [current] Nov 16 14:51:06.233838 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: sd 5:0:0:3: [storvsc] Add. Sense: Invalid command operation code Nov 16 14:51:06.416235 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:51:06Z [verbose] Add: openshift-marketplace:qe-app-registry-9hthz:5eb5be5f-46bd-425a-a486-e2641bb0ad88:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/29c01d2e-8012-44ee-9594-9dcbd2a39acd"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.36/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:51:06.416235 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:51:06.393500 179512 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-marketplace", Name:"qe-app-registry-9hthz", UID:"5eb5be5f-46bd-425a-a486-e2641bb0ad88", APIVersion:"v1", ResourceVersion:"96990", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.36/23] from openshift-sdn Nov 16 14:51:06.417744 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.417616417Z" level=info msg="Got pod network &{Name:qe-app-registry-9hthz Namespace:openshift-marketplace ID:9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9 UID:5eb5be5f-46bd-425a-a486-e2641bb0ad88 NetNS:/var/run/netns/29c01d2e-8012-44ee-9594-9dcbd2a39acd Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:51:06.417833 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:06.417750 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-marketplace/qe-app-registry-9hthz] Nov 16 14:51:06.418145 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.417833419Z" level=info msg="Checking pod openshift-marketplace_qe-app-registry-9hthz for CNI network multus-cni-network (type=multus)" Nov 16 14:51:06.421879 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:51:06.421813 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb5be5f_46bd_425a_a486_e2641bb0ad88.slice/crio-9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9.scope WatchSource:0}: Error finding container 9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9: Status 404 returned error &{%!s(*http.body=&{0xc002f125a0 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:51:06.432752 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.432637328Z" level=info msg="Ran pod sandbox 9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9 with infra container: openshift-marketplace/qe-app-registry-9hthz/POD" id=c98924fa-2be1-460f-855d-8ce29c3bdb2a name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:51:06.433824 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.433786836Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=bcc12ec8-111f-4ea4-872b-dd28e15261bb name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:06.434053 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.434016538Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=bcc12ec8-111f-4ea4-872b-dd28e15261bb name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:06.434876 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.434839044Z" level=info msg="Pulling image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=068e9bdf-7217-42b7-b637-5b5c180194c8 name=/runtime.v1.ImageService/PullImage Nov 16 14:51:06.437550 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.437467764Z" level=info msg="Trying to access \"quay.io/openshift-qe-optional-operators/aosqe-index:v4.10\"" Nov 16 14:51:06.589184 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.589034181Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0084c60f448bf0b9fb140ca70c10b860fd3c7e930f46f78eff2287d0ebf756d6\"" Nov 16 14:51:06.593966 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.593918217Z" level=info msg="Stopping pod sandbox: eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5" id=bf73e30c-a1c9-4bae-8920-e52f623fa369 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:06.593966 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:06.593966817Z" level=info msg="Stopped pod sandbox (already stopped): eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5" id=bf73e30c-a1c9-4bae-8920-e52f623fa369 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:06.595808 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:06.595774 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=590da926-dc2c-4378-a471-69dd415ae65f path="/var/lib/kubelet/pods/590da926-dc2c-4378-a471-69dd415ae65f/volumes" Nov 16 14:51:06.902495 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:06.902328 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerStarted Data:fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf} Nov 16 14:51:06.903317 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:06.903161 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-9hthz" event=&{ID:5eb5be5f-46bd-425a-a486-e2641bb0ad88 Type:ContainerStarted Data:9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9} Nov 16 14:51:06.905692 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:06.905638 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" event=&{ID:add06140-663d-48c0-ac00-643bd58a3f7d Type:ContainerStarted Data:c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e} Nov 16 14:51:07.321198 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:07.321032676Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:415636a81f07dca43e4ec03e27bea870ab8e35da872754afc4cc19b575728cdb\"" Nov 16 14:51:08.017985 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:08.017861813Z" level=info msg="Pulled image: quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7" id=068e9bdf-7217-42b7-b637-5b5c180194c8 name=/runtime.v1.ImageService/PullImage Nov 16 14:51:08.019193 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:08.019122922Z" level=info msg="Checking image status: quay.io/openshift-qe-optional-operators/aosqe-index:v4.10" id=3ac812c6-1ab8-4f6b-bf88-83a0e62ea83c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:08.019365 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:08.019329123Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:955b8c7a3931b968b0bd346474f64f679068be76fe317e18ac368da1adf57f20,RepoTags:[quay.io/openshift-qe-optional-operators/aosqe-index:v4.10],RepoDigests:[quay.io/openshift-qe-optional-operators/aosqe-index@sha256:37469b633c0f08ccead9fc8d560172b118ed4ede7797e8b75d5262bbf93e15e7 quay.io/openshift-qe-optional-operators/aosqe-index@sha256:5c8d127c45a9965860bd01677437fb9ac2ed820b9d54b086ccd040bf6508910b],Size_:340239472,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=3ac812c6-1ab8-4f6b-bf88-83a0e62ea83c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:08.020410 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:08.020366531Z" level=info msg="Creating container: openshift-marketplace/qe-app-registry-9hthz/registry-server" id=b73ef970-e0a5-405a-b29d-c994fde9bd92 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:08.020495 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:08.020480532Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:08.094917 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454.scope. Nov 16 14:51:08.127858 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454. Nov 16 14:51:08.163707 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:08.238711 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:08.238615140Z" level=info msg="Created container ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454: openshift-marketplace/qe-app-registry-9hthz/registry-server" id=b73ef970-e0a5-405a-b29d-c994fde9bd92 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:08.239255 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:08.239217644Z" level=info msg="Starting container: ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454" id=6366acc0-5bbf-4a75-9351-fe8704d69829 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:08.252148 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:08.252082939Z" level=info msg="Started container" PID=180167 containerID=ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454 description=openshift-marketplace/qe-app-registry-9hthz/registry-server id=6366acc0-5bbf-4a75-9351-fe8704d69829 name=/runtime.v1.RuntimeService/StartContainer sandboxID=9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9 Nov 16 14:51:08.913279 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:08.913231 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-9hthz" event=&{ID:5eb5be5f-46bd-425a-a486-e2641bb0ad88 Type:ContainerStarted Data:ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454} Nov 16 14:51:09.234794 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.234646881Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0084c60f448bf0b9fb140ca70c10b860fd3c7e930f46f78eff2287d0ebf756d6" id=9656aa39-a1d9-4b8a-b8df-dd8c4167a76b name=/runtime.v1.ImageService/PullImage Nov 16 14:51:09.235778 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.235707089Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0084c60f448bf0b9fb140ca70c10b860fd3c7e930f46f78eff2287d0ebf756d6" id=2c414671-6807-4133-9d0f-8fe4f6f9ce79 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:09.237103 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.237064299Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:eef7ef901a843590014b64817cba22b6f9a8e4ee810aa39b45f0d24e85849c8f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0084c60f448bf0b9fb140ca70c10b860fd3c7e930f46f78eff2287d0ebf756d6],Size_:317474731,Uid:nil,Username:nobody,Spec:nil,},Info:map[string]string{},}" id=2c414671-6807-4133-9d0f-8fe4f6f9ce79 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:09.238161 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.238102407Z" level=info msg="Creating container: openshift-monitoring/node-exporter-kxmx6/init-textfile" id=10de0e06-a37a-4e58-b935-918830c48c11 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:09.238269 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.238242208Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:09.308747 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-9952054211e894dff4dd1364dcad3bd540a17cb7fc5a80f07db974565d8212e4.scope. Nov 16 14:51:09.339495 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 9952054211e894dff4dd1364dcad3bd540a17cb7fc5a80f07db974565d8212e4. Nov 16 14:51:09.412530 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.412447792Z" level=info msg="Created container 9952054211e894dff4dd1364dcad3bd540a17cb7fc5a80f07db974565d8212e4: openshift-monitoring/node-exporter-kxmx6/init-textfile" id=10de0e06-a37a-4e58-b935-918830c48c11 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:09.413141 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.413084497Z" level=info msg="Starting container: 9952054211e894dff4dd1364dcad3bd540a17cb7fc5a80f07db974565d8212e4" id=e58dee6a-9955-42d0-9522-2eecc70201ac name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:09.427621 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.427494303Z" level=info msg="Started container" PID=180262 containerID=9952054211e894dff4dd1364dcad3bd540a17cb7fc5a80f07db974565d8212e4 description=openshift-monitoring/node-exporter-kxmx6/init-textfile id=e58dee6a-9955-42d0-9522-2eecc70201ac name=/runtime.v1.RuntimeService/StartContainer sandboxID=855d812de5e04cf95ba97f58b76bd5839a4b5706b80db63d618286d8e31644e8 Nov 16 14:51:09.553246 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9952054211e894dff4dd1364dcad3bd540a17cb7fc5a80f07db974565d8212e4.scope: Succeeded. Nov 16 14:51:09.553923 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9952054211e894dff4dd1364dcad3bd540a17cb7fc5a80f07db974565d8212e4.scope: Consumed 176ms CPU time Nov 16 14:51:09.554238 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9952054211e894dff4dd1364dcad3bd540a17cb7fc5a80f07db974565d8212e4.scope: Succeeded. Nov 16 14:51:09.554894 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9952054211e894dff4dd1364dcad3bd540a17cb7fc5a80f07db974565d8212e4.scope: Consumed 38ms CPU time Nov 16 14:51:09.917613 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:09.917573 1563 generic.go:296] "Generic (PLEG): container finished" podID=89cfb802-14c7-4c79-bc9c-015efafee773 containerID="9952054211e894dff4dd1364dcad3bd540a17cb7fc5a80f07db974565d8212e4" exitCode=0 Nov 16 14:51:09.918089 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:09.917689 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kxmx6" event=&{ID:89cfb802-14c7-4c79-bc9c-015efafee773 Type:ContainerDied Data:9952054211e894dff4dd1364dcad3bd540a17cb7fc5a80f07db974565d8212e4} Nov 16 14:51:09.918429 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.918378921Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0084c60f448bf0b9fb140ca70c10b860fd3c7e930f46f78eff2287d0ebf756d6" id=b89449ae-1fb1-49d4-ab68-9713e57cee23 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:09.960942 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.960849934Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:eef7ef901a843590014b64817cba22b6f9a8e4ee810aa39b45f0d24e85849c8f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0084c60f448bf0b9fb140ca70c10b860fd3c7e930f46f78eff2287d0ebf756d6],Size_:317474731,Uid:nil,Username:nobody,Spec:nil,},Info:map[string]string{},}" id=b89449ae-1fb1-49d4-ab68-9713e57cee23 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:09.962106 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.962033643Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:415636a81f07dca43e4ec03e27bea870ab8e35da872754afc4cc19b575728cdb" id=7842386f-86fa-4a57-996a-67aaea91a2f6 name=/runtime.v1.ImageService/PullImage Nov 16 14:51:09.963145 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.963055451Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:415636a81f07dca43e4ec03e27bea870ab8e35da872754afc4cc19b575728cdb" id=3dc1a546-ff29-4843-b968-e24071072be4 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:09.964567 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.964481361Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:4b1f43c95dae6ae2456299c22670bf9d5d8d59653113150a06448e63a57b7802,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:415636a81f07dca43e4ec03e27bea870ab8e35da872754afc4cc19b575728cdb],Size_:308849654,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=3dc1a546-ff29-4843-b968-e24071072be4 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:09.965916 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.965873171Z" level=info msg="Creating container: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/prom-label-proxy" id=f411dd45-981c-40ff-98be-42c443df90ef name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:09.966009 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.965982972Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:09.966111 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.966073273Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0084c60f448bf0b9fb140ca70c10b860fd3c7e930f46f78eff2287d0ebf756d6" id=72ceada2-ba86-451f-be3a-c847b551cdc1 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:09.967671 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.967592384Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:eef7ef901a843590014b64817cba22b6f9a8e4ee810aa39b45f0d24e85849c8f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0084c60f448bf0b9fb140ca70c10b860fd3c7e930f46f78eff2287d0ebf756d6],Size_:317474731,Uid:nil,Username:nobody,Spec:nil,},Info:map[string]string{},}" id=72ceada2-ba86-451f-be3a-c847b551cdc1 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:09.968543 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.968497891Z" level=info msg="Creating container: openshift-monitoring/node-exporter-kxmx6/node-exporter" id=51c81e3a-b6fc-4460-aeaf-6e16dd9b5235 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:09.968656 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:09.968597591Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:10.029198 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619.scope. Nov 16 14:51:10.076821 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-107c7041f827c1f3af92eb979d579b0bf04fc75ac1cf3930c1ca054a512f502c.scope. Nov 16 14:51:10.078307 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619. Nov 16 14:51:10.102506 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 107c7041f827c1f3af92eb979d579b0bf04fc75ac1cf3930c1ca054a512f502c. Nov 16 14:51:10.119076 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:10.136028 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:10.206503 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.206361844Z" level=info msg="Created container 107c7041f827c1f3af92eb979d579b0bf04fc75ac1cf3930c1ca054a512f502c: openshift-monitoring/node-exporter-kxmx6/node-exporter" id=51c81e3a-b6fc-4460-aeaf-6e16dd9b5235 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:10.207059 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.207016949Z" level=info msg="Starting container: 107c7041f827c1f3af92eb979d579b0bf04fc75ac1cf3930c1ca054a512f502c" id=25ed8229-a4d5-425b-8bf5-3f099ef6bbb9 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:10.221306 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.221249354Z" level=info msg="Started container" PID=180421 containerID=107c7041f827c1f3af92eb979d579b0bf04fc75ac1cf3930c1ca054a512f502c description=openshift-monitoring/node-exporter-kxmx6/node-exporter id=25ed8229-a4d5-425b-8bf5-3f099ef6bbb9 name=/runtime.v1.RuntimeService/StartContainer sandboxID=855d812de5e04cf95ba97f58b76bd5839a4b5706b80db63d618286d8e31644e8 Nov 16 14:51:10.229294 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.229241212Z" level=info msg="Created container 8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/prom-label-proxy" id=f411dd45-981c-40ff-98be-42c443df90ef name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:10.229857 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.229817017Z" level=info msg="Starting container: 8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619" id=a28e13fd-f56e-454d-8a43-2e446e1eb4ea name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:10.242668 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.242620011Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=7a79314f-ca77-44d9-8ef4-18ddba6fa175 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:10.243040 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.242869213Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=7a79314f-ca77-44d9-8ef4-18ddba6fa175 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:10.243884 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.243839020Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=4c9ac7df-30d8-413b-bd79-6a95a1e27ccb name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:10.244118 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.244049021Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=4c9ac7df-30d8-413b-bd79-6a95a1e27ccb name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:10.245380 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.245342931Z" level=info msg="Creating container: openshift-monitoring/node-exporter-kxmx6/kube-rbac-proxy" id=42260028-4afc-45a4-9c44-7af57487ec8b name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:10.245472 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.245459132Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:10.254572 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.254530599Z" level=info msg="Started container" PID=180411 containerID=8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619 description=openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/prom-label-proxy id=a28e13fd-f56e-454d-8a43-2e446e1eb4ea name=/runtime.v1.RuntimeService/StartContainer sandboxID=4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c Nov 16 14:51:10.283715 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.283650213Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=958418bb-f0c8-4804-ab77-c72729bf5815 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:10.283955 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.283921715Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=958418bb-f0c8-4804-ab77-c72729bf5815 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:10.284963 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.284920622Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=85e1c3cd-6f63-40ea-bb11-0a806ef386a5 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:10.285165 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.285115924Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=85e1c3cd-6f63-40ea-bb11-0a806ef386a5 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:10.286230 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.286194032Z" level=info msg="Creating container: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy-rules" id=5f0af29b-e897-4814-b82e-aec15c39abad name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:10.286316 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.286294432Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:10.330274 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-04a14af699e7040c43c8f060f5969df68ada079d564f68b76ddd20a6b401004a.scope. Nov 16 14:51:10.365759 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 04a14af699e7040c43c8f060f5969df68ada079d564f68b76ddd20a6b401004a. Nov 16 14:51:10.366734 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665.scope. Nov 16 14:51:10.371858 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.371786 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 14:51:10.372330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.372110 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=d002c6da-b2c9-44ec-827f-97e4f83396f4 containerName="alertmanager" containerID="cri-o://62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a" gracePeriod=120 Nov 16 14:51:10.372330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.372149 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=d002c6da-b2c9-44ec-827f-97e4f83396f4 containerName="kube-rbac-proxy-metric" containerID="cri-o://be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a" gracePeriod=120 Nov 16 14:51:10.372330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.372191 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=d002c6da-b2c9-44ec-827f-97e4f83396f4 containerName="alertmanager-proxy" containerID="cri-o://f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953" gracePeriod=120 Nov 16 14:51:10.372330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.372234 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=d002c6da-b2c9-44ec-827f-97e4f83396f4 containerName="kube-rbac-proxy" containerID="cri-o://dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc" gracePeriod=120 Nov 16 14:51:10.372330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.372258 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=d002c6da-b2c9-44ec-827f-97e4f83396f4 containerName="config-reloader" containerID="cri-o://8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba" gracePeriod=120 Nov 16 14:51:10.372330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.372193 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=d002c6da-b2c9-44ec-827f-97e4f83396f4 containerName="prom-label-proxy" containerID="cri-o://d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f" gracePeriod=120 Nov 16 14:51:10.372564 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.372439567Z" level=info msg="Stopping container: 62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a (timeout: 120s)" id=74307116-ab60-4e7d-9d4f-fed9f638abae name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:51:10.372564 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.372491167Z" level=info msg="Stopping container: dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc (timeout: 120s)" id=eee34cf8-cd90-4c2b-a710-e6cbaf17ebab name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:51:10.372564 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.372528967Z" level=info msg="Stopping container: 8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba (timeout: 120s)" id=96ef6d7e-e9fd-4a02-a4ae-e6920ff9584f name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:51:10.372669 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.372492167Z" level=info msg="Stopping container: f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953 (timeout: 120s)" id=1f1c82e3-0fb8-4de1-a2e1-4dc26a7bbc7d name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:51:10.372669 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.372448867Z" level=info msg="Stopping container: d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f (timeout: 120s)" id=c84f0036-0680-47b6-8c17-251b9877c8be name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:51:10.372734 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.372441867Z" level=info msg="Stopping container: be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a (timeout: 120s)" id=a2a1e8ff-501f-4753-8c2d-ba8b3838fff9 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:51:10.407617 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc.scope: Succeeded. Nov 16 14:51:10.408684 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc.scope: Consumed 891ms CPU time Nov 16 14:51:10.409109 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953.scope: Succeeded. Nov 16 14:51:10.409723 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953.scope: Consumed 24.456s CPU time Nov 16 14:51:10.409969 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f.scope: Succeeded. Nov 16 14:51:10.410576 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f.scope: Consumed 38ms CPU time Nov 16 14:51:10.410802 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba.scope: Succeeded. Nov 16 14:51:10.411626 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba.scope: Consumed 761ms CPU time Nov 16 14:51:10.412319 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953.scope: Succeeded. Nov 16 14:51:10.412858 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953.scope: Consumed 40ms CPU time Nov 16 14:51:10.413036 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba.scope: Succeeded. Nov 16 14:51:10.413655 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba.scope: Consumed 40ms CPU time Nov 16 14:51:10.413832 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc.scope: Succeeded. Nov 16 14:51:10.414434 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc.scope: Consumed 39ms CPU time Nov 16 14:51:10.417339 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f.scope: Succeeded. Nov 16 14:51:10.417959 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f.scope: Consumed 39ms CPU time Nov 16 14:51:10.421323 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665. Nov 16 14:51:10.426086 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:10.459213 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:10.492516 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.492452050Z" level=info msg="Created container 04a14af699e7040c43c8f060f5969df68ada079d564f68b76ddd20a6b401004a: openshift-monitoring/node-exporter-kxmx6/kube-rbac-proxy" id=42260028-4afc-45a4-9c44-7af57487ec8b name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:10.493211 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.493120355Z" level=info msg="Starting container: 04a14af699e7040c43c8f060f5969df68ada079d564f68b76ddd20a6b401004a" id=65b2144f-ce82-4715-9d77-67bdc95593bc name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:10.509541 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.508048765Z" level=info msg="Started container" PID=180562 containerID=04a14af699e7040c43c8f060f5969df68ada079d564f68b76ddd20a6b401004a description=openshift-monitoring/node-exporter-kxmx6/kube-rbac-proxy id=65b2144f-ce82-4715-9d77-67bdc95593bc name=/runtime.v1.RuntimeService/StartContainer sandboxID=855d812de5e04cf95ba97f58b76bd5839a4b5706b80db63d618286d8e31644e8 Nov 16 14:51:10.561932 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.561842861Z" level=info msg="Created container f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy-rules" id=5f0af29b-e897-4814-b82e-aec15c39abad name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:10.562449 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.562410965Z" level=info msg="Starting container: f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665" id=fedc1931-72ef-49e3-b4b5-a97a2258c876 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:10.576609 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:10.576552869Z" level=info msg="Started container" PID=180598 containerID=f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665 description=openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy-rules id=fedc1931-72ef-49e3-b4b5-a97a2258c876 name=/runtime.v1.RuntimeService/StartContainer sandboxID=4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c Nov 16 14:51:10.923144 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.923082 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerStarted Data:f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665} Nov 16 14:51:10.923144 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.923178 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerStarted Data:8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619} Nov 16 14:51:10.925326 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.925300 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d002c6da-b2c9-44ec-827f-97e4f83396f4/alertmanager-proxy/0.log" Nov 16 14:51:10.925699 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.925670 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d002c6da-b2c9-44ec-827f-97e4f83396f4/config-reloader/0.log" Nov 16 14:51:10.926114 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.926082 1563 generic.go:296] "Generic (PLEG): container finished" podID=d002c6da-b2c9-44ec-827f-97e4f83396f4 containerID="d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f" exitCode=0 Nov 16 14:51:10.926114 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.926109 1563 generic.go:296] "Generic (PLEG): container finished" podID=d002c6da-b2c9-44ec-827f-97e4f83396f4 containerID="dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc" exitCode=0 Nov 16 14:51:10.926114 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.926123 1563 generic.go:296] "Generic (PLEG): container finished" podID=d002c6da-b2c9-44ec-827f-97e4f83396f4 containerID="f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953" exitCode=2 Nov 16 14:51:10.926284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.926149 1563 generic.go:296] "Generic (PLEG): container finished" podID=d002c6da-b2c9-44ec-827f-97e4f83396f4 containerID="8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba" exitCode=2 Nov 16 14:51:10.926284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.926162 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerDied Data:d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f} Nov 16 14:51:10.926284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.926195 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerDied Data:dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc} Nov 16 14:51:10.926284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.926213 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerDied Data:f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953} Nov 16 14:51:10.926284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.926231 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerDied Data:8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba} Nov 16 14:51:10.928025 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.927996 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kxmx6" event=&{ID:89cfb802-14c7-4c79-bc9c-015efafee773 Type:ContainerStarted Data:04a14af699e7040c43c8f060f5969df68ada079d564f68b76ddd20a6b401004a} Nov 16 14:51:10.928025 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:10.928032 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kxmx6" event=&{ID:89cfb802-14c7-4c79-bc9c-015efafee773 Type:ContainerStarted Data:107c7041f827c1f3af92eb979d579b0bf04fc75ac1cf3930c1ca054a512f502c} Nov 16 14:51:11.302339 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-69e6654fcaa4cc86a15d2ce88744265b3ecd7a8b3c2e440ef1ea962e160f658a-merged.mount: Succeeded. Nov 16 14:51:11.303151 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-69e6654fcaa4cc86a15d2ce88744265b3ecd7a8b3c2e440ef1ea962e160f658a-merged.mount: Consumed 0 CPU time Nov 16 14:51:11.480959 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a.scope: Succeeded. Nov 16 14:51:11.481635 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a.scope: Consumed 6.389s CPU time Nov 16 14:51:11.720016 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:11.719961787Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=45063d60-fd24-427d-98ed-199a453ea959 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:11.720608 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:11.720224388Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=45063d60-fd24-427d-98ed-199a453ea959 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:11.721179 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:11.721089895Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=f41c784f-b551-4e58-aa6d-d037f065d465 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:11.721337 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:11.721301196Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=f41c784f-b551-4e58-aa6d-d037f065d465 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:11.722365 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:11.722323704Z" level=info msg="Creating container: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy-metrics" id=9477e97e-9549-4041-8f5a-06a0343ce2e2 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:11.722453 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:11.722434605Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:11.734374 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a.scope: Succeeded. Nov 16 14:51:11.735040 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a.scope: Consumed 31.036s CPU time Nov 16 14:51:11.764415 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a.scope: Succeeded. Nov 16 14:51:11.765041 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a.scope: Consumed 39ms CPU time Nov 16 14:51:11.768929 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a.scope: Succeeded. Nov 16 14:51:11.769536 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a.scope: Consumed 45ms CPU time Nov 16 14:51:11.782169 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-aeb95a9db8e81254e1f38693e179565e7be042b26b1dcc7875c740cf2a5c5a66-merged.mount: Succeeded. Nov 16 14:51:11.782902 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-aeb95a9db8e81254e1f38693e179565e7be042b26b1dcc7875c740cf2a5c5a66-merged.mount: Consumed 0 CPU time Nov 16 14:51:11.910041 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:11.909954885Z" level=info msg="Stopped container 8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba: openshift-monitoring/alertmanager-main-0/config-reloader" id=96ef6d7e-e9fd-4a02-a4ae-e6920ff9584f name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:51:11.934198 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:11.934161 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d002c6da-b2c9-44ec-827f-97e4f83396f4/alertmanager-proxy/0.log" Nov 16 14:51:11.934775 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:11.934583 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d002c6da-b2c9-44ec-827f-97e4f83396f4/config-reloader/0.log" Nov 16 14:51:11.934989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:11.934965 1563 generic.go:296] "Generic (PLEG): container finished" podID=d002c6da-b2c9-44ec-827f-97e4f83396f4 containerID="be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a" exitCode=0 Nov 16 14:51:11.934989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:11.934991 1563 generic.go:296] "Generic (PLEG): container finished" podID=d002c6da-b2c9-44ec-827f-97e4f83396f4 containerID="62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a" exitCode=0 Nov 16 14:51:11.935074 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:11.935054 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerDied Data:be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a} Nov 16 14:51:11.935113 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:11.935089 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerDied Data:62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a} Nov 16 14:51:11.970657 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-a16113f750dfc74c801607cd28f2e017c164056dcdddd0451971e41525a3bfaa-merged.mount: Succeeded. Nov 16 14:51:11.971581 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-a16113f750dfc74c801607cd28f2e017c164056dcdddd0451971e41525a3bfaa-merged.mount: Consumed 0 CPU time Nov 16 14:51:12.114898 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:12.114824693Z" level=info msg="Stopped container f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953: openshift-monitoring/alertmanager-main-0/alertmanager-proxy" id=1f1c82e3-0fb8-4de1-a2e1-4dc26a7bbc7d name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:51:12.151000 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-7b409ae415ff20d3c625e093b2076d4a905ee629554de42b1a3df6ecc9de4e32-merged.mount: Succeeded. Nov 16 14:51:12.151863 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-7b409ae415ff20d3c625e093b2076d4a905ee629554de42b1a3df6ecc9de4e32-merged.mount: Consumed 0 CPU time Nov 16 14:51:12.259664 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:12.259384258Z" level=info msg="Stopped container d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f: openshift-monitoring/alertmanager-main-0/prom-label-proxy" id=c84f0036-0680-47b6-8c17-251b9877c8be name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:51:12.271233 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-3cc11ada9c37f2a5823c0e065f4a866e605c5d69c79ac54d1a657c04ca058601-merged.mount: Succeeded. Nov 16 14:51:12.272059 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:12.271973150Z" level=info msg="Stopped container dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy" id=eee34cf8-cd90-4c2b-a710-e6cbaf17ebab name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:51:12.272261 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-3cc11ada9c37f2a5823c0e065f4a866e605c5d69c79ac54d1a657c04ca058601-merged.mount: Consumed 0 CPU time Nov 16 14:51:12.365909 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-fbdb208ee2db663393e329a5ac1728f69efe1c2122ccd285dc5c78e4309d7238-merged.mount: Succeeded. Nov 16 14:51:12.366804 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-fbdb208ee2db663393e329a5ac1728f69efe1c2122ccd285dc5c78e4309d7238-merged.mount: Consumed 0 CPU time Nov 16 14:51:12.366931 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:12.366843349Z" level=info msg="Stopped container 62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a: openshift-monitoring/alertmanager-main-0/alertmanager" id=74307116-ab60-4e7d-9d4f-fed9f638abae name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:51:12.481355 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:12.481298391Z" level=info msg="Stopped container be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric" id=a2a1e8ff-501f-4753-8c2d-ba8b3838fff9 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:51:12.481850 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:12.481811095Z" level=info msg="Stopping pod sandbox: 765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85" id=9a90eab5-c066-4bd2-9458-3473a211cd9a name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:12.482146 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:12.482102897Z" level=info msg="Got pod network &{Name:alertmanager-main-0 Namespace:openshift-monitoring ID:765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85 UID:d002c6da-b2c9-44ec-827f-97e4f83396f4 NetNS:/var/run/netns/104df8d8-bf96-4ded-830e-506de358626a Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:51:12.482301 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:12.482272198Z" level=info msg="Deleting pod openshift-monitoring_alertmanager-main-0 from CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:51:12.537995 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0.scope. Nov 16 14:51:12.577574 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0. Nov 16 14:51:12.617491 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:12.734966 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:12.734892558Z" level=info msg="Created container 0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy-metrics" id=9477e97e-9549-4041-8f5a-06a0343ce2e2 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:12.735539 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:12.735498363Z" level=info msg="Starting container: 0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0" id=6a840af2-b528-4355-96c5-350e43eb7946 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:12.748067 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:12.748027655Z" level=info msg="Started container" PID=180877 containerID=0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0 description=openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy-metrics id=6a840af2-b528-4355-96c5-350e43eb7946 name=/runtime.v1.RuntimeService/StartContainer sandboxID=4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c Nov 16 14:51:12.940816 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:12.940740 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerStarted Data:0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0} Nov 16 14:51:12.941313 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:12.941050 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:51:12.948068 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:12.948005 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" Nov 16 14:51:15.771120 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00215|connmgr|INFO|br0<->unix#472: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:51:15.804616 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00216|connmgr|INFO|br0<->unix#475: 4 flow_mods in the last 0 s (4 deletes) Nov 16 14:51:15.835792 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00217|bridge|INFO|bridge br0: deleted interface veth8eb10799 on port 8 Nov 16 14:51:15.843158 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth8eb10799 left promiscuous mode Nov 16 14:51:15.853524 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:51:15Z [verbose] Del: openshift-monitoring:alertmanager-main-0:d002c6da-b2c9-44ec-827f-97e4f83396f4:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 14:51:15.863344 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-104df8d8\x2dbf96\x2d4ded\x2d830e\x2d506de358626a.mount: Succeeded. Nov 16 14:51:15.864164 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-104df8d8\x2dbf96\x2d4ded\x2d830e\x2d506de358626a.mount: Consumed 0 CPU time Nov 16 14:51:15.887906 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-104df8d8\x2dbf96\x2d4ded\x2d830e\x2d506de358626a.mount: Succeeded. Nov 16 14:51:15.888775 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-104df8d8\x2dbf96\x2d4ded\x2d830e\x2d506de358626a.mount: Consumed 0 CPU time Nov 16 14:51:15.914666 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-104df8d8\x2dbf96\x2d4ded\x2d830e\x2d506de358626a.mount: Succeeded. Nov 16 14:51:15.915575 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-104df8d8\x2dbf96\x2d4ded\x2d830e\x2d506de358626a.mount: Consumed 0 CPU time Nov 16 14:51:16.263437 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:16.263342533Z" level=info msg="Stopped pod sandbox: 765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85" id=9a90eab5-c066-4bd2-9458-3473a211cd9a name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:16.272418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.272387 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d002c6da-b2c9-44ec-827f-97e4f83396f4/alertmanager-proxy/0.log" Nov 16 14:51:16.272873 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.272848 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d002c6da-b2c9-44ec-827f-97e4f83396f4/config-reloader/0.log" Nov 16 14:51:16.302176 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-958da3ec6c2ff4a50359e44b09290479ea533a40afa7b87057026d68b98d520c-merged.mount: Succeeded. Nov 16 14:51:16.302992 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-958da3ec6c2ff4a50359e44b09290479ea533a40afa7b87057026d68b98d520c-merged.mount: Consumed 0 CPU time Nov 16 14:51:16.303012 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85-userdata-shm.mount: Succeeded. Nov 16 14:51:16.303709 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85-userdata-shm.mount: Consumed 0 CPU time Nov 16 14:51:16.364464 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.364363 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdd6h\" (UniqueName: \"kubernetes.io/projected/d002c6da-b2c9-44ec-827f-97e4f83396f4-kube-api-access-bdd6h\") pod \"d002c6da-b2c9-44ec-827f-97e4f83396f4\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " Nov 16 14:51:16.364464 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.364440 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-config-volume\") pod \"d002c6da-b2c9-44ec-827f-97e4f83396f4\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " Nov 16 14:51:16.364464 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.364475 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d002c6da-b2c9-44ec-827f-97e4f83396f4-alertmanager-trusted-ca-bundle\") pod \"d002c6da-b2c9-44ec-827f-97e4f83396f4\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " Nov 16 14:51:16.364788 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.364502 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d002c6da-b2c9-44ec-827f-97e4f83396f4-tls-assets\") pod \"d002c6da-b2c9-44ec-827f-97e4f83396f4\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " Nov 16 14:51:16.364788 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.364555 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d002c6da-b2c9-44ec-827f-97e4f83396f4-alertmanager-main-db\") pod \"d002c6da-b2c9-44ec-827f-97e4f83396f4\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " Nov 16 14:51:16.364788 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.364607 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-main-proxy\") pod \"d002c6da-b2c9-44ec-827f-97e4f83396f4\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " Nov 16 14:51:16.364788 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.364652 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d002c6da-b2c9-44ec-827f-97e4f83396f4\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " Nov 16 14:51:16.364788 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.364684 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d002c6da-b2c9-44ec-827f-97e4f83396f4-metrics-client-ca\") pod \"d002c6da-b2c9-44ec-827f-97e4f83396f4\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " Nov 16 14:51:16.364788 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.364719 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-kube-rbac-proxy\") pod \"d002c6da-b2c9-44ec-827f-97e4f83396f4\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " Nov 16 14:51:16.364788 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.364748 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-main-tls\") pod \"d002c6da-b2c9-44ec-827f-97e4f83396f4\" (UID: \"d002c6da-b2c9-44ec-827f-97e4f83396f4\") " Nov 16 14:51:16.365307 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:51:16.365020 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/d002c6da-b2c9-44ec-827f-97e4f83396f4/volumes/kubernetes.io~configmap/metrics-client-ca: clearQuota called, but quotas disabled Nov 16 14:51:16.365307 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:51:16.365112 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/d002c6da-b2c9-44ec-827f-97e4f83396f4/volumes/kubernetes.io~empty-dir/alertmanager-main-db: clearQuota called, but quotas disabled Nov 16 14:51:16.365307 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.365274 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d002c6da-b2c9-44ec-827f-97e4f83396f4-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d002c6da-b2c9-44ec-827f-97e4f83396f4" (UID: "d002c6da-b2c9-44ec-827f-97e4f83396f4"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Nov 16 14:51:16.365307 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.365274 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d002c6da-b2c9-44ec-827f-97e4f83396f4-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d002c6da-b2c9-44ec-827f-97e4f83396f4" (UID: "d002c6da-b2c9-44ec-827f-97e4f83396f4"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 14:51:16.365533 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:51:16.365499 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/d002c6da-b2c9-44ec-827f-97e4f83396f4/volumes/kubernetes.io~configmap/alertmanager-trusted-ca-bundle: clearQuota called, but quotas disabled Nov 16 14:51:16.365750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.365719 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d002c6da-b2c9-44ec-827f-97e4f83396f4-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d002c6da-b2c9-44ec-827f-97e4f83396f4" (UID: "d002c6da-b2c9-44ec-827f-97e4f83396f4"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 14:51:16.377857 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dbdd6h.mount: Succeeded. Nov 16 14:51:16.378735 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dbdd6h.mount: Consumed 0 CPU time Nov 16 14:51:16.378756 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dmain\x2dproxy.mount: Succeeded. Nov 16 14:51:16.379449 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dmain\x2dproxy.mount: Consumed 0 CPU time Nov 16 14:51:16.379465 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dkube\x2drbac\x2dproxy\x2dmetric.mount: Succeeded. Nov 16 14:51:16.380160 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dkube\x2drbac\x2dproxy\x2dmetric.mount: Consumed 0 CPU time Nov 16 14:51:16.380177 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dmain\x2dtls.mount: Succeeded. Nov 16 14:51:16.380954 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dmain\x2dtls.mount: Consumed 0 CPU time Nov 16 14:51:16.380971 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7esecret-config\x2dvolume.mount: Succeeded. Nov 16 14:51:16.381645 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7esecret-config\x2dvolume.mount: Consumed 0 CPU time Nov 16 14:51:16.381663 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7eprojected-tls\x2dassets.mount: Succeeded. Nov 16 14:51:16.382316 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7eprojected-tls\x2dassets.mount: Consumed 0 CPU time Nov 16 14:51:16.382331 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dkube\x2drbac\x2dproxy.mount: Succeeded. Nov 16 14:51:16.383061 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d002c6da\x2db2c9\x2d44ec\x2d827f\x2d97e4f83396f4-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dkube\x2drbac\x2dproxy.mount: Consumed 0 CPU time Nov 16 14:51:16.396440 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.396389 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-main-proxy" (OuterVolumeSpecName: "secret-alertmanager-main-proxy") pod "d002c6da-b2c9-44ec-827f-97e4f83396f4" (UID: "d002c6da-b2c9-44ec-827f-97e4f83396f4"). InnerVolumeSpecName "secret-alertmanager-main-proxy". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 14:51:16.401423 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.401372 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d002c6da-b2c9-44ec-827f-97e4f83396f4-kube-api-access-bdd6h" (OuterVolumeSpecName: "kube-api-access-bdd6h") pod "d002c6da-b2c9-44ec-827f-97e4f83396f4" (UID: "d002c6da-b2c9-44ec-827f-97e4f83396f4"). InnerVolumeSpecName "kube-api-access-bdd6h". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:51:16.406440 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.406402 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-config-volume" (OuterVolumeSpecName: "config-volume") pod "d002c6da-b2c9-44ec-827f-97e4f83396f4" (UID: "d002c6da-b2c9-44ec-827f-97e4f83396f4"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 14:51:16.408395 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.408340 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d002c6da-b2c9-44ec-827f-97e4f83396f4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d002c6da-b2c9-44ec-827f-97e4f83396f4" (UID: "d002c6da-b2c9-44ec-827f-97e4f83396f4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:51:16.412445 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.412371 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d002c6da-b2c9-44ec-827f-97e4f83396f4" (UID: "d002c6da-b2c9-44ec-827f-97e4f83396f4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 14:51:16.421404 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.421367 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d002c6da-b2c9-44ec-827f-97e4f83396f4" (UID: "d002c6da-b2c9-44ec-827f-97e4f83396f4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 14:51:16.431465 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.431400 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d002c6da-b2c9-44ec-827f-97e4f83396f4" (UID: "d002c6da-b2c9-44ec-827f-97e4f83396f4"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 14:51:16.466359 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.466262 1563 reconciler.go:300] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-config-volume\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:16.466359 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.466321 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-bdd6h\" (UniqueName: \"kubernetes.io/projected/d002c6da-b2c9-44ec-827f-97e4f83396f4-kube-api-access-bdd6h\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:16.466359 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.466336 1563 reconciler.go:300] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d002c6da-b2c9-44ec-827f-97e4f83396f4-tls-assets\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:16.466359 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.466353 1563 reconciler.go:300] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d002c6da-b2c9-44ec-827f-97e4f83396f4-alertmanager-trusted-ca-bundle\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:16.466359 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.466369 1563 reconciler.go:300] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d002c6da-b2c9-44ec-827f-97e4f83396f4-alertmanager-main-db\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:16.466359 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.466384 1563 reconciler.go:300] "Volume detached for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-main-proxy\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:16.466725 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.466406 1563 reconciler.go:300] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:16.466725 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.466422 1563 reconciler.go:300] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d002c6da-b2c9-44ec-827f-97e4f83396f4-metrics-client-ca\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:16.466725 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.466437 1563 reconciler.go:300] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-main-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:16.466725 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.466451 1563 reconciler.go:300] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d002c6da-b2c9-44ec-827f-97e4f83396f4-secret-alertmanager-kube-rbac-proxy\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:51:16.608633 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-podd002c6da_b2c9_44ec_827f_97e4f83396f4.slice. Nov 16 14:51:16.608691 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-podd002c6da_b2c9_44ec_827f_97e4f83396f4.slice: Consumed 1min 3.819s CPU time Nov 16 14:51:16.954905 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.954693 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d002c6da-b2c9-44ec-827f-97e4f83396f4/alertmanager-proxy/0.log" Nov 16 14:51:16.955196 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.955150 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d002c6da-b2c9-44ec-827f-97e4f83396f4/config-reloader/0.log" Nov 16 14:51:16.955534 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.955511 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:d002c6da-b2c9-44ec-827f-97e4f83396f4 Type:ContainerDied Data:765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85} Nov 16 14:51:16.955631 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:16.955557 1563 scope.go:110] "RemoveContainer" containerID="d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f" Nov 16 14:51:16.956559 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:16.956512236Z" level=info msg="Removing container: d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f" id=35d28af7-58a9-45b9-8ad0-a7a4a3e2a421 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:17.048241 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:17.048177211Z" level=info msg="Removed container d833ba6f20a8443e64178966bd58a8a5b916a0f6a4adb548f2687dc725c1274f: openshift-monitoring/alertmanager-main-0/prom-label-proxy" id=35d28af7-58a9-45b9-8ad0-a7a4a3e2a421 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:17.048649 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:17.048599 1563 scope.go:110] "RemoveContainer" containerID="be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a" Nov 16 14:51:17.049789 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:17.049749222Z" level=info msg="Removing container: be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a" id=298d4a43-d6b3-4706-94ef-a588742b9a6d name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:17.155213 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:17.155158998Z" level=info msg="Removed container be2984e688d46c84d9ab3d0377f7521511fc2f5f9a9ed1ba5092ad445cccf48a: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric" id=298d4a43-d6b3-4706-94ef-a588742b9a6d name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:17.155628 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:17.155585 1563 scope.go:110] "RemoveContainer" containerID="dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc" Nov 16 14:51:17.156708 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:17.156666609Z" level=info msg="Removing container: dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc" id=f77a162b-a3d1-4c5c-851c-a35383971859 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:17.251721 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:17.251485507Z" level=info msg="Removed container dae5afe32459ea4140d15c4d12cce21e1c4528e07c56ec64dd02f305959cf0cc: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy" id=f77a162b-a3d1-4c5c-851c-a35383971859 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:17.251924 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:17.251888 1563 scope.go:110] "RemoveContainer" containerID="f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953" Nov 16 14:51:17.253286 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:17.253248920Z" level=info msg="Removing container: f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953" id=576da398-f9f4-474b-bd75-cd6bbd76bc69 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:17.308966 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:17.308877430Z" level=info msg="Removed container f53325ceda84931a786305678e1fcc8c8dbd1c1f965614fdf2066a2be40b2953: openshift-monitoring/alertmanager-main-0/alertmanager-proxy" id=576da398-f9f4-474b-bd75-cd6bbd76bc69 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:17.309241 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:17.309211 1563 scope.go:110] "RemoveContainer" containerID="8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba" Nov 16 14:51:17.310291 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:17.310251640Z" level=info msg="Removing container: 8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba" id=517de342-bef9-4e3c-a0bf-b9499e8dcb0d name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:17.354507 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:17.354395765Z" level=info msg="Removed container 8e54c4e8f0a1450fa5f3119dbf60b05e05c4cbbe4aa0af4f7fdfad5a30adf1ba: openshift-monitoring/alertmanager-main-0/config-reloader" id=517de342-bef9-4e3c-a0bf-b9499e8dcb0d name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:17.354790 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:17.354757 1563 scope.go:110] "RemoveContainer" containerID="62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a" Nov 16 14:51:17.355921 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:17.355877676Z" level=info msg="Removing container: 62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a" id=6cc8e044-104e-492d-a208-0814e3b299dc name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:17.732217 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:17.732178 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" Nov 16 14:51:18.614937 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:18.614872748Z" level=info msg="Removed container 62d414dc8b6525e6654e85b98f4becbf6edb832e86d8c8fdb08f5960e994eb3a: openshift-monitoring/alertmanager-main-0/alertmanager" id=6cc8e044-104e-492d-a208-0814e3b299dc name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:51:18.636177 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.632854 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 14:51:18.657285 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.657241 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 14:51:18.675540 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.675466 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/qe-app-registry-9hthz" Nov 16 14:51:18.675540 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.675521 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/qe-app-registry-9hthz" Nov 16 14:51:18.745741 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.745691 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/qe-app-registry-9hthz" Nov 16 14:51:18.783105 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783015 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 14:51:18.783105 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783115 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:51:18.783354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:18.783244 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="kube-rbac-proxy" Nov 16 14:51:18.783354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783259 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="kube-rbac-proxy" Nov 16 14:51:18.783354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:18.783271 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="prom-label-proxy" Nov 16 14:51:18.783354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783279 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="prom-label-proxy" Nov 16 14:51:18.783354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:18.783291 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="alertmanager-proxy" Nov 16 14:51:18.783354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783300 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="alertmanager-proxy" Nov 16 14:51:18.783354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:18.783309 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="590da926-dc2c-4378-a471-69dd415ae65f" containerName="registry-server" Nov 16 14:51:18.783354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783318 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="590da926-dc2c-4378-a471-69dd415ae65f" containerName="registry-server" Nov 16 14:51:18.783354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:18.783329 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="alertmanager" Nov 16 14:51:18.783354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783339 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="alertmanager" Nov 16 14:51:18.783354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:18.783350 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="kube-rbac-proxy-metric" Nov 16 14:51:18.783354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783359 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="kube-rbac-proxy-metric" Nov 16 14:51:18.783701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:18.783372 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="config-reloader" Nov 16 14:51:18.783701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783380 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="config-reloader" Nov 16 14:51:18.783701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783463 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="config-reloader" Nov 16 14:51:18.783701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783477 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="prom-label-proxy" Nov 16 14:51:18.783701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783487 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="alertmanager" Nov 16 14:51:18.783701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783496 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="alertmanager-proxy" Nov 16 14:51:18.783701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783507 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="kube-rbac-proxy-metric" Nov 16 14:51:18.783701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783516 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="d002c6da-b2c9-44ec-827f-97e4f83396f4" containerName="kube-rbac-proxy" Nov 16 14:51:18.783701 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.783526 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="590da926-dc2c-4378-a471-69dd415ae65f" containerName="registry-server" Nov 16 14:51:18.794215 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pode0a5ccca_d055_4b4c_820e_515fbf3f92d5.slice. Nov 16 14:51:18.889878 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.889705 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.889878 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.889803 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.890066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.889880 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7lr9\" (UniqueName: \"kubernetes.io/projected/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-kube-api-access-n7lr9\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.890066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.889929 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-config-volume\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.890066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.889981 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-main-proxy\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.890066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.890013 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.890066 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.890041 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.890263 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.890074 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.890263 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.890157 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.890263 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.890228 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.991305 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.991222 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.991305 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.991296 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.991519 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.991335 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-n7lr9\" (UniqueName: \"kubernetes.io/projected/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-kube-api-access-n7lr9\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.991519 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.991370 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-config-volume\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.991519 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.991416 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-main-proxy\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.991519 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.991451 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.991519 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.991482 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.991519 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.991516 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.991699 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.991555 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.991699 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.991589 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.992568 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.992504 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.992888 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.992854 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.993057 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.993029 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.995027 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.994992 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.995225 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.995197 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.995286 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.995229 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.995697 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.995599 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.995697 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.995651 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-main-proxy\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:18.995831 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:18.995791 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-config-volume\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:19.019464 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:19.019428 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7lr9\" (UniqueName: \"kubernetes.io/projected/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-kube-api-access-n7lr9\") pod \"alertmanager-main-0\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:19.028336 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:19.028301 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/qe-app-registry-9hthz" Nov 16 14:51:19.107589 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:19.107518 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:19.108316 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:19.108247984Z" level=info msg="Running pod sandbox: openshift-monitoring/alertmanager-main-0/POD" id=3706a43e-0e72-4185-bbe7-7f316dce42ec name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:51:19.108420 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:19.108334485Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:19.177567 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:19.177397494Z" level=info msg="Got pod network &{Name:alertmanager-main-0 Namespace:openshift-monitoring ID:a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe UID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 NetNS:/var/run/netns/da68e262-d66c-49ee-9218-827c778cb004 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:51:19.177567 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:19.177450694Z" level=info msg="Adding pod openshift-monitoring_alertmanager-main-0 to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:51:19.222278 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:51:19.246207 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): vethb22da9eb: link is not ready Nov 16 14:51:19.247535 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethb22da9eb: link becomes ready Nov 16 14:51:19.247635 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:51:19.252363 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[181200]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:51:19.252558 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[181200]: Could not generate persistent MAC address for vethb22da9eb: No such file or directory Nov 16 14:51:19.252770 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610279.2526] device (vethb22da9eb): carrier: link connected Nov 16 14:51:19.253137 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610279.2531] manager: (vethb22da9eb): new Veth device (/org/freedesktop/NetworkManager/Devices/80) Nov 16 14:51:19.279179 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610279.2790] manager: (vethb22da9eb): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/81) Nov 16 14:51:19.288075 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00218|bridge|INFO|bridge br0: added interface vethb22da9eb on port 38 Nov 16 14:51:19.288495 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethb22da9eb entered promiscuous mode Nov 16 14:51:19.323391 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00219|connmgr|INFO|br0<->unix#478: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:51:19.370498 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00220|connmgr|INFO|br0<->unix#481: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:51:20.595726 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:20.595680 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=d002c6da-b2c9-44ec-827f-97e4f83396f4 path="/var/lib/kubelet/pods/d002c6da-b2c9-44ec-827f-97e4f83396f4/volumes" Nov 16 14:51:21.357452 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:51:21Z [verbose] Add: openshift-monitoring:alertmanager-main-0:e0a5ccca-d055-4b4c-820e-515fbf3f92d5:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/da68e262-d66c-49ee-9218-827c778cb004"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.37/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:51:21.357452 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:51:21.317362 181173 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-monitoring", Name:"alertmanager-main-0", UID:"e0a5ccca-d055-4b4c-820e-515fbf3f92d5", APIVersion:"v1", ResourceVersion:"97804", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.37/23] from openshift-sdn Nov 16 14:51:21.358084 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.358035766Z" level=info msg="Got pod network &{Name:alertmanager-main-0 Namespace:openshift-monitoring ID:a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe UID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 NetNS:/var/run/netns/da68e262-d66c-49ee-9218-827c778cb004 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:51:21.358266 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.358235868Z" level=info msg="Checking pod openshift-monitoring_alertmanager-main-0 for CNI network multus-cni-network (type=multus)" Nov 16 14:51:21.359331 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:21.359265 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 14:51:21.360908 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:51:21.360866 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0a5ccca_d055_4b4c_820e_515fbf3f92d5.slice/crio-a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe.scope WatchSource:0}: Error finding container a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe: Status 404 returned error &{%!s(*http.body=&{0xc0025d6978 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:51:21.371053 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.370959261Z" level=info msg="Ran pod sandbox a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe with infra container: openshift-monitoring/alertmanager-main-0/POD" id=3706a43e-0e72-4185-bbe7-7f316dce42ec name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:51:21.372146 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.372075070Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a577c235d1c867f23bb6e2243abba44bf63f4218cd9a9307c06bfd701861dbe" id=c7f30c59-3ea9-494e-90ae-92bdfdfaecc4 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:21.372461 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.372386672Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a577c235d1c867f23bb6e2243abba44bf63f4218cd9a9307c06bfd701861dbe not found" id=c7f30c59-3ea9-494e-90ae-92bdfdfaecc4 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:21.373395 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.373299879Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a577c235d1c867f23bb6e2243abba44bf63f4218cd9a9307c06bfd701861dbe" id=169e7604-04b0-4e0f-b3dd-c3999f03790f name=/runtime.v1.ImageService/PullImage Nov 16 14:51:21.376354 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.376276801Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a577c235d1c867f23bb6e2243abba44bf63f4218cd9a9307c06bfd701861dbe\"" Nov 16 14:51:21.718671 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.718507423Z" level=info msg="Stopping pod sandbox: 765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85" id=4bcd9067-a215-4dac-ac69-19feb5ca6248 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.718671 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.718559523Z" level=info msg="Stopped pod sandbox (already stopped): 765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85" id=4bcd9067-a215-4dac-ac69-19feb5ca6248 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.718905 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.718869226Z" level=info msg="Removing pod sandbox: 765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85" id=225236f4-f030-4f70-9d5d-0930d015cbd5 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.736581 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.736494856Z" level=info msg="Removed pod sandbox: 765ca4c9f5fd9d17241fa9a8681706083f4098ce1f3c3a8e6b1720a021fcef85" id=225236f4-f030-4f70-9d5d-0930d015cbd5 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.736948 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.736909459Z" level=info msg="Stopping pod sandbox: e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e" id=480773e7-2a0c-45cb-99c2-214837995db7 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.737006 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.736953659Z" level=info msg="Stopped pod sandbox (already stopped): e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e" id=480773e7-2a0c-45cb-99c2-214837995db7 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.737251 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.737215961Z" level=info msg="Removing pod sandbox: e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e" id=15434fe6-0e7b-4f61-b7aa-622d5d3861eb name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.756385 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.756293401Z" level=info msg="Removed pod sandbox: e88ec8ae2a5a620e422332d1839bc18af7a145232f9ae157afa55ab38439363e" id=15434fe6-0e7b-4f61-b7aa-622d5d3861eb name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.756768 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.756725405Z" level=info msg="Stopping pod sandbox: dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669" id=a8c0794b-0f35-4784-8578-8c8d3d6c9306 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.756768 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.756767405Z" level=info msg="Stopped pod sandbox (already stopped): dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669" id=a8c0794b-0f35-4784-8578-8c8d3d6c9306 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.757077 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.757046807Z" level=info msg="Removing pod sandbox: dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669" id=a54e02cf-0247-4d73-96f8-af0d4e5e4d12 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.774674 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.774636237Z" level=info msg="Removed pod sandbox: dd1066654a6dd7de7a686fe2052b9029a76bc6849e74d0a2bc952994deae8669" id=a54e02cf-0247-4d73-96f8-af0d4e5e4d12 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.775061 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.775029840Z" level=info msg="Stopping pod sandbox: 56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58" id=e9567888-a9dd-458d-a21f-5de37f75a2bd name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.775155 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.775065940Z" level=info msg="Stopped pod sandbox (already stopped): 56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58" id=e9567888-a9dd-458d-a21f-5de37f75a2bd name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.775373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.775343242Z" level=info msg="Removing pod sandbox: 56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58" id=2d756b99-48c9-40a1-8164-1f9be01d7b3b name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.794302 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.794264881Z" level=info msg="Removed pod sandbox: 56e6b9a2f719b3cd3afcd87abc9d85ace7e38d6a40c1f415f3b4b93fec47fa58" id=2d756b99-48c9-40a1-8164-1f9be01d7b3b name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.794615 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.794580284Z" level=info msg="Stopping pod sandbox: eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5" id=898a83fc-a3be-4866-80d3-ca43e2ca58a9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.794696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.794620684Z" level=info msg="Stopped pod sandbox (already stopped): eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5" id=898a83fc-a3be-4866-80d3-ca43e2ca58a9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.794955 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.794921386Z" level=info msg="Removing pod sandbox: eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5" id=8295e5da-e287-4cd3-a758-9ac99ca7ab00 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.813945 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.813504223Z" level=info msg="Removed pod sandbox: eb022b9f1ea524c234d7f4e8caea5c8c8bf8c1a0e7418db4ee908d5b9c89a6b5" id=8295e5da-e287-4cd3-a758-9ac99ca7ab00 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.814084 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.813994727Z" level=info msg="Stopping pod sandbox: 50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc" id=c711a9df-e331-47e9-a143-2f3387331bab name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.814084 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.814029827Z" level=info msg="Stopped pod sandbox (already stopped): 50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc" id=c711a9df-e331-47e9-a143-2f3387331bab name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.814458 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.814420930Z" level=info msg="Removing pod sandbox: 50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc" id=58065cdc-f916-4e80-8e97-3dd450e849f7 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.834515 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.834401177Z" level=info msg="Removed pod sandbox: 50b0ce206f4641f2f614dcc98f9eddccc8ee5f3c99f22df757ecf2431b51e1bc" id=58065cdc-f916-4e80-8e97-3dd450e849f7 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.834985 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.834910881Z" level=info msg="Stopping pod sandbox: 502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b" id=098a4944-9e16-4619-8cbf-5da2acc369a6 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.834985 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.834954881Z" level=info msg="Stopped pod sandbox (already stopped): 502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b" id=098a4944-9e16-4619-8cbf-5da2acc369a6 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:51:21.835264 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.835231783Z" level=info msg="Removing pod sandbox: 502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b" id=8365011f-c598-4862-9ef8-ac2f92d7c294 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.852897 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:21.852823413Z" level=info msg="Removed pod sandbox: 502036d84002e7762a8b197b50883a48c2cce443c335df64e027d8edfb56596b" id=8365011f-c598-4862-9ef8-ac2f92d7c294 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:51:21.977430 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:21.977253 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerStarted Data:a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe} Nov 16 14:51:22.682551 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:22.682451228Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a577c235d1c867f23bb6e2243abba44bf63f4218cd9a9307c06bfd701861dbe\"" Nov 16 14:51:23.036636 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:51:23.036441 1563 controller.go:187] failed to update lease, error: Put "https://api-int.ropatil1611az.qe.azure.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ropatil1611az-gs6zj-worker-westus-j4ngg?timeout=10s": context deadline exceeded Nov 16 14:51:25.686232 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:25.686194 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" Nov 16 14:51:25.878985 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:25.878925387Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a577c235d1c867f23bb6e2243abba44bf63f4218cd9a9307c06bfd701861dbe" id=169e7604-04b0-4e0f-b3dd-c3999f03790f name=/runtime.v1.ImageService/PullImage Nov 16 14:51:25.879942 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:25.879898895Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a577c235d1c867f23bb6e2243abba44bf63f4218cd9a9307c06bfd701861dbe" id=70045464-7269-4ded-b8ff-d9ca990ad61a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:25.881489 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:25.881435106Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:7d9460c8d8a57033bb26751af2a4a6df48cce87a1f6a191dc2752045f704f80b,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a577c235d1c867f23bb6e2243abba44bf63f4218cd9a9307c06bfd701861dbe],Size_:353116796,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=70045464-7269-4ded-b8ff-d9ca990ad61a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:25.882707 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:25.882670715Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/alertmanager" id=4da3983c-a286-4de7-81fe-4bd0b1ab289a name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:25.882800 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:25.882773216Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:25.953120 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972.scope. Nov 16 14:51:25.988111 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972. Nov 16 14:51:26.025661 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:26.087675 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.087616226Z" level=info msg="Created container 96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972: openshift-monitoring/alertmanager-main-0/alertmanager" id=4da3983c-a286-4de7-81fe-4bd0b1ab289a name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:26.088239 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.088199330Z" level=info msg="Starting container: 96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972" id=8aa80402-063b-46ae-9da8-785e7f88a5bc name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:26.099844 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.099788315Z" level=info msg="Started container" PID=181495 containerID=96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972 description=openshift-monitoring/alertmanager-main-0/alertmanager id=8aa80402-063b-46ae-9da8-785e7f88a5bc name=/runtime.v1.RuntimeService/StartContainer sandboxID=a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe Nov 16 14:51:26.122639 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.122583683Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3" id=e10894dc-44b4-4196-8f97-2caca94a9992 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.122863 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.122828685Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:f4a6bddcf29f1036d73e15b666897a7d42426b1a799e533d71d356d6f22b6826,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3],Size_:344385412,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=e10894dc-44b4-4196-8f97-2caca94a9992 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.128803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.128765329Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3" id=8b5e2a81-c42d-4370-a988-2212873281fc name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.129006 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.128971030Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:f4a6bddcf29f1036d73e15b666897a7d42426b1a799e533d71d356d6f22b6826,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3],Size_:344385412,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=8b5e2a81-c42d-4370-a988-2212873281fc name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.130108 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.130071338Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/config-reloader" id=0c5967d6-5468-4a57-8760-00026d93d2ad name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:26.130230 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.130199439Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:26.205520 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518.scope. Nov 16 14:51:26.226670 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518. Nov 16 14:51:26.264010 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:26.327749 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.327657497Z" level=info msg="Created container 27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518: openshift-monitoring/alertmanager-main-0/config-reloader" id=0c5967d6-5468-4a57-8760-00026d93d2ad name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:26.328365 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.328331002Z" level=info msg="Starting container: 27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518" id=c4c842ed-9f82-4377-ae13-b6164b355cea name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:26.339711 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.339662786Z" level=info msg="Started container" PID=181550 containerID=27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518 description=openshift-monitoring/alertmanager-main-0/config-reloader id=c4c842ed-9f82-4377-ae13-b6164b355cea name=/runtime.v1.RuntimeService/StartContainer sandboxID=a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe Nov 16 14:51:26.359482 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.359432632Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634" id=a8709001-2ae7-4c25-88f2-37737317db69 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.359712 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.359677434Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:2f3a1ac7bb93faa2312d7fca21ca26dc55cbcd57502b470ad0bc40866a6ce73d,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634],Size_:340927490,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=a8709001-2ae7-4c25-88f2-37737317db69 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.364832 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.364751871Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634" id=d6636a40-d8a7-48da-a675-88d6ca5760f9 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.364973 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.364939473Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:2f3a1ac7bb93faa2312d7fca21ca26dc55cbcd57502b470ad0bc40866a6ce73d,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634],Size_:340927490,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=d6636a40-d8a7-48da-a675-88d6ca5760f9 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.366061 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.366024781Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/alertmanager-proxy" id=20219d1f-ffe5-4673-94dd-e35b57950c24 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:26.366178 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.366152582Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:26.430619 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651.scope. Nov 16 14:51:26.462639 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651. Nov 16 14:51:26.504916 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:26.571865 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.571760504Z" level=info msg="Created container c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651: openshift-monitoring/alertmanager-main-0/alertmanager-proxy" id=20219d1f-ffe5-4673-94dd-e35b57950c24 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:26.572576 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.572486909Z" level=info msg="Starting container: c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651" id=499d1044-2b14-4143-970d-3121299055f4 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:26.584387 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.584341597Z" level=info msg="Started container" PID=181600 containerID=c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651 description=openshift-monitoring/alertmanager-main-0/alertmanager-proxy id=499d1044-2b14-4143-970d-3121299055f4 name=/runtime.v1.RuntimeService/StartContainer sandboxID=a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe Nov 16 14:51:26.627880 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.627811919Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=0560cca9-4fc9-418c-9011-dce02dd195a1 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.628104 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.628070721Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=0560cca9-4fc9-418c-9011-dce02dd195a1 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.629151 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.629036028Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=92df1171-af67-404a-87c2-c0b5a7986441 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.629329 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.629292830Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=92df1171-af67-404a-87c2-c0b5a7986441 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.630617 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.630580139Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy" id=c74f3309-d38e-4677-8d4d-4046cec7ae39 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:26.630747 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.630731841Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:26.697381 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1.scope. Nov 16 14:51:26.725775 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1. Nov 16 14:51:26.761750 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:26.835459 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.835324755Z" level=info msg="Created container aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy" id=c74f3309-d38e-4677-8d4d-4046cec7ae39 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:26.836123 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.836050461Z" level=info msg="Starting container: aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1" id=84f7a187-6cd1-4241-998a-d9941a25d6cf name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:26.848105 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.848025749Z" level=info msg="Started container" PID=181649 containerID=aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1 description=openshift-monitoring/alertmanager-main-0/kube-rbac-proxy id=84f7a187-6cd1-4241-998a-d9941a25d6cf name=/runtime.v1.RuntimeService/StartContainer sandboxID=a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe Nov 16 14:51:26.871178 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.871115120Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=4b3f6d2a-d2ec-4f2e-bf89-ff78db62248f name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.871426 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.871384122Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=4b3f6d2a-d2ec-4f2e-bf89-ff78db62248f name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.872292 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.872231328Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=109b3be2-059a-4754-b9d2-c7e6a135be9f name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.872467 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.872423930Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=109b3be2-059a-4754-b9d2-c7e6a135be9f name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:26.873584 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.873522238Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric" id=ad6305b4-cf69-48fc-b3ed-497a6f160d83 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:26.873685 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:26.873670239Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:26.928283 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3.scope. Nov 16 14:51:26.952463 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3. Nov 16 14:51:26.993984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:26.993860 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerStarted Data:aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1} Nov 16 14:51:26.993984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:26.993910 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerStarted Data:c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651} Nov 16 14:51:26.993984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:26.993932 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerStarted Data:27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518} Nov 16 14:51:26.993984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:26.993947 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerStarted Data:96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972} Nov 16 14:51:26.994153 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:27.062411 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:27.062349636Z" level=info msg="Created container d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric" id=ad6305b4-cf69-48fc-b3ed-497a6f160d83 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:27.063113 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:27.062989441Z" level=info msg="Starting container: d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3" id=5269f6cf-8329-4c7a-8932-fe278a83d254 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:27.076560 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:27.076495341Z" level=info msg="Started container" PID=181697 containerID=d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3 description=openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric id=5269f6cf-8329-4c7a-8932-fe278a83d254 name=/runtime.v1.RuntimeService/StartContainer sandboxID=a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe Nov 16 14:51:27.099507 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:27.099453611Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:415636a81f07dca43e4ec03e27bea870ab8e35da872754afc4cc19b575728cdb" id=82ae7888-1157-410f-9833-2c13c016575d name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:27.099814 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:27.099775913Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:4b1f43c95dae6ae2456299c22670bf9d5d8d59653113150a06448e63a57b7802,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:415636a81f07dca43e4ec03e27bea870ab8e35da872754afc4cc19b575728cdb],Size_:308849654,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=82ae7888-1157-410f-9833-2c13c016575d name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:27.100704 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:27.100661320Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:415636a81f07dca43e4ec03e27bea870ab8e35da872754afc4cc19b575728cdb" id=161e741c-2cac-4e9e-a3a7-b3d34b34516c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:27.100896 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:27.100861021Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:4b1f43c95dae6ae2456299c22670bf9d5d8d59653113150a06448e63a57b7802,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:415636a81f07dca43e4ec03e27bea870ab8e35da872754afc4cc19b575728cdb],Size_:308849654,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=161e741c-2cac-4e9e-a3a7-b3d34b34516c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:51:27.101813 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:27.101775628Z" level=info msg="Creating container: openshift-monitoring/alertmanager-main-0/prom-label-proxy" id=014fa07a-4976-48ce-a39e-92e8854380cb name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:27.101926 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:27.101899529Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:51:27.154927 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5.scope. Nov 16 14:51:27.175940 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5. Nov 16 14:51:27.210562 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:51:27.288082 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:27.287897706Z" level=info msg="Created container eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5: openshift-monitoring/alertmanager-main-0/prom-label-proxy" id=014fa07a-4976-48ce-a39e-92e8854380cb name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:51:27.288628 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:27.288588611Z" level=info msg="Starting container: eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5" id=edc72fd3-6dd4-433a-88e3-f4cc291cc20b name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:51:27.299791 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:51:27.299702193Z" level=info msg="Started container" PID=181759 containerID=eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5 description=openshift-monitoring/alertmanager-main-0/prom-label-proxy id=edc72fd3-6dd4-433a-88e3-f4cc291cc20b name=/runtime.v1.RuntimeService/StartContainer sandboxID=a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe Nov 16 14:51:28.000264 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:28.000177 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerStarted Data:eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5} Nov 16 14:51:28.000264 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:28.000242 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerStarted Data:d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3} Nov 16 14:51:28.719317 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.qaHv91.mount: Succeeded. Nov 16 14:51:28.719342 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.h5haG9.mount: Succeeded. Nov 16 14:51:29.108939 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:29.108820 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:45.321770 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.30dTSI.mount: Succeeded. Nov 16 14:51:49.172313 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:51:49.172275 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/alertmanager-main-0" Nov 16 14:51:58.718729 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.aB8WFs.mount: Succeeded. Nov 16 14:52:08.722178 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.ySeKip.mount: Succeeded. Nov 16 14:52:08.722205 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.s3jhpw.mount: Succeeded. Nov 16 14:52:18.971767 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.BJahWW.mount: Succeeded. Nov 16 14:52:18.971817 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.Fya4I2.mount: Succeeded. Nov 16 14:52:20.310333 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.z6AVxD.mount: Succeeded. Nov 16 14:52:38.721966 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.lUpyx5.mount: Succeeded. Nov 16 14:52:38.721991 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.GnqDXa.mount: Succeeded. Nov 16 14:52:48.719291 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.VE5ujw.mount: Succeeded. Nov 16 14:53:08.720856 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.z30iBg.mount: Succeeded. Nov 16 14:53:15.312537 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.HCn2dn.mount: Succeeded. Nov 16 14:53:20.492391 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:20.492174101Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=53ce04f1-45e1-4699-9d6e-58f3748567d6 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:20.616323 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:20.492439403Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=53ce04f1-45e1-4699-9d6e-58f3748567d6 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:33.180444 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.180328 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z] Nov 16 14:53:33.180444 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.180451 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:53:33.191992 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod6037f731_36d1_49c8_a598_34b98d642a8e.slice. Nov 16 14:53:33.309401 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.309358 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cloud-config\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-host-cloud-config\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.309401 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.309411 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-devices-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-sys-devices-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.309641 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.309455 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scsi-host-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-scsi-host-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.309641 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.309504 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzmmx\" (UniqueName: \"kubernetes.io/projected/6037f731-36d1-49c8-a598-34b98d642a8e-kube-api-access-pzmmx\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.309641 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.309551 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"msi\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-msi\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.309641 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.309580 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-mountpoint-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.309641 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.309620 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6037f731-36d1-49c8-a598-34b98d642a8e-non-standard-root-system-trust-ca-bundle\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.309871 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.309744 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-registration-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.309871 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.309787 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-device-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.309871 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.309835 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"merged-cloud-config\" (UniqueName: \"kubernetes.io/empty-dir/6037f731-36d1-49c8-a598-34b98d642a8e-merged-cloud-config\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.309871 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.309865 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-socket-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.410453 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410355 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-cloud-config\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-host-cloud-config\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.410453 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410444 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"sys-devices-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-sys-devices-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.410678 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410488 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"scsi-host-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-scsi-host-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.410678 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410497 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-cloud-config\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-host-cloud-config\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.410678 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410526 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-pzmmx\" (UniqueName: \"kubernetes.io/projected/6037f731-36d1-49c8-a598-34b98d642a8e-kube-api-access-pzmmx\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.410678 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410561 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"msi\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-msi\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.410678 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410578 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"scsi-host-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-scsi-host-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.410678 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410594 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-mountpoint-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.410678 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410624 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"msi\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-msi\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.410678 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410631 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6037f731-36d1-49c8-a598-34b98d642a8e-non-standard-root-system-trust-ca-bundle\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.410678 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410619 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"sys-devices-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-sys-devices-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.410678 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410666 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-registration-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.411002 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410692 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-mountpoint-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.411002 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410713 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-device-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.411002 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410756 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"merged-cloud-config\" (UniqueName: \"kubernetes.io/empty-dir/6037f731-36d1-49c8-a598-34b98d642a8e-merged-cloud-config\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.411002 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410767 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-registration-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.411002 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410790 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-socket-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.411002 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410826 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-device-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.411002 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.410982 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6037f731-36d1-49c8-a598-34b98d642a8e-socket-dir\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.411246 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.411152 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"merged-cloud-config\" (UniqueName: \"kubernetes.io/empty-dir/6037f731-36d1-49c8-a598-34b98d642a8e-merged-cloud-config\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.411900 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.411761 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6037f731-36d1-49c8-a598-34b98d642a8e-non-standard-root-system-trust-ca-bundle\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.432049 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.431848 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzmmx\" (UniqueName: \"kubernetes.io/projected/6037f731-36d1-49c8-a598-34b98d642a8e-kube-api-access-pzmmx\") pod \"azure-file-csi-driver-node-pzz7z\" (UID: \"6037f731-36d1-49c8-a598-34b98d642a8e\") " pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.509576 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:33.509456 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" Nov 16 14:53:33.510375 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:33.510297043Z" level=info msg="Running pod sandbox: openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/POD" id=712dd3f6-3bb4-407e-9328-73ae7c9c9046 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:53:33.510777 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:33.510396143Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:53:33.562775 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:33.562662035Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=712dd3f6-3bb4-407e-9328-73ae7c9c9046 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:53:33.567638 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:53:33.567592 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6037f731_36d1_49c8_a598_34b98d642a8e.slice/crio-16cab1ec386fae6ba266f25656c4261b3fe6e7a09e9329a4bcc65a031bfab5cc.scope WatchSource:0}: Error finding container 16cab1ec386fae6ba266f25656c4261b3fe6e7a09e9329a4bcc65a031bfab5cc: Status 404 returned error &{%!s(*http.body=&{0xc002ebdb00 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:53:33.581623 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:33.581581877Z" level=info msg="Ran pod sandbox 16cab1ec386fae6ba266f25656c4261b3fe6e7a09e9329a4bcc65a031bfab5cc with infra container: openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/POD" id=712dd3f6-3bb4-407e-9328-73ae7c9c9046 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:53:33.582671 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:33.582619984Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:949e941fd91b2355bf6b755c5f036d170af7e07f5396d9a90177f5871fcbf6ba" id=6d5d4c2d-2fea-4f9a-8878-18be96b6dcd3 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:33.582909 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:33.582880986Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:949e941fd91b2355bf6b755c5f036d170af7e07f5396d9a90177f5871fcbf6ba not found" id=6d5d4c2d-2fea-4f9a-8878-18be96b6dcd3 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:33.583475 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:33.583435690Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:949e941fd91b2355bf6b755c5f036d170af7e07f5396d9a90177f5871fcbf6ba" id=8d19996e-97a7-4632-8801-f93fd09dda5e name=/runtime.v1.ImageService/PullImage Nov 16 14:53:33.586428 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:33.586390113Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:949e941fd91b2355bf6b755c5f036d170af7e07f5396d9a90177f5871fcbf6ba\"" Nov 16 14:53:34.407675 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:34.407634 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" event=&{ID:6037f731-36d1-49c8-a598-34b98d642a8e Type:ContainerStarted Data:16cab1ec386fae6ba266f25656c4261b3fe6e7a09e9329a4bcc65a031bfab5cc} Nov 16 14:53:34.827067 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:34.826906453Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:949e941fd91b2355bf6b755c5f036d170af7e07f5396d9a90177f5871fcbf6ba\"" Nov 16 14:53:38.762035 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:38.761948415Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:949e941fd91b2355bf6b755c5f036d170af7e07f5396d9a90177f5871fcbf6ba" id=8d19996e-97a7-4632-8801-f93fd09dda5e name=/runtime.v1.ImageService/PullImage Nov 16 14:53:38.763018 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:38.762979325Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:949e941fd91b2355bf6b755c5f036d170af7e07f5396d9a90177f5871fcbf6ba" id=f46e3f11-7379-4a11-9a9b-8a85c121016a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:38.764792 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:38.764719741Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5a4effe21da47ccc0ff77199890d3c1763c74455a1b0c52b3ebadb60c2b341ae,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:949e941fd91b2355bf6b755c5f036d170af7e07f5396d9a90177f5871fcbf6ba],Size_:438790711,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=f46e3f11-7379-4a11-9a9b-8a85c121016a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:38.765815 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:38.765770451Z" level=info msg="Creating container: openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/azure-inject-credentials" id=41c60aa4-8b12-4fc7-9ea0-dbf59a2b10fa name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:53:38.765926 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:38.765909552Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:53:38.828197 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-26c6d1476d77d9f2592ab82eded05572cb64042aa814bb61c786b250da7bd3cd.scope. Nov 16 14:53:38.856820 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 26c6d1476d77d9f2592ab82eded05572cb64042aa814bb61c786b250da7bd3cd. Nov 16 14:53:38.893100 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:53:38.955384 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:38.955285036Z" level=info msg="Created container 26c6d1476d77d9f2592ab82eded05572cb64042aa814bb61c786b250da7bd3cd: openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/azure-inject-credentials" id=41c60aa4-8b12-4fc7-9ea0-dbf59a2b10fa name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:53:38.955943 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:38.955893542Z" level=info msg="Starting container: 26c6d1476d77d9f2592ab82eded05572cb64042aa814bb61c786b250da7bd3cd" id=826c6c67-72bc-4aa6-96d3-2f37da48cab8 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:53:38.968629 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:38.968526961Z" level=info msg="Started container" PID=185389 containerID=26c6d1476d77d9f2592ab82eded05572cb64042aa814bb61c786b250da7bd3cd description=openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/azure-inject-credentials id=826c6c67-72bc-4aa6-96d3-2f37da48cab8 name=/runtime.v1.RuntimeService/StartContainer sandboxID=16cab1ec386fae6ba266f25656c4261b3fe6e7a09e9329a4bcc65a031bfab5cc Nov 16 14:53:38.988043 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-26c6d1476d77d9f2592ab82eded05572cb64042aa814bb61c786b250da7bd3cd.scope: Succeeded. Nov 16 14:53:38.988814 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-26c6d1476d77d9f2592ab82eded05572cb64042aa814bb61c786b250da7bd3cd.scope: Consumed 31ms CPU time Nov 16 14:53:38.989167 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-26c6d1476d77d9f2592ab82eded05572cb64042aa814bb61c786b250da7bd3cd.scope: Succeeded. Nov 16 14:53:38.989740 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-26c6d1476d77d9f2592ab82eded05572cb64042aa814bb61c786b250da7bd3cd.scope: Consumed 38ms CPU time Nov 16 14:53:39.425612 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:39.425571 1563 generic.go:296] "Generic (PLEG): container finished" podID=6037f731-36d1-49c8-a598-34b98d642a8e containerID="26c6d1476d77d9f2592ab82eded05572cb64042aa814bb61c786b250da7bd3cd" exitCode=0 Nov 16 14:53:39.425612 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:39.425630 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" event=&{ID:6037f731-36d1-49c8-a598-34b98d642a8e Type:ContainerDied Data:26c6d1476d77d9f2592ab82eded05572cb64042aa814bb61c786b250da7bd3cd} Nov 16 14:53:40.311687 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.mLPxXu.mount: Succeeded. Nov 16 14:53:40.430064 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:40.429972025Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d51579abb284b9171580255238488fba387ec30d001261ec21b30e243898c98c" id=f52a87da-7a87-430f-bee7-9287459f212a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:40.430635 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:40.430397329Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d51579abb284b9171580255238488fba387ec30d001261ec21b30e243898c98c not found" id=f52a87da-7a87-430f-bee7-9287459f212a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:40.431107 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:40.431052235Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d51579abb284b9171580255238488fba387ec30d001261ec21b30e243898c98c" id=032e7870-88d7-4b85-b4ae-2e593ac0eeef name=/runtime.v1.ImageService/PullImage Nov 16 14:53:40.434594 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:40.434523468Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d51579abb284b9171580255238488fba387ec30d001261ec21b30e243898c98c\"" Nov 16 14:53:41.686115 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:41.685970355Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d51579abb284b9171580255238488fba387ec30d001261ec21b30e243898c98c\"" Nov 16 14:53:45.311050 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.uPIfgg.mount: Succeeded. Nov 16 14:53:46.611905 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:46.611820090Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d51579abb284b9171580255238488fba387ec30d001261ec21b30e243898c98c" id=032e7870-88d7-4b85-b4ae-2e593ac0eeef name=/runtime.v1.ImageService/PullImage Nov 16 14:53:46.617059 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:46.617020136Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d51579abb284b9171580255238488fba387ec30d001261ec21b30e243898c98c" id=cbdbb504-5b7f-4a01-a8a1-df9aa78b28fc name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:46.618786 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:46.618713251Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:978498604756b15a325e1238dc90fd4f4b2942f458d6273de0dcc42d17914cf0,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d51579abb284b9171580255238488fba387ec30d001261ec21b30e243898c98c],Size_:477828753,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=cbdbb504-5b7f-4a01-a8a1-df9aa78b28fc name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:46.620224 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:46.620169064Z" level=info msg="Creating container: openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/csi-driver" id=69df49fe-e171-49f2-8edd-2f7f2f05e1e7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:53:46.620307 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:46.620277065Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:53:46.691074 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-0f29a7a418ba35721df6bb5339038476ce3a274a934dbcb141965950aa5e0b14.scope. Nov 16 14:53:46.721322 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 0f29a7a418ba35721df6bb5339038476ce3a274a934dbcb141965950aa5e0b14. Nov 16 14:53:46.803306 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:46.803219775Z" level=info msg="Created container 0f29a7a418ba35721df6bb5339038476ce3a274a934dbcb141965950aa5e0b14: openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/csi-driver" id=69df49fe-e171-49f2-8edd-2f7f2f05e1e7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:53:46.804001 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:46.803951682Z" level=info msg="Starting container: 0f29a7a418ba35721df6bb5339038476ce3a274a934dbcb141965950aa5e0b14" id=366c40eb-06e1-498c-bd05-ddb91c8c5ebe name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:53:46.814965 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:46.814910178Z" level=info msg="Started container" PID=185548 containerID=0f29a7a418ba35721df6bb5339038476ce3a274a934dbcb141965950aa5e0b14 description=openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/csi-driver id=366c40eb-06e1-498c-bd05-ddb91c8c5ebe name=/runtime.v1.RuntimeService/StartContainer sandboxID=16cab1ec386fae6ba266f25656c4261b3fe6e7a09e9329a4bcc65a031bfab5cc Nov 16 14:53:46.837945 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:46.837829980Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f430448d09b0fbb6b90291fa808dbf00e081eb36196b23a459eab69fbeca761" id=a5d27ccc-8df2-4cdb-9cdc-b0b189f18753 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:46.838218 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:46.838188183Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f430448d09b0fbb6b90291fa808dbf00e081eb36196b23a459eab69fbeca761 not found" id=a5d27ccc-8df2-4cdb-9cdc-b0b189f18753 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:46.838879 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:46.838797888Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f430448d09b0fbb6b90291fa808dbf00e081eb36196b23a459eab69fbeca761" id=239b2b76-cc00-4c6d-b7b4-92730b85168a name=/runtime.v1.ImageService/PullImage Nov 16 14:53:46.842215 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:46.842177818Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f430448d09b0fbb6b90291fa808dbf00e081eb36196b23a459eab69fbeca761\"" Nov 16 14:53:47.453771 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:47.453733 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" event=&{ID:6037f731-36d1-49c8-a598-34b98d642a8e Type:ContainerStarted Data:0f29a7a418ba35721df6bb5339038476ce3a274a934dbcb141965950aa5e0b14} Nov 16 14:53:48.101073 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:48.100984200Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f430448d09b0fbb6b90291fa808dbf00e081eb36196b23a459eab69fbeca761\"" Nov 16 14:53:52.529373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:52.529242332Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f430448d09b0fbb6b90291fa808dbf00e081eb36196b23a459eab69fbeca761" id=239b2b76-cc00-4c6d-b7b4-92730b85168a name=/runtime.v1.ImageService/PullImage Nov 16 14:53:52.530561 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:52.530487143Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f430448d09b0fbb6b90291fa808dbf00e081eb36196b23a459eab69fbeca761" id=0835810a-9cf1-4c85-98f9-aaa25e77309e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:52.532142 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:52.532009155Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:96126ea68fc9819028bfd7e1923225924fa5cdbe2aee3ad53fb86f04f8e27da2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f430448d09b0fbb6b90291fa808dbf00e081eb36196b23a459eab69fbeca761],Size_:350900510,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=0835810a-9cf1-4c85-98f9-aaa25e77309e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:52.533330 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:52.533276065Z" level=info msg="Creating container: openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/csi-node-driver-registrar" id=aa2cdcad-5cc6-4f8d-8274-8163f07f5767 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:53:52.533412 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:52.533394566Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:53:52.595537 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-7a36172dccaf0299d594e9913f6d986397ebd2e42962d6ac7f68a6842f1357d2.scope. Nov 16 14:53:52.633859 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 7a36172dccaf0299d594e9913f6d986397ebd2e42962d6ac7f68a6842f1357d2. Nov 16 14:53:52.703757 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:52.703678762Z" level=info msg="Created container 7a36172dccaf0299d594e9913f6d986397ebd2e42962d6ac7f68a6842f1357d2: openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/csi-node-driver-registrar" id=aa2cdcad-5cc6-4f8d-8274-8163f07f5767 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:53:52.704171 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:52.704121066Z" level=info msg="Starting container: 7a36172dccaf0299d594e9913f6d986397ebd2e42962d6ac7f68a6842f1357d2" id=e2d231e8-c37a-413c-affe-de4b0dcd9e1f name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:53:52.715770 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:52.715722861Z" level=info msg="Started container" PID=185715 containerID=7a36172dccaf0299d594e9913f6d986397ebd2e42962d6ac7f68a6842f1357d2 description=openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/csi-node-driver-registrar id=e2d231e8-c37a-413c-affe-de4b0dcd9e1f name=/runtime.v1.RuntimeService/StartContainer sandboxID=16cab1ec386fae6ba266f25656c4261b3fe6e7a09e9329a4bcc65a031bfab5cc Nov 16 14:53:52.727062 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:52.727026 1563 plugin_watcher.go:203] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/file.csi.azure.com-reg.sock" Nov 16 14:53:52.737863 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:52.737815942Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6976cb9aa79c0ab33783342f2b5a6066cda4c177d0205c73b8f63e6a0641338e" id=9c1bfead-26f1-4e07-b04f-e66c6b2ec25c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:52.738152 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:52.738087744Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6976cb9aa79c0ab33783342f2b5a6066cda4c177d0205c73b8f63e6a0641338e not found" id=9c1bfead-26f1-4e07-b04f-e66c6b2ec25c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:52.738790 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:52.738720550Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6976cb9aa79c0ab33783342f2b5a6066cda4c177d0205c73b8f63e6a0641338e" id=dbe71395-2d73-4db0-b3e4-0c40ad7df804 name=/runtime.v1.ImageService/PullImage Nov 16 14:53:52.741946 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:52.741880176Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6976cb9aa79c0ab33783342f2b5a6066cda4c177d0205c73b8f63e6a0641338e\"" Nov 16 14:53:53.101644 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:53.101547 1563 reconciler.go:160] "OperationExecutor.RegisterPlugin started" plugin={SocketPath:/var/lib/kubelet/plugins_registry/file.csi.azure.com-reg.sock Timestamp:2022-11-16 14:53:52.727055854 +0000 UTC m=+10837.663831576 Handler: Name:} Nov 16 14:53:53.104053 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:53.103990 1563 csi_plugin.go:99] kubernetes.io/csi: Trying to validate a new CSI Driver with name: file.csi.azure.com endpoint: /var/lib/kubelet/plugins/file.csi.azure.com/csi.sock versions: 1.0.0 Nov 16 14:53:53.104053 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:53.104057 1563 csi_plugin.go:112] kubernetes.io/csi: Register new plugin with name: file.csi.azure.com at endpoint: /var/lib/kubelet/plugins/file.csi.azure.com/csi.sock Nov 16 14:53:53.134772 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:53:53.134704 1563 nodeinfomanager.go:566] Invalid attach limit value 0 cannot be added to CSINode object for "file.csi.azure.com" Nov 16 14:53:53.478181 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:53.478020 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" event=&{ID:6037f731-36d1-49c8-a598-34b98d642a8e Type:ContainerStarted Data:7a36172dccaf0299d594e9913f6d986397ebd2e42962d6ac7f68a6842f1357d2} Nov 16 14:53:53.987974 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:53.987874890Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6976cb9aa79c0ab33783342f2b5a6066cda4c177d0205c73b8f63e6a0641338e\"" Nov 16 14:53:56.543844 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:56.543789044Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6976cb9aa79c0ab33783342f2b5a6066cda4c177d0205c73b8f63e6a0641338e" id=dbe71395-2d73-4db0-b3e4-0c40ad7df804 name=/runtime.v1.ImageService/PullImage Nov 16 14:53:56.544976 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:56.544890753Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6976cb9aa79c0ab33783342f2b5a6066cda4c177d0205c73b8f63e6a0641338e" id=d3cc6a03-fc20-4b8b-bc1e-6eb8a8a2ea82 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:56.546692 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:56.546620367Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:e00c8045676e7d065c67da533fd2ca0aff54a5dc7f41c8b88f6b051860907dad,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6976cb9aa79c0ab33783342f2b5a6066cda4c177d0205c73b8f63e6a0641338e],Size_:348932379,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=d3cc6a03-fc20-4b8b-bc1e-6eb8a8a2ea82 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:53:56.547755 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:56.547682276Z" level=info msg="Creating container: openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/csi-liveness-probe" id=0269e5f5-0df3-4602-94e5-7baa451aa0e7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:53:56.547833 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:56.547783777Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:53:56.612503 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-a332b01ff4d82b79c9d218486500b0b83419462e198205283e44ffc0826f6ff6.scope. Nov 16 14:53:56.642903 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container a332b01ff4d82b79c9d218486500b0b83419462e198205283e44ffc0826f6ff6. Nov 16 14:53:56.678660 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:53:56.741944 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:56.741866768Z" level=info msg="Created container a332b01ff4d82b79c9d218486500b0b83419462e198205283e44ffc0826f6ff6: openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/csi-liveness-probe" id=0269e5f5-0df3-4602-94e5-7baa451aa0e7 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:53:56.742592 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:56.742548974Z" level=info msg="Starting container: a332b01ff4d82b79c9d218486500b0b83419462e198205283e44ffc0826f6ff6" id=f07c8cc8-58ab-479a-967c-ebc9f23e89a7 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:53:56.753957 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:53:56.753857566Z" level=info msg="Started container" PID=185817 containerID=a332b01ff4d82b79c9d218486500b0b83419462e198205283e44ffc0826f6ff6 description=openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z/csi-liveness-probe id=f07c8cc8-58ab-479a-967c-ebc9f23e89a7 name=/runtime.v1.RuntimeService/StartContainer sandboxID=16cab1ec386fae6ba266f25656c4261b3fe6e7a09e9329a4bcc65a031bfab5cc Nov 16 14:53:57.494276 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:53:57.494225 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-file-csi-driver-node-pzz7z" event=&{ID:6037f731-36d1-49c8-a598-34b98d642a8e Type:ContainerStarted Data:a332b01ff4d82b79c9d218486500b0b83419462e198205283e44ffc0826f6ff6} Nov 16 14:55:30.713786 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:30.713692 1563 configmap.go:200] Couldn't get configMap openshift-multus/cni-binary-copy-script: configmap "cni-binary-copy-script" not found Nov 16 14:55:30.714564 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:30.713862 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy podName:d38704ca-571d-40ee-84b3-71c1d45d56f4 nodeName:}" failed. No retries permitted until 2022-11-16 14:55:31.213837825 +0000 UTC m=+10936.150613447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cni-binary-copy" (UniqueName: "kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy") pod "multus-5wch2" (UID: "d38704ca-571d-40ee-84b3-71c1d45d56f4") : configmap "cni-binary-copy-script" not found Nov 16 14:55:31.218391 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:31.218292 1563 configmap.go:200] Couldn't get configMap openshift-multus/cni-binary-copy-script: configmap "cni-binary-copy-script" not found Nov 16 14:55:31.218391 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:31.218406 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy podName:d38704ca-571d-40ee-84b3-71c1d45d56f4 nodeName:}" failed. No retries permitted until 2022-11-16 14:55:32.218383542 +0000 UTC m=+10937.155159264 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cni-binary-copy" (UniqueName: "kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy") pod "multus-5wch2" (UID: "d38704ca-571d-40ee-84b3-71c1d45d56f4") : configmap "cni-binary-copy-script" not found Nov 16 14:55:32.231295 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:32.231205 1563 configmap.go:200] Couldn't get configMap openshift-multus/cni-binary-copy-script: configmap "cni-binary-copy-script" not found Nov 16 14:55:32.231295 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:32.231324 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy podName:d38704ca-571d-40ee-84b3-71c1d45d56f4 nodeName:}" failed. No retries permitted until 2022-11-16 14:55:34.231289599 +0000 UTC m=+10939.168065221 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cni-binary-copy" (UniqueName: "kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy") pod "multus-5wch2" (UID: "d38704ca-571d-40ee-84b3-71c1d45d56f4") : configmap "cni-binary-copy-script" not found Nov 16 14:55:34.251440 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:34.251374 1563 configmap.go:200] Couldn't get configMap openshift-multus/cni-binary-copy-script: configmap "cni-binary-copy-script" not found Nov 16 14:55:34.251974 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:34.251481 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy podName:d38704ca-571d-40ee-84b3-71c1d45d56f4 nodeName:}" failed. No retries permitted until 2022-11-16 14:55:38.251460279 +0000 UTC m=+10943.188236001 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cni-binary-copy" (UniqueName: "kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy") pod "multus-5wch2" (UID: "d38704ca-571d-40ee-84b3-71c1d45d56f4") : configmap "cni-binary-copy-script" not found Nov 16 14:55:35.024121 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:35.024077 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-sdn/sdn-pgwdx] Nov 16 14:55:35.024584 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:35.024490 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-sdn/sdn-pgwdx" podUID=45cd7283-fcc9-4e61-b2b1-517445b7bd5a containerName="kube-rbac-proxy" containerID="cri-o://516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8" gracePeriod=30 Nov 16 14:55:35.024814 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:35.024751832Z" level=info msg="Stopping container: 516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8 (timeout: 30s)" id=74daf162-3412-4032-8d38-354f64a8f35c name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:35.068767 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.D8ZUO5.mount: Succeeded. Nov 16 14:55:35.079746 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:35.079665807Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": REMOVE" Nov 16 14:55:35.081261 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:35.081224 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-sdn/sdn-pgwdx" podUID=45cd7283-fcc9-4e61-b2b1-517445b7bd5a containerName="drop-icmp" containerID="cri-o://adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19" gracePeriod=30 Nov 16 14:55:35.081588 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:35.081545120Z" level=info msg="Stopping container: adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19 (timeout: 30s)" id=27151048-5045-45de-8c2f-f974b7d92a02 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:35.082309 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:35.082278 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-sdn/sdn-pgwdx" podUID=45cd7283-fcc9-4e61-b2b1-517445b7bd5a containerName="sdn" containerID="cri-o://962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a" gracePeriod=30 Nov 16 14:55:35.082565 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:35.082519627Z" level=info msg="Stopping container: 962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a (timeout: 30s)" id=3985e792-9288-44e1-b63a-7899c6aaecdd name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:35.110806 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19.scope: Succeeded. Nov 16 14:55:35.111604 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19.scope: Consumed 2.850s CPU time Nov 16 14:55:35.116493 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19.scope: Succeeded. Nov 16 14:55:35.117061 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19.scope: Consumed 37ms CPU time Nov 16 14:55:35.272865 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-562e870189dd298285c44a678c49be1a42e12dfae4b58cf9369666167e942d9e-merged.mount: Succeeded. Nov 16 14:55:35.273694 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-562e870189dd298285c44a678c49be1a42e12dfae4b58cf9369666167e942d9e-merged.mount: Consumed 0 CPU time Nov 16 14:55:35.360901 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:35.360814 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-multus/multus-5wch2] Nov 16 14:55:35.361425 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:35.361077 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-multus/multus-5wch2" podUID=d38704ca-571d-40ee-84b3-71c1d45d56f4 containerName="kube-multus" containerID="cri-o://9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2" gracePeriod=30 Nov 16 14:55:35.361478 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:35.361401130Z" level=info msg="Stopping container: 9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2 (timeout: 30s)" id=6f978cde-a9b5-4a11-a941-4174ad5100dc name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:35.537726 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:35.537614333Z" level=info msg="Stopped container adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19: openshift-sdn/sdn-pgwdx/drop-icmp" id=27151048-5045-45de-8c2f-f974b7d92a02 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:35.537683 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a-runc.tG6llW.mount: Succeeded. Nov 16 14:55:35.557462 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:35.557417 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-sdn/sdn-pgwdx" podUID=45cd7283-fcc9-4e61-b2b1-517445b7bd5a containerName="sdn" probeResult=failure output="" Nov 16 14:55:35.615813 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a.scope: Succeeded. Nov 16 14:55:35.616611 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a.scope: Consumed 141ms CPU time Nov 16 14:55:35.619244 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a.scope: Succeeded. Nov 16 14:55:35.619844 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a.scope: Consumed 2min 8.665s CPU time Nov 16 14:55:35.806625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:35.806582 1563 generic.go:296] "Generic (PLEG): container finished" podID=45cd7283-fcc9-4e61-b2b1-517445b7bd5a containerID="adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19" exitCode=143 Nov 16 14:55:35.806625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:35.806623 1563 generic.go:296] "Generic (PLEG): container finished" podID=45cd7283-fcc9-4e61-b2b1-517445b7bd5a containerID="962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a" exitCode=0 Nov 16 14:55:35.806951 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:35.806657 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-sdn/sdn-pgwdx" event=&{ID:45cd7283-fcc9-4e61-b2b1-517445b7bd5a Type:ContainerDied Data:adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19} Nov 16 14:55:35.806951 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:35.806692 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-sdn/sdn-pgwdx" event=&{ID:45cd7283-fcc9-4e61-b2b1-517445b7bd5a Type:ContainerDied Data:962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a} Nov 16 14:55:35.900637 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2.scope: Succeeded. Nov 16 14:55:35.901353 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2.scope: Consumed 2min 33.939s CPU time Nov 16 14:55:35.925317 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2.scope: Succeeded. Nov 16 14:55:35.925987 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2.scope: Consumed 34ms CPU time Nov 16 14:55:36.001932 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:36.001779501Z" level=info msg="Stopped container 962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a: openshift-sdn/sdn-pgwdx/sdn" id=3985e792-9288-44e1-b63a-7899c6aaecdd name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:36.058905 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-802d9bc8fdd498eda77622d001b61f73d60d61de77d854d9a4850c7b5e403820-merged.mount: Succeeded. Nov 16 14:55:36.059951 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-802d9bc8fdd498eda77622d001b61f73d60d61de77d854d9a4850c7b5e403820-merged.mount: Consumed 0 CPU time Nov 16 14:55:36.059972 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-fae89971d1f4cf8b3993adbf5a7c04d976f6f20b09c8fff0c480abf53a9b4c26-merged.mount: Succeeded. Nov 16 14:55:36.060829 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-fae89971d1f4cf8b3993adbf5a7c04d976f6f20b09c8fff0c480abf53a9b4c26-merged.mount: Consumed 0 CPU time Nov 16 14:55:36.131474 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8.scope: Succeeded. Nov 16 14:55:36.132318 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8.scope: Consumed 2.958s CPU time Nov 16 14:55:36.172493 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8.scope: Succeeded. Nov 16 14:55:36.173161 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8.scope: Consumed 36ms CPU time Nov 16 14:55:36.187759 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:36.187637870Z" level=info msg="Stopped container 9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2: openshift-multus/multus-5wch2/kube-multus" id=6f978cde-a9b5-4a11-a941-4174ad5100dc name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:36.188257 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:36.188218774Z" level=info msg="Stopping pod sandbox: 31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427" id=7eaae1d1-5a18-4d1c-bfb6-24c7f1c79a84 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:55:36.197455 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-30240509\x2d523b\x2d42a8\x2d9e4b\x2d60a7bd3e7e9d.mount: Succeeded. Nov 16 14:55:36.198290 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-30240509\x2d523b\x2d42a8\x2d9e4b\x2d60a7bd3e7e9d.mount: Consumed 0 CPU time Nov 16 14:55:36.218047 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-30240509\x2d523b\x2d42a8\x2d9e4b\x2d60a7bd3e7e9d.mount: Succeeded. Nov 16 14:55:36.218856 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-30240509\x2d523b\x2d42a8\x2d9e4b\x2d60a7bd3e7e9d.mount: Consumed 0 CPU time Nov 16 14:55:36.248223 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-30240509\x2d523b\x2d42a8\x2d9e4b\x2d60a7bd3e7e9d.mount: Succeeded. Nov 16 14:55:36.248939 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-30240509\x2d523b\x2d42a8\x2d9e4b\x2d60a7bd3e7e9d.mount: Consumed 0 CPU time Nov 16 14:55:36.278640 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427-userdata-shm.mount: Succeeded. Nov 16 14:55:36.279521 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427-userdata-shm.mount: Consumed 0 CPU time Nov 16 14:55:36.476480 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:36.476295240Z" level=info msg="Stopped pod sandbox: 31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427" id=7eaae1d1-5a18-4d1c-bfb6-24c7f1c79a84 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:55:36.674228 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674151 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srjjp\" (UniqueName: \"kubernetes.io/projected/d38704ca-571d-40ee-84b3-71c1d45d56f4-kube-api-access-srjjp\") pod \"d38704ca-571d-40ee-84b3-71c1d45d56f4\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " Nov 16 14:55:36.674228 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674233 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-multus-cni-dir\") pod \"d38704ca-571d-40ee-84b3-71c1d45d56f4\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " Nov 16 14:55:36.674709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674278 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-system-cni-dir\") pod \"d38704ca-571d-40ee-84b3-71c1d45d56f4\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " Nov 16 14:55:36.674709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674307 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-os-release\") pod \"d38704ca-571d-40ee-84b3-71c1d45d56f4\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " Nov 16 14:55:36.674709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674349 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy\") pod \"d38704ca-571d-40ee-84b3-71c1d45d56f4\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " Nov 16 14:55:36.674709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674336 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-multus-cni-dir" (OuterVolumeSpecName: "multus-cni-dir") pod "d38704ca-571d-40ee-84b3-71c1d45d56f4" (UID: "d38704ca-571d-40ee-84b3-71c1d45d56f4"). InnerVolumeSpecName "multus-cni-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:36.674709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674379 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-cnibin\") pod \"d38704ca-571d-40ee-84b3-71c1d45d56f4\" (UID: \"d38704ca-571d-40ee-84b3-71c1d45d56f4\") " Nov 16 14:55:36.674709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674367 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-system-cni-dir" (OuterVolumeSpecName: "system-cni-dir") pod "d38704ca-571d-40ee-84b3-71c1d45d56f4" (UID: "d38704ca-571d-40ee-84b3-71c1d45d56f4"). InnerVolumeSpecName "system-cni-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:36.674709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674403 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-os-release" (OuterVolumeSpecName: "os-release") pod "d38704ca-571d-40ee-84b3-71c1d45d56f4" (UID: "d38704ca-571d-40ee-84b3-71c1d45d56f4"). InnerVolumeSpecName "os-release". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:36.674709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674469 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-cnibin" (OuterVolumeSpecName: "cnibin") pod "d38704ca-571d-40ee-84b3-71c1d45d56f4" (UID: "d38704ca-571d-40ee-84b3-71c1d45d56f4"). InnerVolumeSpecName "cnibin". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:36.674709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:55:36.674553 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/d38704ca-571d-40ee-84b3-71c1d45d56f4/volumes/kubernetes.io~configmap/cni-binary-copy: clearQuota called, but quotas disabled Nov 16 14:55:36.674709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674555 1563 reconciler.go:300] "Volume detached for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-system-cni-dir\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:36.674709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674625 1563 reconciler.go:300] "Volume detached for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-os-release\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:36.674709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674644 1563 reconciler.go:300] "Volume detached for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-cnibin\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:36.674709 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674660 1563 reconciler.go:300] "Volume detached for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d38704ca-571d-40ee-84b3-71c1d45d56f4-multus-cni-dir\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:36.675104 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.674763 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy" (OuterVolumeSpecName: "cni-binary-copy") pod "d38704ca-571d-40ee-84b3-71c1d45d56f4" (UID: "d38704ca-571d-40ee-84b3-71c1d45d56f4"). InnerVolumeSpecName "cni-binary-copy". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 14:55:36.698463 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.698416 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d38704ca-571d-40ee-84b3-71c1d45d56f4-kube-api-access-srjjp" (OuterVolumeSpecName: "kube-api-access-srjjp") pod "d38704ca-571d-40ee-84b3-71c1d45d56f4" (UID: "d38704ca-571d-40ee-84b3-71c1d45d56f4"). InnerVolumeSpecName "kube-api-access-srjjp". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:55:36.776299 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.776091 1563 reconciler.go:300] "Volume detached for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d38704ca-571d-40ee-84b3-71c1d45d56f4-cni-binary-copy\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:36.776299 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.776179 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-srjjp\" (UniqueName: \"kubernetes.io/projected/d38704ca-571d-40ee-84b3-71c1d45d56f4-kube-api-access-srjjp\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:36.811236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.811161 1563 generic.go:296] "Generic (PLEG): container finished" podID=d38704ca-571d-40ee-84b3-71c1d45d56f4 containerID="9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2" exitCode=0 Nov 16 14:55:36.811236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.811217 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5wch2" event=&{ID:d38704ca-571d-40ee-84b3-71c1d45d56f4 Type:ContainerDied Data:9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2} Nov 16 14:55:36.811431 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.811274 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5wch2" event=&{ID:d38704ca-571d-40ee-84b3-71c1d45d56f4 Type:ContainerDied Data:31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427} Nov 16 14:55:36.811431 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.811298 1563 scope.go:110] "RemoveContainer" containerID="9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2" Nov 16 14:55:36.812311 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:36.812269534Z" level=info msg="Removing container: 9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2" id=5c24a1ac-92b0-4e44-bed5-cdf3f28b9d5d name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:55:36.813463 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.813436 1563 generic.go:296] "Generic (PLEG): container finished" podID=45cd7283-fcc9-4e61-b2b1-517445b7bd5a containerID="516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8" exitCode=0 Nov 16 14:55:36.813565 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.813481 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-sdn/sdn-pgwdx" event=&{ID:45cd7283-fcc9-4e61-b2b1-517445b7bd5a Type:ContainerDied Data:516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8} Nov 16 14:55:36.817593 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:36.817496769Z" level=info msg="Stopped container 516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8: openshift-sdn/sdn-pgwdx/kube-rbac-proxy" id=74daf162-3412-4032-8d38-354f64a8f35c name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:36.817963 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:36.817924272Z" level=info msg="Stopping pod sandbox: 6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75" id=e4eae03d-da6d-4ffa-a123-8d4f51d1bc00 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:55:36.818729 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-podd38704ca_571d_40ee_84b3_71c1d45d56f4.slice. Nov 16 14:55:36.818796 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-podd38704ca_571d_40ee_84b3_71c1d45d56f4.slice: Consumed 2min 33.974s CPU time Nov 16 14:55:36.849010 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:36.848958584Z" level=info msg="Removed container 9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2: openshift-multus/multus-5wch2/kube-multus" id=5c24a1ac-92b0-4e44-bed5-cdf3f28b9d5d name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:55:36.849271 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.849239 1563 scope.go:110] "RemoveContainer" containerID="9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2" Nov 16 14:55:36.849693 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:36.849670 1563 remote_runtime.go:597] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2\": container with ID starting with 9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2 not found: ID does not exist" containerID="9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2" Nov 16 14:55:36.849764 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.849718 1563 pod_container_deletor.go:52] "DeleteContainer returned error" containerID={Type:cri-o ID:9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2} err="failed to get container status \"9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2\": rpc error: code = NotFound desc = could not find container \"9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2\": container with ID starting with 9db3237df775ab525e752b6083ac12d2cc7f403abee860a1efa6cc8e917b08b2 not found: ID does not exist" Nov 16 14:55:36.855462 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.855421 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-multus/multus-5wch2] Nov 16 14:55:36.863672 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.863555 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-multus/multus-5wch2] Nov 16 14:55:36.896891 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.896795 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-multus/multus-vm4st] Nov 16 14:55:36.896891 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.896862 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:55:36.897067 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:36.896976 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="d38704ca-571d-40ee-84b3-71c1d45d56f4" containerName="kube-multus" Nov 16 14:55:36.897067 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.896990 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38704ca-571d-40ee-84b3-71c1d45d56f4" containerName="kube-multus" Nov 16 14:55:36.897152 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:36.897081 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="d38704ca-571d-40ee-84b3-71c1d45d56f4" containerName="kube-multus" Nov 16 14:55:36.907765 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-podf2b609ff_b0ed_4929_b767_953432cca02b.slice. Nov 16 14:55:37.059331 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-dde49ce29cb1b7773663041493d889970716d622dd4aad918980e4175c1b8205-merged.mount: Succeeded. Nov 16 14:55:37.060211 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-dde49ce29cb1b7773663041493d889970716d622dd4aad918980e4175c1b8205-merged.mount: Consumed 0 CPU time Nov 16 14:55:37.060236 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-2e77d2274839879065038766dfd6f622829d4fca26f2400d5e9ef6572821f405-merged.mount: Succeeded. Nov 16 14:55:37.060916 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-2e77d2274839879065038766dfd6f622829d4fca26f2400d5e9ef6572821f405-merged.mount: Consumed 0 CPU time Nov 16 14:55:37.060933 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-8a70c4954bb8ac07129f6128cc633ec5c9f890aa145e38e124c9149077232b6e-merged.mount: Succeeded. Nov 16 14:55:37.061660 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-8a70c4954bb8ac07129f6128cc633ec5c9f890aa145e38e124c9149077232b6e-merged.mount: Consumed 0 CPU time Nov 16 14:55:37.061696 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-dbc9157c\x2dc6fb\x2d49ee\x2d9436\x2dcfb0096bcedd.mount: Succeeded. Nov 16 14:55:37.062424 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-dbc9157c\x2dc6fb\x2d49ee\x2d9436\x2dcfb0096bcedd.mount: Consumed 0 CPU time Nov 16 14:55:37.062439 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-dbc9157c\x2dc6fb\x2d49ee\x2d9436\x2dcfb0096bcedd.mount: Succeeded. Nov 16 14:55:37.063109 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-dbc9157c\x2dc6fb\x2d49ee\x2d9436\x2dcfb0096bcedd.mount: Consumed 0 CPU time Nov 16 14:55:37.063124 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-dbc9157c\x2dc6fb\x2d49ee\x2d9436\x2dcfb0096bcedd.mount: Succeeded. Nov 16 14:55:37.063774 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-dbc9157c\x2dc6fb\x2d49ee\x2d9436\x2dcfb0096bcedd.mount: Consumed 0 CPU time Nov 16 14:55:37.063792 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75-userdata-shm.mount: Succeeded. Nov 16 14:55:37.064246 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.064202053Z" level=info msg="Stopped pod sandbox: 6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75" id=e4eae03d-da6d-4ffa-a123-8d4f51d1bc00 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:55:37.064433 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75-userdata-shm.mount: Consumed 0 CPU time Nov 16 14:55:37.064451 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d38704ca\x2d571d\x2d40ee\x2d84b3\x2d71c1d45d56f4-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dsrjjp.mount: Succeeded. Nov 16 14:55:37.065094 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-d38704ca\x2d571d\x2d40ee\x2d84b3\x2d71c1d45d56f4-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dsrjjp.mount: Consumed 0 CPU time Nov 16 14:55:37.078379 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.078343 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b609ff-b0ed-4929-b767-953432cca02b-multus-cni-dir\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.078533 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.078402 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2b609ff-b0ed-4929-b767-953432cca02b-cnibin\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.078533 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.078441 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2b609ff-b0ed-4929-b767-953432cca02b-cni-binary-copy\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.078609 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.078543 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lk7l\" (UniqueName: \"kubernetes.io/projected/f2b609ff-b0ed-4929-b767-953432cca02b-kube-api-access-9lk7l\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.078609 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.078602 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b609ff-b0ed-4929-b767-953432cca02b-system-cni-dir\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.078707 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.078672 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2b609ff-b0ed-4929-b767-953432cca02b-os-release\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.179435 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179381 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-slash\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.179435 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179438 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host-var-lib-cni-networks-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-lib-cni-networks-openshift-sdn\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.179686 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179478 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host-var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-dbus\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.179686 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179517 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9cd9\" (UniqueName: \"kubernetes.io/projected/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-kube-api-access-l9cd9\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.179686 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179523 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-slash" (OuterVolumeSpecName: "host-slash") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "host-slash". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:37.179686 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179536 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-lib-cni-networks-openshift-sdn" (OuterVolumeSpecName: "host-var-lib-cni-networks-openshift-sdn") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "host-var-lib-cni-networks-openshift-sdn". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:37.179686 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179553 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-run-netns\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.179686 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179562 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-dbus" (OuterVolumeSpecName: "host-var-run-dbus") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "host-var-run-dbus". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:37.179686 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179597 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-run-netns" (OuterVolumeSpecName: "host-run-netns") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "host-run-netns". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:37.179686 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179617 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host-var-run\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.179686 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179663 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"sdn-metrics-certs\" (UniqueName: \"kubernetes.io/secret/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-sdn-metrics-certs\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.179686 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179693 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-cni-bin\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179696 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run" (OuterVolumeSpecName: "host-var-run") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "host-var-run". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179723 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host-modules\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-modules\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179732 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-cni-bin" (OuterVolumeSpecName: "host-cni-bin") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "host-cni-bin". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179757 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-env-overrides\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179790 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-config\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179823 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host-var-run-netns\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-netns\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179756 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-modules" (OuterVolumeSpecName: "host-modules") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "host-modules". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179846 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-etc-sysconfig\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179862 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-netns" (OuterVolumeSpecName: "host-var-run-netns") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "host-var-run-netns". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179872 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host-var-run-ovs\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-ovs\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179893 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-etc-sysconfig" (OuterVolumeSpecName: "etc-sysconfig") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "etc-sysconfig". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179903 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host-cni-conf\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-cni-conf\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179923 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-cni-conf" (OuterVolumeSpecName: "host-cni-conf") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "host-cni-conf". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179950 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host-var-run-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-kubernetes\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179988 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host-var-run-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-openshift-sdn\") pod \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\" (UID: \"45cd7283-fcc9-4e61-b2b1-517445b7bd5a\") " Nov 16 14:55:37.180000 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.179960 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-ovs" (OuterVolumeSpecName: "host-var-run-ovs") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "host-var-run-ovs". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180000 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-kubernetes" (OuterVolumeSpecName: "host-var-run-kubernetes") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "host-var-run-kubernetes". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:55:37.180051 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/45cd7283-fcc9-4e61-b2b1-517445b7bd5a/volumes/kubernetes.io~configmap/env-overrides: clearQuota called, but quotas disabled Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:55:37.180070 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/45cd7283-fcc9-4e61-b2b1-517445b7bd5a/volumes/kubernetes.io~configmap/config: clearQuota called, but quotas disabled Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180098 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2b609ff-b0ed-4929-b767-953432cca02b-cni-binary-copy\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180107 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-openshift-sdn" (OuterVolumeSpecName: "host-var-run-openshift-sdn") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "host-var-run-openshift-sdn". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180180 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-9lk7l\" (UniqueName: \"kubernetes.io/projected/f2b609ff-b0ed-4929-b767-953432cca02b-kube-api-access-9lk7l\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180226 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b609ff-b0ed-4929-b767-953432cca02b-system-cni-dir\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180236 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-env-overrides" (OuterVolumeSpecName: "env-overrides") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "env-overrides". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180267 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2b609ff-b0ed-4929-b767-953432cca02b-os-release\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180314 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b609ff-b0ed-4929-b767-953432cca02b-multus-cni-dir\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180344 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2b609ff-b0ed-4929-b767-953432cca02b-cnibin\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180391 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b609ff-b0ed-4929-b767-953432cca02b-system-cni-dir\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180398 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-config" (OuterVolumeSpecName: "config") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180394 1563 reconciler.go:300] "Volume detached for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-run-netns\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180451 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2b609ff-b0ed-4929-b767-953432cca02b-cnibin\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180459 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2b609ff-b0ed-4929-b767-953432cca02b-os-release\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180452 1563 reconciler.go:300] "Volume detached for volume \"host-var-run\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180490 1563 reconciler.go:300] "Volume detached for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-cni-bin\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.180524 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180513 1563 reconciler.go:300] "Volume detached for volume \"host-modules\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-modules\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.181100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180528 1563 reconciler.go:300] "Volume detached for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-env-overrides\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.181100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180528 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b609ff-b0ed-4929-b767-953432cca02b-multus-cni-dir\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.181100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180549 1563 reconciler.go:300] "Volume detached for volume \"host-var-run-netns\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-netns\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.181100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180564 1563 reconciler.go:300] "Volume detached for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-etc-sysconfig\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.181100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180579 1563 reconciler.go:300] "Volume detached for volume \"host-var-run-ovs\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-ovs\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.181100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180593 1563 reconciler.go:300] "Volume detached for volume \"host-cni-conf\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-cni-conf\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.181100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180607 1563 reconciler.go:300] "Volume detached for volume \"host-var-run-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-kubernetes\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.181100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180622 1563 reconciler.go:300] "Volume detached for volume \"host-var-run-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-openshift-sdn\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.181100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180638 1563 reconciler.go:300] "Volume detached for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-slash\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.181100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180653 1563 reconciler.go:300] "Volume detached for volume \"host-var-lib-cni-networks-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-lib-cni-networks-openshift-sdn\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.181100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180667 1563 reconciler.go:300] "Volume detached for volume \"host-var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-host-var-run-dbus\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.181100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.180818 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2b609ff-b0ed-4929-b767-953432cca02b-cni-binary-copy\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.191355 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-45cd7283\x2dfcc9\x2d4e61\x2db2b1\x2d517445b7bd5a-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dl9cd9.mount: Succeeded. Nov 16 14:55:37.192358 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-45cd7283\x2dfcc9\x2d4e61\x2db2b1\x2d517445b7bd5a-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dl9cd9.mount: Consumed 0 CPU time Nov 16 14:55:37.192380 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-45cd7283\x2dfcc9\x2d4e61\x2db2b1\x2d517445b7bd5a-volumes-kubernetes.io\x7esecret-sdn\x2dmetrics\x2dcerts.mount: Succeeded. Nov 16 14:55:37.193061 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-45cd7283\x2dfcc9\x2d4e61\x2db2b1\x2d517445b7bd5a-volumes-kubernetes.io\x7esecret-sdn\x2dmetrics\x2dcerts.mount: Consumed 0 CPU time Nov 16 14:55:37.199489 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.199407 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-sdn-metrics-certs" (OuterVolumeSpecName: "sdn-metrics-certs") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "sdn-metrics-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 14:55:37.199489 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.199405 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-kube-api-access-l9cd9" (OuterVolumeSpecName: "kube-api-access-l9cd9") pod "45cd7283-fcc9-4e61-b2b1-517445b7bd5a" (UID: "45cd7283-fcc9-4e61-b2b1-517445b7bd5a"). InnerVolumeSpecName "kube-api-access-l9cd9". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:55:37.205061 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.204976 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lk7l\" (UniqueName: \"kubernetes.io/projected/f2b609ff-b0ed-4929-b767-953432cca02b-kube-api-access-9lk7l\") pod \"multus-vm4st\" (UID: \"f2b609ff-b0ed-4929-b767-953432cca02b\") " pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.221382 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.221335 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vm4st" Nov 16 14:55:37.222064 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.221990530Z" level=info msg="Running pod sandbox: openshift-multus/multus-vm4st/POD" id=e27e5af3-20ec-45fa-9ab7-34a84b409035 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:55:37.222462 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.222079331Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:55:37.281546 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.281455 1563 reconciler.go:300] "Volume detached for volume \"sdn-metrics-certs\" (UniqueName: \"kubernetes.io/secret/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-sdn-metrics-certs\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.281546 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.281542 1563 reconciler.go:300] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-config\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.281546 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.281563 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-l9cd9\" (UniqueName: \"kubernetes.io/projected/45cd7283-fcc9-4e61-b2b1-517445b7bd5a-kube-api-access-l9cd9\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:37.286666 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.286566071Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=e27e5af3-20ec-45fa-9ab7-34a84b409035 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:55:37.290911 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:55:37.290875 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b609ff_b0ed_4929_b767_953432cca02b.slice/crio-e2b2cb66b616b6c1507eef8968b94261de8913bb52c3988f5f78d4508aa41633.scope WatchSource:0}: Error finding container e2b2cb66b616b6c1507eef8968b94261de8913bb52c3988f5f78d4508aa41633: Status 404 returned error &{%!s(*http.body=&{0xc003628108 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:55:37.306955 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.306890210Z" level=info msg="Ran pod sandbox e2b2cb66b616b6c1507eef8968b94261de8913bb52c3988f5f78d4508aa41633 with infra container: openshift-multus/multus-vm4st/POD" id=e27e5af3-20ec-45fa-9ab7-34a84b409035 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:55:37.308041 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.308000918Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6c99d61b704589581b000dc6dd224449caf9419879e49af19ccfae10831d9cea" id=1c6ada0d-1e95-4099-bc42-86c86cd8e478 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:37.308338 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.308306420Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6c99d61b704589581b000dc6dd224449caf9419879e49af19ccfae10831d9cea not found" id=1c6ada0d-1e95-4099-bc42-86c86cd8e478 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:37.308794 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.308757 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 14:55:37.309595 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.309496928Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6c99d61b704589581b000dc6dd224449caf9419879e49af19ccfae10831d9cea" id=4babd03e-a192-4b39-879f-102841b57269 name=/runtime.v1.ImageService/PullImage Nov 16 14:55:37.313462 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.313427255Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6c99d61b704589581b000dc6dd224449caf9419879e49af19ccfae10831d9cea\"" Nov 16 14:55:37.376480 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.376372 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-multus/network-metrics-daemon-hr6jx] Nov 16 14:55:37.376688 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.376675 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 containerName="network-metrics-daemon" containerID="cri-o://a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc" gracePeriod=30 Nov 16 14:55:37.376851 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.376801 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-multus/network-metrics-daemon-hr6jx" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 containerName="kube-rbac-proxy" containerID="cri-o://4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2" gracePeriod=30 Nov 16 14:55:37.377183 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.377106689Z" level=info msg="Stopping container: 4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2 (timeout: 30s)" id=51f6ae44-a3f4-4b16-87b5-ba89f9a838b6 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:37.377308 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.377106589Z" level=info msg="Stopping container: a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc (timeout: 30s)" id=6308c838-8b5f-493e-be92-08a8592c4c57 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:37.405080 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc.scope: Succeeded. Nov 16 14:55:37.405769 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc.scope: Consumed 9.453s CPU time Nov 16 14:55:37.409967 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc.scope: Succeeded. Nov 16 14:55:37.410629 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc.scope: Consumed 38ms CPU time Nov 16 14:55:37.817994 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.817949 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vm4st" event=&{ID:f2b609ff-b0ed-4929-b767-953432cca02b Type:ContainerStarted Data:e2b2cb66b616b6c1507eef8968b94261de8913bb52c3988f5f78d4508aa41633} Nov 16 14:55:37.819904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.819844 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-sdn/sdn-pgwdx" event=&{ID:45cd7283-fcc9-4e61-b2b1-517445b7bd5a Type:ContainerDied Data:6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75} Nov 16 14:55:37.819904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.819890 1563 scope.go:110] "RemoveContainer" containerID="adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19" Nov 16 14:55:37.820923 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.820878718Z" level=info msg="Removing container: adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19" id=39aad4ed-1f52-4c8f-b53b-d8e76ea8fc39 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:55:37.821172 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.821147 1563 generic.go:296] "Generic (PLEG): container finished" podID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 containerID="a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc" exitCode=0 Nov 16 14:55:37.821172 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.821164 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hr6jx" event=&{ID:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Type:ContainerDied Data:a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc} Nov 16 14:55:37.828816 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod45cd7283_fcc9_4e61_b2b1_517445b7bd5a.slice. Nov 16 14:55:37.828892 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod45cd7283_fcc9_4e61_b2b1_517445b7bd5a.slice: Consumed 2min 14.690s CPU time Nov 16 14:55:37.879577 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.879482718Z" level=info msg="Stopped container a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc: openshift-multus/network-metrics-daemon-hr6jx/network-metrics-daemon" id=6308c838-8b5f-493e-be92-08a8592c4c57 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:37.902631 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.902521576Z" level=info msg="Removed container adabef26b9b8f137b7621d99f20490f5bccfe27c749302a6148ae72b3d4faf19: openshift-sdn/sdn-pgwdx/drop-icmp" id=39aad4ed-1f52-4c8f-b53b-d8e76ea8fc39 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:55:37.902871 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.902836 1563 scope.go:110] "RemoveContainer" containerID="516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8" Nov 16 14:55:37.903917 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.903881985Z" level=info msg="Removing container: 516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8" id=dc7679ac-6eae-43fb-93a4-cf52af9b2330 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:55:37.942869 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.942777651Z" level=info msg="Removed container 516a3a869bad3f3dce9b321ba7c20450c1e4b55c89da8154e483a320dbc6bec8: openshift-sdn/sdn-pgwdx/kube-rbac-proxy" id=dc7679ac-6eae-43fb-93a4-cf52af9b2330 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:55:37.943207 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:37.943169 1563 scope.go:110] "RemoveContainer" containerID="962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a" Nov 16 14:55:37.944401 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.944363561Z" level=info msg="Removing container: 962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a" id=fd149cd6-8232-41fd-a6e6-949e3e7f90db name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:55:37.979630 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:37.979504501Z" level=info msg="Removed container 962dbf1ba068351a20d68aa936eb92b0cb5223489ea2d8b4e96937f948e2578a: openshift-sdn/sdn-pgwdx/sdn" id=fd149cd6-8232-41fd-a6e6-949e3e7f90db name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:55:38.058247 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-79aa848a52f70f7f85983988a2a49e87b139a8ebab4a66a23ce00ba70ef9a204-merged.mount: Succeeded. Nov 16 14:55:38.059076 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-79aa848a52f70f7f85983988a2a49e87b139a8ebab4a66a23ce00ba70ef9a204-merged.mount: Consumed 0 CPU time Nov 16 14:55:38.485157 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2.scope: Succeeded. Nov 16 14:55:38.485796 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2.scope: Consumed 5.013s CPU time Nov 16 14:55:38.489731 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2.scope: Succeeded. Nov 16 14:55:38.490451 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2.scope: Consumed 38ms CPU time Nov 16 14:55:38.517474 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:38.517422973Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6c99d61b704589581b000dc6dd224449caf9419879e49af19ccfae10831d9cea\"" Nov 16 14:55:38.557748 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-c896644646e861f0e3dc5dec9b2baa81b7b42b04439a3cb5c8acd4c8294d6aef-merged.mount: Succeeded. Nov 16 14:55:38.558643 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-c896644646e861f0e3dc5dec9b2baa81b7b42b04439a3cb5c8acd4c8294d6aef-merged.mount: Consumed 0 CPU time Nov 16 14:55:38.597345 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:38.597291 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=d38704ca-571d-40ee-84b3-71c1d45d56f4 path="/var/lib/kubelet/pods/d38704ca-571d-40ee-84b3-71c1d45d56f4/volumes" Nov 16 14:55:38.771973 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.7ynKqg.mount: Succeeded. Nov 16 14:55:38.772005 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.fJLp5d.mount: Succeeded. Nov 16 14:55:38.777762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:38.777704450Z" level=info msg="Stopped container 4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2: openshift-multus/network-metrics-daemon-hr6jx/kube-rbac-proxy" id=51f6ae44-a3f4-4b16-87b5-ba89f9a838b6 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:38.778291 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:38.778249253Z" level=info msg="Stopping pod sandbox: 2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a" id=c90d77db-9237-4b3f-8d69-f89cb560a759 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:55:38.778667 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:38.778548155Z" level=info msg="Got pod network &{Name:network-metrics-daemon-hr6jx Namespace:openshift-multus ID:2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a UID:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 NetNS:/var/run/netns/ebb09e20-5775-49e7-8cfc-a0ae9c03e9d4 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:55:38.778748 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:38.778719957Z" level=info msg="Deleting pod openshift-multus_network-metrics-daemon-hr6jx from CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:55:38.827411 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:38.827374 1563 generic.go:296] "Generic (PLEG): container finished" podID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 containerID="4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2" exitCode=0 Nov 16 14:55:38.827411 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:38.827426 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hr6jx" event=&{ID:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Type:ContainerDied Data:4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2} Nov 16 14:55:39.028740 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.028585 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-sdn/sdn-pgwdx] Nov 16 14:55:39.045969 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.045930 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-sdn/sdn-pgwdx] Nov 16 14:55:39.079621 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.079535 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-sdn/sdn-tqwqj] Nov 16 14:55:39.079621 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.079630 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:55:39.079909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:39.079766 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="45cd7283-fcc9-4e61-b2b1-517445b7bd5a" containerName="drop-icmp" Nov 16 14:55:39.079909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.079779 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="45cd7283-fcc9-4e61-b2b1-517445b7bd5a" containerName="drop-icmp" Nov 16 14:55:39.079909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:39.079795 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="45cd7283-fcc9-4e61-b2b1-517445b7bd5a" containerName="sdn" Nov 16 14:55:39.079909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.079804 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="45cd7283-fcc9-4e61-b2b1-517445b7bd5a" containerName="sdn" Nov 16 14:55:39.079909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:39.079815 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="45cd7283-fcc9-4e61-b2b1-517445b7bd5a" containerName="kube-rbac-proxy" Nov 16 14:55:39.079909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.079823 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="45cd7283-fcc9-4e61-b2b1-517445b7bd5a" containerName="kube-rbac-proxy" Nov 16 14:55:39.080115 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.079964 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="45cd7283-fcc9-4e61-b2b1-517445b7bd5a" containerName="drop-icmp" Nov 16 14:55:39.080115 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.080031 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="45cd7283-fcc9-4e61-b2b1-517445b7bd5a" containerName="sdn" Nov 16 14:55:39.080115 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.080045 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="45cd7283-fcc9-4e61-b2b1-517445b7bd5a" containerName="kube-rbac-proxy" Nov 16 14:55:39.090326 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod4dbb404b_812a_4297_b9ee_2cefe58d83ae.slice. Nov 16 14:55:39.198870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.198738 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxpnn\" (UniqueName: \"kubernetes.io/projected/4dbb404b-812a-4297-b9ee-2cefe58d83ae-kube-api-access-jxpnn\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.198870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.198830 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-run-ovs\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-ovs\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.198870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.198868 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-cni-bin\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.198870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.198897 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-run-netns\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199190 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.198933 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-run-netns\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-netns\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199190 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.198968 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-run-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-kubernetes\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199190 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.199086 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4dbb404b-812a-4297-b9ee-2cefe58d83ae-env-overrides\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199190 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.199148 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sdn-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dbb404b-812a-4297-b9ee-2cefe58d83ae-sdn-metrics-certs\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199190 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.199186 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-run-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-openshift-sdn\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199345 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.199213 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-conf\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-cni-conf\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199345 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.199241 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-etc-sysconfig\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199345 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.199267 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-dbus\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199345 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.199297 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-lib-cni-networks-openshift-sdn\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199345 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.199328 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-run\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199497 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.199436 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-modules\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-modules\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199497 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.199496 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-slash\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199564 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.199536 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-mtu-migration\" (UniqueName: \"kubernetes.io/configmap/4dbb404b-812a-4297-b9ee-2cefe58d83ae-config-mtu-migration\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.199601 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.199572 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4dbb404b-812a-4297-b9ee-2cefe58d83ae-config\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301165 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.300950 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-jxpnn\" (UniqueName: \"kubernetes.io/projected/4dbb404b-812a-4297-b9ee-2cefe58d83ae-kube-api-access-jxpnn\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301165 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301020 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-run-ovs\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-ovs\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301165 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301064 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-cni-bin\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301165 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301096 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-run-netns\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301165 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301160 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-run-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-kubernetes\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301466 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301207 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-run-ovs\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-ovs\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301466 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301232 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-cni-bin\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301466 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301287 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-run-netns\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301466 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301362 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-run-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-kubernetes\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301592 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301546 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-run-netns\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-netns\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301592 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301585 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4dbb404b-812a-4297-b9ee-2cefe58d83ae-env-overrides\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301710 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301643 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"sdn-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dbb404b-812a-4297-b9ee-2cefe58d83ae-sdn-metrics-certs\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301765 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301709 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-run-netns\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-netns\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301765 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301714 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-cni-conf\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-cni-conf\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301830 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301773 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-cni-conf\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-cni-conf\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301830 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301794 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-run-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-openshift-sdn\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301897 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301832 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-etc-sysconfig\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301897 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301862 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-dbus\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301962 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301898 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-lib-cni-networks-openshift-sdn\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301962 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301908 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-etc-sysconfig\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301962 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301915 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-run-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-openshift-sdn\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.301962 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301931 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-var-run\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.302181 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301963 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-run-dbus\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run-dbus\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.302181 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301969 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-run\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-run\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.302181 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301987 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-modules\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-modules\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.302181 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.301998 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-openshift-sdn\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-var-lib-cni-networks-openshift-sdn\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.302181 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.302024 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-slash\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.302181 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.302058 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"config-mtu-migration\" (UniqueName: \"kubernetes.io/configmap/4dbb404b-812a-4297-b9ee-2cefe58d83ae-config-mtu-migration\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.302181 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.302060 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-modules\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-modules\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.302181 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.302083 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dbb404b-812a-4297-b9ee-2cefe58d83ae-host-slash\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.302181 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.302098 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4dbb404b-812a-4297-b9ee-2cefe58d83ae-config\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.302452 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.302185 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4dbb404b-812a-4297-b9ee-2cefe58d83ae-env-overrides\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.302635 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.302606 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"config-mtu-migration\" (UniqueName: \"kubernetes.io/configmap/4dbb404b-812a-4297-b9ee-2cefe58d83ae-config-mtu-migration\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.302705 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.302659 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4dbb404b-812a-4297-b9ee-2cefe58d83ae-config\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.304700 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.304624 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"sdn-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dbb404b-812a-4297-b9ee-2cefe58d83ae-sdn-metrics-certs\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.324494 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.324421 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxpnn\" (UniqueName: \"kubernetes.io/projected/4dbb404b-812a-4297-b9ee-2cefe58d83ae-kube-api-access-jxpnn\") pod \"sdn-tqwqj\" (UID: \"4dbb404b-812a-4297-b9ee-2cefe58d83ae\") " pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.409508 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.409404 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-sdn/sdn-tqwqj" Nov 16 14:55:39.410248 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:39.410182767Z" level=info msg="Running pod sandbox: openshift-sdn/sdn-tqwqj/POD" id=296ee49a-25c0-4547-b1c3-13e399b42eb8 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:55:39.410341 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:39.410267368Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:55:39.465908 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:39.465839547Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=296ee49a-25c0-4547-b1c3-13e399b42eb8 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:55:39.469842 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:55:39.469797 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dbb404b_812a_4297_b9ee_2cefe58d83ae.slice/crio-605128267a341e642f57d415cf84d2506656701e809788b615a15d374bf4cc9d.scope WatchSource:0}: Error finding container 605128267a341e642f57d415cf84d2506656701e809788b615a15d374bf4cc9d: Status 404 returned error &{%!s(*http.body=&{0xc002fe1110 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:55:39.486553 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:39.486482088Z" level=info msg="Ran pod sandbox 605128267a341e642f57d415cf84d2506656701e809788b615a15d374bf4cc9d with infra container: openshift-sdn/sdn-tqwqj/POD" id=296ee49a-25c0-4547-b1c3-13e399b42eb8 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:55:39.487774 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:39.487687996Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fdff80528a499774554a70304d4fb942f08a3b875af6e3b5c4b3e930b3c2aaa3" id=edf1951b-9036-4734-87d3-d972a314fb1b name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:39.487989 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:39.487959298Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fdff80528a499774554a70304d4fb942f08a3b875af6e3b5c4b3e930b3c2aaa3 not found" id=edf1951b-9036-4734-87d3-d972a314fb1b name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:39.488809 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:39.488754503Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fdff80528a499774554a70304d4fb942f08a3b875af6e3b5c4b3e930b3c2aaa3" id=ed1fdb5f-650a-4220-bb56-c34d0647bc2a name=/runtime.v1.ImageService/PullImage Nov 16 14:55:39.491594 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:39.491527022Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fdff80528a499774554a70304d4fb942f08a3b875af6e3b5c4b3e930b3c2aaa3\"" Nov 16 14:55:39.833147 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:39.833056 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-sdn/sdn-tqwqj" event=&{ID:4dbb404b-812a-4297-b9ee-2cefe58d83ae Type:ContainerStarted Data:605128267a341e642f57d415cf84d2506656701e809788b615a15d374bf4cc9d} Nov 16 14:55:40.596004 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:40.595928 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=45cd7283-fcc9-4e61-b2b1-517445b7bd5a path="/var/lib/kubelet/pods/45cd7283-fcc9-4e61-b2b1-517445b7bd5a/volumes" Nov 16 14:55:40.816472 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:40.816377166Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fdff80528a499774554a70304d4fb942f08a3b875af6e3b5c4b3e930b3c2aaa3\"" Nov 16 14:55:43.210440 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.210374365Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6c99d61b704589581b000dc6dd224449caf9419879e49af19ccfae10831d9cea" id=4babd03e-a192-4b39-879f-102841b57269 name=/runtime.v1.ImageService/PullImage Nov 16 14:55:43.211600 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.211555989Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6c99d61b704589581b000dc6dd224449caf9419879e49af19ccfae10831d9cea" id=785bf116-87de-4a62-b2c8-8bc083a6f046 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:43.213281 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.213228423Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5470cde07684980e3b22be2bc068067b5004d13080cf46547f7dc7f3ecab09b0,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6c99d61b704589581b000dc6dd224449caf9419879e49af19ccfae10831d9cea],Size_:471431625,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=785bf116-87de-4a62-b2c8-8bc083a6f046 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:43.214287 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.214226543Z" level=info msg="Creating container: openshift-multus/multus-vm4st/kube-multus" id=33c5b00d-0bd4-4a8b-9a25-be26a4f4aed1 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:55:43.214385 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.214337345Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:55:43.277940 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-8a3237d70a1973d749e3e907c6e70904fcab6f1ffa0550f07b21e49a8eddb6a6.scope. Nov 16 14:55:43.309048 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 8a3237d70a1973d749e3e907c6e70904fcab6f1ffa0550f07b21e49a8eddb6a6. Nov 16 14:55:43.389651 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.389584986Z" level=info msg="Created container 8a3237d70a1973d749e3e907c6e70904fcab6f1ffa0550f07b21e49a8eddb6a6: openshift-multus/multus-vm4st/kube-multus" id=33c5b00d-0bd4-4a8b-9a25-be26a4f4aed1 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:55:43.390311 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.390265300Z" level=info msg="Starting container: 8a3237d70a1973d749e3e907c6e70904fcab6f1ffa0550f07b21e49a8eddb6a6" id=6f1532b6-c469-41b7-8e72-d80cf2b55a98 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:55:43.403970 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.403917476Z" level=info msg="Started container" PID=188400 containerID=8a3237d70a1973d749e3e907c6e70904fcab6f1ffa0550f07b21e49a8eddb6a6 description=openshift-multus/multus-vm4st/kube-multus id=6f1532b6-c469-41b7-8e72-d80cf2b55a98 name=/runtime.v1.RuntimeService/StartContainer sandboxID=e2b2cb66b616b6c1507eef8968b94261de8913bb52c3988f5f78d4508aa41633 Nov 16 14:55:43.412942 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.412873657Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/upgrade_c833f4aa-56ce-4406-a7ab-c769f5cf2f97\": CREATE" Nov 16 14:55:43.426064 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.426004422Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:55:43.426064 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.426046523Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:55:43.454009 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.453906686Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/multus\": CREATE" Nov 16 14:55:43.467201 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.467066952Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:55:43.467201 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.467105753Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:55:43.467201 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:43.467137453Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/upgrade_c833f4aa-56ce-4406-a7ab-c769f5cf2f97\": REMOVE" Nov 16 14:55:43.846281 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:43.846229 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vm4st" event=&{ID:f2b609ff-b0ed-4929-b767-953432cca02b Type:ContainerStarted Data:8a3237d70a1973d749e3e907c6e70904fcab6f1ffa0550f07b21e49a8eddb6a6} Nov 16 14:55:47.375378 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:47.375281 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-dns/node-resolver-l5trv] Nov 16 14:55:47.375908 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:47.375552 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-dns/node-resolver-l5trv" podUID=40251af6-202f-415a-aa28-c362b501dbd3 containerName="dns-node-resolver" containerID="cri-o://91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228" gracePeriod=30 Nov 16 14:55:47.376024 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:47.375971045Z" level=info msg="Stopping container: 91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228 (timeout: 30s)" id=4cf4c2ca-fdee-4c6d-86a8-9c1c1fe4a87d name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:47.406743 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228.scope: Succeeded. Nov 16 14:55:47.407395 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228.scope: Consumed 35ms CPU time Nov 16 14:55:47.410677 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228.scope: Succeeded. Nov 16 14:55:47.411278 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228.scope: Consumed 3.531s CPU time Nov 16 14:55:47.860047 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:47.860004 1563 generic.go:296] "Generic (PLEG): container finished" podID=40251af6-202f-415a-aa28-c362b501dbd3 containerID="91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228" exitCode=0 Nov 16 14:55:47.860240 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:47.860076 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l5trv" event=&{ID:40251af6-202f-415a-aa28-c362b501dbd3 Type:ContainerDied Data:91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228} Nov 16 14:55:48.158752 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-c06085de182ce15237aef41c6986b45b849442dc4c21d21ead8b7143a9526c2d-merged.mount: Succeeded. Nov 16 14:55:48.159828 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-c06085de182ce15237aef41c6986b45b849442dc4c21d21ead8b7143a9526c2d-merged.mount: Consumed 0 CPU time Nov 16 14:55:56.706415 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.2Ja6Ps.mount: Succeeded. Nov 16 14:55:56.706451 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.h3wMVj.mount: Succeeded. Nov 16 14:55:57.157035 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:57.156912250Z" level=info msg="Stopped container 91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228: openshift-dns/node-resolver-l5trv/dns-node-resolver" id=4cf4c2ca-fdee-4c6d-86a8-9c1c1fe4a87d name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:55:57.157726 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:57.157684857Z" level=info msg="Stopping pod sandbox: 57035fc21822c56b1677df50b84a40801ec6643b61f910c3ed96ed7ba49c71b8" id=0e8247ce-07ad-44dd-8879-37f58674440c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:55:57.167955 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-c9c6cdd9\x2d086e\x2d4610\x2da2bc\x2d75b459ed90f3.mount: Succeeded. Nov 16 14:55:57.168760 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-c9c6cdd9\x2d086e\x2d4610\x2da2bc\x2d75b459ed90f3.mount: Consumed 0 CPU time Nov 16 14:55:57.186257 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-c9c6cdd9\x2d086e\x2d4610\x2da2bc\x2d75b459ed90f3.mount: Succeeded. Nov 16 14:55:57.187104 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-c9c6cdd9\x2d086e\x2d4610\x2da2bc\x2d75b459ed90f3.mount: Consumed 0 CPU time Nov 16 14:55:57.210737 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-c9c6cdd9\x2d086e\x2d4610\x2da2bc\x2d75b459ed90f3.mount: Succeeded. Nov 16 14:55:57.211566 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-c9c6cdd9\x2d086e\x2d4610\x2da2bc\x2d75b459ed90f3.mount: Consumed 0 CPU time Nov 16 14:55:57.704863 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-99642f28aa032271a3f354491bfa8d4a3e178c94502e09d795b291bc2e7863e6-merged.mount: Succeeded. Nov 16 14:55:57.706091 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-99642f28aa032271a3f354491bfa8d4a3e178c94502e09d795b291bc2e7863e6-merged.mount: Consumed 0 CPU time Nov 16 14:55:57.706163 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-57035fc21822c56b1677df50b84a40801ec6643b61f910c3ed96ed7ba49c71b8-userdata-shm.mount: Succeeded. Nov 16 14:55:57.707274 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-57035fc21822c56b1677df50b84a40801ec6643b61f910c3ed96ed7ba49c71b8-userdata-shm.mount: Consumed 0 CPU time Nov 16 14:55:58.369036 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.368981386Z" level=info msg="Stopped pod sandbox: 57035fc21822c56b1677df50b84a40801ec6643b61f910c3ed96ed7ba49c71b8" id=0e8247ce-07ad-44dd-8879-37f58674440c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:55:58.490538 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.490414 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwbs4\" (UniqueName: \"kubernetes.io/projected/40251af6-202f-415a-aa28-c362b501dbd3-kube-api-access-xwbs4\") pod \"40251af6-202f-415a-aa28-c362b501dbd3\" (UID: \"40251af6-202f-415a-aa28-c362b501dbd3\") " Nov 16 14:55:58.490538 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.490525 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/40251af6-202f-415a-aa28-c362b501dbd3-hosts-file\") pod \"40251af6-202f-415a-aa28-c362b501dbd3\" (UID: \"40251af6-202f-415a-aa28-c362b501dbd3\") " Nov 16 14:55:58.491114 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.490840 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/40251af6-202f-415a-aa28-c362b501dbd3-hosts-file" (OuterVolumeSpecName: "hosts-file") pod "40251af6-202f-415a-aa28-c362b501dbd3" (UID: "40251af6-202f-415a-aa28-c362b501dbd3"). InnerVolumeSpecName "hosts-file". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:55:58.502612 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-40251af6\x2d202f\x2d415a\x2daa28\x2dc362b501dbd3-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dxwbs4.mount: Succeeded. Nov 16 14:55:58.503502 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-40251af6\x2d202f\x2d415a\x2daa28\x2dc362b501dbd3-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dxwbs4.mount: Consumed 0 CPU time Nov 16 14:55:58.507491 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.507386 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40251af6-202f-415a-aa28-c362b501dbd3-kube-api-access-xwbs4" (OuterVolumeSpecName: "kube-api-access-xwbs4") pod "40251af6-202f-415a-aa28-c362b501dbd3" (UID: "40251af6-202f-415a-aa28-c362b501dbd3"). InnerVolumeSpecName "kube-api-access-xwbs4". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:55:58.523976 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.523929133Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fdff80528a499774554a70304d4fb942f08a3b875af6e3b5c4b3e930b3c2aaa3" id=ed1fdb5f-650a-4220-bb56-c34d0647bc2a name=/runtime.v1.ImageService/PullImage Nov 16 14:55:58.525006 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.524941442Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fdff80528a499774554a70304d4fb942f08a3b875af6e3b5c4b3e930b3c2aaa3" id=eb3e5eff-760f-4ed4-8922-69354665da1c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:58.526774 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.526719957Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:e4232caea9a6167f81a011601e649a7f1b41f851785081f560a8facb37b1e1dd,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fdff80528a499774554a70304d4fb942f08a3b875af6e3b5c4b3e930b3c2aaa3],Size_:610803746,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=eb3e5eff-760f-4ed4-8922-69354665da1c name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:58.530716 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.530677891Z" level=info msg="Creating container: openshift-sdn/sdn-tqwqj/sdn" id=3adf32d8-8a1d-48aa-90c2-310cd7c8e711 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:55:58.530818 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.530795792Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:55:58.590534 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b.scope. Nov 16 14:55:58.591293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.591257 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-xwbs4\" (UniqueName: \"kubernetes.io/projected/40251af6-202f-415a-aa28-c362b501dbd3-kube-api-access-xwbs4\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:58.591403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.591303 1563 reconciler.go:300] "Volume detached for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/40251af6-202f-415a-aa28-c362b501dbd3-hosts-file\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:55:58.605298 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod40251af6_202f_415a_aa28_c362b501dbd3.slice. Nov 16 14:55:58.605358 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod40251af6_202f_415a_aa28_c362b501dbd3.slice: Consumed 3.567s CPU time Nov 16 14:55:58.632008 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b. Nov 16 14:55:58.702431 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.702371285Z" level=info msg="Created container d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b: openshift-sdn/sdn-tqwqj/sdn" id=3adf32d8-8a1d-48aa-90c2-310cd7c8e711 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:55:58.703169 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.703114286Z" level=info msg="Starting container: d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b" id=e158a2a1-af9e-4e2f-b165-62318e74436a name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:55:58.718066 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.qte7PV.mount: Succeeded. Nov 16 14:55:58.725799 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.725738930Z" level=info msg="Started container" PID=188757 containerID=d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b description=openshift-sdn/sdn-tqwqj/sdn id=e158a2a1-af9e-4e2f-b165-62318e74436a name=/runtime.v1.RuntimeService/StartContainer sandboxID=605128267a341e642f57d415cf84d2506656701e809788b615a15d374bf4cc9d Nov 16 14:55:58.735296 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.735225949Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": CREATE" Nov 16 14:55:58.749068 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.749015275Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:55:58.749068 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.749054476Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:55:58.749068 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.749073076Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 14:55:58.749667 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.749629277Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=aa14244d-1a84-409b-803f-57da87cfc34a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:58.750196 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.750158378Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=aa14244d-1a84-409b-803f-57da87cfc34a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:58.756842 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.756803491Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=b69728e6-b136-4cc1-8c3a-201b7c8a7a44 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:58.757021 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.756988991Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=b69728e6-b136-4cc1-8c3a-201b7c8a7a44 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:58.758110 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.758076193Z" level=info msg="Creating container: openshift-sdn/sdn-tqwqj/kube-rbac-proxy" id=374205ad-95b9-426b-b86a-bd0766ccc35d name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:55:58.758236 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.758191493Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:55:58.761056 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.761017399Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:55:58.761056 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.761049399Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:55:58.761207 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.761065699Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/openshift-sdn\": WRITE" Nov 16 14:55:58.775222 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.775119226Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:55:58.775222 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.775212326Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:55:58.816244 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-99f513b500a2b2f9ce65000cd446dff9de6b483677b052591965727418b780c7.scope. Nov 16 14:55:58.847575 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 99f513b500a2b2f9ce65000cd446dff9de6b483677b052591965727418b780c7. Nov 16 14:55:58.881453 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:55:58.897673 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.897616 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-sdn/sdn-tqwqj" event=&{ID:4dbb404b-812a-4297-b9ee-2cefe58d83ae Type:ContainerStarted Data:d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b} Nov 16 14:55:58.898947 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.898918 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l5trv" event=&{ID:40251af6-202f-415a-aa28-c362b501dbd3 Type:ContainerDied Data:57035fc21822c56b1677df50b84a40801ec6643b61f910c3ed96ed7ba49c71b8} Nov 16 14:55:58.899048 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.898967 1563 scope.go:110] "RemoveContainer" containerID="91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228" Nov 16 14:55:58.899955 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.899912968Z" level=info msg="Removing container: 91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228" id=7a6ca29d-3cb5-4d71-b276-2d00c33388a5 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:55:58.932350 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.932290 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-dns/node-resolver-l5trv] Nov 16 14:55:58.932514 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.932361831Z" level=info msg="Removed container 91ab9cdaf5fe63157540f95139f4cf03c99b0e86f30767efa4b74effcf13e228: openshift-dns/node-resolver-l5trv/dns-node-resolver" id=7a6ca29d-3cb5-4d71-b276-2d00c33388a5 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:55:58.942222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.941762 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-dns/node-resolver-l5trv] Nov 16 14:55:58.954428 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.954349173Z" level=info msg="Created container 99f513b500a2b2f9ce65000cd446dff9de6b483677b052591965727418b780c7: openshift-sdn/sdn-tqwqj/kube-rbac-proxy" id=374205ad-95b9-426b-b86a-bd0766ccc35d name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:55:58.954856 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.954805774Z" level=info msg="Starting container: 99f513b500a2b2f9ce65000cd446dff9de6b483677b052591965727418b780c7" id=f03fcb27-e796-4490-be10-5f58521db290 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:55:58.966682 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.966637297Z" level=info msg="Started container" PID=188895 containerID=99f513b500a2b2f9ce65000cd446dff9de6b483677b052591965727418b780c7 description=openshift-sdn/sdn-tqwqj/kube-rbac-proxy id=f03fcb27-e796-4490-be10-5f58521db290 name=/runtime.v1.RuntimeService/StartContainer sandboxID=605128267a341e642f57d415cf84d2506656701e809788b615a15d374bf4cc9d Nov 16 14:55:58.967909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.967401 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-dns/node-resolver-27gwx] Nov 16 14:55:58.967909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.967471 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:55:58.967909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:55:58.967574 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="40251af6-202f-415a-aa28-c362b501dbd3" containerName="dns-node-resolver" Nov 16 14:55:58.967909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.967588 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="40251af6-202f-415a-aa28-c362b501dbd3" containerName="dns-node-resolver" Nov 16 14:55:58.967909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:58.967681 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="40251af6-202f-415a-aa28-c362b501dbd3" containerName="dns-node-resolver" Nov 16 14:55:58.980047 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod8c7ca98a_3d4d_4a6f_b3ad_34b00a3427a2.slice. Nov 16 14:55:58.993221 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.993175349Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fdff80528a499774554a70304d4fb942f08a3b875af6e3b5c4b3e930b3c2aaa3" id=8982bbb9-b159-45b4-bb1a-62d40a572d25 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:58.996528 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.996488255Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:e4232caea9a6167f81a011601e649a7f1b41f851785081f560a8facb37b1e1dd,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fdff80528a499774554a70304d4fb942f08a3b875af6e3b5c4b3e930b3c2aaa3],Size_:610803746,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=8982bbb9-b159-45b4-bb1a-62d40a572d25 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:58.997328 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.997289657Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fdff80528a499774554a70304d4fb942f08a3b875af6e3b5c4b3e930b3c2aaa3" id=fad4b18a-a409-42b3-b6f3-2baee69ed2e2 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:58.999181 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:58.999073760Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:e4232caea9a6167f81a011601e649a7f1b41f851785081f560a8facb37b1e1dd,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fdff80528a499774554a70304d4fb942f08a3b875af6e3b5c4b3e930b3c2aaa3],Size_:610803746,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=fad4b18a-a409-42b3-b6f3-2baee69ed2e2 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:59.000204 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:59.000169162Z" level=info msg="Creating container: openshift-sdn/sdn-tqwqj/drop-icmp" id=3a169195-93a7-453f-93a0-0b7d978dc538 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:55:59.000293 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:59.000277763Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:55:59.065583 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-41a41203dea9a0dcaf1e376035c362166126858943d10a730063134730c5cd07.scope. Nov 16 14:55:59.085994 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 41a41203dea9a0dcaf1e376035c362166126858943d10a730063134730c5cd07. Nov 16 14:55:59.095506 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:59.095432 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2-hosts-file\") pod \"node-resolver-27gwx\" (UID: \"8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2\") " pod="openshift-dns/node-resolver-27gwx" Nov 16 14:55:59.095506 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:59.095505 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-958xx\" (UniqueName: \"kubernetes.io/projected/8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2-kube-api-access-958xx\") pod \"node-resolver-27gwx\" (UID: \"8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2\") " pod="openshift-dns/node-resolver-27gwx" Nov 16 14:55:59.126096 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00221|connmgr|INFO|br0<->unix#490: 4 flow_mods in the last 0 s (4 adds) Nov 16 14:55:59.160010 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:59.159880872Z" level=info msg="Created container 41a41203dea9a0dcaf1e376035c362166126858943d10a730063134730c5cd07: openshift-sdn/sdn-tqwqj/drop-icmp" id=3a169195-93a7-453f-93a0-0b7d978dc538 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:55:59.160551 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:59.160515273Z" level=info msg="Starting container: 41a41203dea9a0dcaf1e376035c362166126858943d10a730063134730c5cd07" id=50f95ce4-6099-4d7a-bf45-9526f5f6accf name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:55:59.174193 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:59.174123800Z" level=info msg="Started container" PID=188988 containerID=41a41203dea9a0dcaf1e376035c362166126858943d10a730063134730c5cd07 description=openshift-sdn/sdn-tqwqj/drop-icmp id=50f95ce4-6099-4d7a-bf45-9526f5f6accf name=/runtime.v1.RuntimeService/StartContainer sandboxID=605128267a341e642f57d415cf84d2506656701e809788b615a15d374bf4cc9d Nov 16 14:55:59.196413 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00222|connmgr|INFO|br0<->unix#494: 1 flow_mods in the last 0 s (1 adds) Nov 16 14:55:59.196680 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:59.196645 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-958xx\" (UniqueName: \"kubernetes.io/projected/8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2-kube-api-access-958xx\") pod \"node-resolver-27gwx\" (UID: \"8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2\") " pod="openshift-dns/node-resolver-27gwx" Nov 16 14:55:59.196804 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:59.196743 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2-hosts-file\") pod \"node-resolver-27gwx\" (UID: \"8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2\") " pod="openshift-dns/node-resolver-27gwx" Nov 16 14:55:59.196855 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:59.196845 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2-hosts-file\") pod \"node-resolver-27gwx\" (UID: \"8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2\") " pod="openshift-dns/node-resolver-27gwx" Nov 16 14:55:59.221308 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:59.221273 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-958xx\" (UniqueName: \"kubernetes.io/projected/8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2-kube-api-access-958xx\") pod \"node-resolver-27gwx\" (UID: \"8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2\") " pod="openshift-dns/node-resolver-27gwx" Nov 16 14:55:59.294557 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:59.294454 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-27gwx" Nov 16 14:55:59.295293 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:59.295097234Z" level=info msg="Running pod sandbox: openshift-dns/node-resolver-27gwx/POD" id=ddf9a191-011c-4a1b-be3b-eaf7e69794c4 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:55:59.295390 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:59.295313334Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:55:59.337184 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00223|connmgr|INFO|br0<->unix#499: 3 flow_mods in the last 0 s (3 adds) Nov 16 14:55:59.366007 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00224|connmgr|INFO|br0<->unix#502: 1 flow_mods in the last 0 s (1 adds) Nov 16 14:55:59.368860 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:59.368798877Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=ddf9a191-011c-4a1b-be3b-eaf7e69794c4 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:55:59.372922 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:55:59.372835 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c7ca98a_3d4d_4a6f_b3ad_34b00a3427a2.slice/crio-e87b2f605404e539c9ac5412ace52d2646b00639ba116cad735f33c9dd795753.scope WatchSource:0}: Error finding container e87b2f605404e539c9ac5412ace52d2646b00639ba116cad735f33c9dd795753: Status 404 returned error &{%!s(*http.body=&{0xc001fbb7d0 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:55:59.380656 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:59.380617700Z" level=info msg="Ran pod sandbox e87b2f605404e539c9ac5412ace52d2646b00639ba116cad735f33c9dd795753 with infra container: openshift-dns/node-resolver-27gwx/POD" id=ddf9a191-011c-4a1b-be3b-eaf7e69794c4 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:55:59.381927 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:59.381890202Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4aec10de69951bafb2f44821a49e72e192c36dc803224766ad8a44f23f22ec9d" id=6194734f-136f-4439-9747-eca05be73b7d name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:59.383973 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:59.382194403Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4aec10de69951bafb2f44821a49e72e192c36dc803224766ad8a44f23f22ec9d not found" id=6194734f-136f-4439-9747-eca05be73b7d name=/runtime.v1.ImageService/ImageStatus Nov 16 14:55:59.383973 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:59.383689206Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4aec10de69951bafb2f44821a49e72e192c36dc803224766ad8a44f23f22ec9d" id=68593a00-00de-4e68-981e-f6b530d10c30 name=/runtime.v1.ImageService/PullImage Nov 16 14:55:59.386534 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:55:59.386501611Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4aec10de69951bafb2f44821a49e72e192c36dc803224766ad8a44f23f22ec9d\"" Nov 16 14:55:59.393064 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00225|connmgr|INFO|br0<->unix#505: 3 flow_mods in the last 0 s (3 adds) Nov 16 14:55:59.423320 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00226|connmgr|INFO|br0<->unix#508: 1 flow_mods in the last 0 s (1 adds) Nov 16 14:55:59.449298 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00227|connmgr|INFO|br0<->unix#511: 3 flow_mods in the last 0 s (3 adds) Nov 16 14:55:59.473562 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00228|connmgr|INFO|br0<->unix#514: 1 flow_mods in the last 0 s (1 adds) Nov 16 14:55:59.497965 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00229|connmgr|INFO|br0<->unix#517: 3 flow_mods in the last 0 s (3 adds) Nov 16 14:55:59.523035 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00230|connmgr|INFO|br0<->unix#520: 1 flow_mods in the last 0 s (1 adds) Nov 16 14:55:59.548675 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00231|connmgr|INFO|br0<->unix#523: 3 flow_mods in the last 0 s (3 adds) Nov 16 14:55:59.573339 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00232|connmgr|INFO|br0<->unix#526: 1 flow_mods in the last 0 s (1 adds) Nov 16 14:55:59.856027 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00233|connmgr|INFO|br0<->unix#529: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:55:59.889863 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00234|connmgr|INFO|br0<->unix#532: 4 flow_mods in the last 0 s (4 deletes) Nov 16 14:55:59.904755 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:59.904689 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-sdn/sdn-tqwqj" event=&{ID:4dbb404b-812a-4297-b9ee-2cefe58d83ae Type:ContainerStarted Data:41a41203dea9a0dcaf1e376035c362166126858943d10a730063134730c5cd07} Nov 16 14:55:59.904755 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:59.904734 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-sdn/sdn-tqwqj" event=&{ID:4dbb404b-812a-4297-b9ee-2cefe58d83ae Type:ContainerStarted Data:99f513b500a2b2f9ce65000cd446dff9de6b483677b052591965727418b780c7} Nov 16 14:55:59.905555 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:55:59.905534 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-27gwx" event=&{ID:8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2 Type:ContainerStarted Data:e87b2f605404e539c9ac5412ace52d2646b00639ba116cad735f33c9dd795753} Nov 16 14:55:59.920400 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00235|bridge|INFO|bridge br0: deleted interface vethe8e66c39 on port 6 Nov 16 14:55:59.926248 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethe8e66c39 left promiscuous mode Nov 16 14:55:59.941656 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:55:59Z [verbose] Del: openshift-multus:network-metrics-daemon-hr6jx:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 14:55:59.951194 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-ebb09e20\x2d5775\x2d49e7\x2d8cfc\x2da0ae9c03e9d4.mount: Succeeded. Nov 16 14:55:59.952076 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-ebb09e20\x2d5775\x2d49e7\x2d8cfc\x2da0ae9c03e9d4.mount: Consumed 0 CPU time Nov 16 14:55:59.972519 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-ebb09e20\x2d5775\x2d49e7\x2d8cfc\x2da0ae9c03e9d4.mount: Succeeded. Nov 16 14:55:59.973375 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-ebb09e20\x2d5775\x2d49e7\x2d8cfc\x2da0ae9c03e9d4.mount: Consumed 0 CPU time Nov 16 14:55:59.999015 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-ebb09e20\x2d5775\x2d49e7\x2d8cfc\x2da0ae9c03e9d4.mount: Succeeded. Nov 16 14:55:59.999914 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-ebb09e20\x2d5775\x2d49e7\x2d8cfc\x2da0ae9c03e9d4.mount: Consumed 0 CPU time Nov 16 14:56:00.037984 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a-userdata-shm.mount: Succeeded. Nov 16 14:56:00.038910 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a-userdata-shm.mount: Consumed 0 CPU time Nov 16 14:56:00.595014 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:00.594972 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=40251af6-202f-415a-aa28-c362b501dbd3 path="/var/lib/kubelet/pods/40251af6-202f-415a-aa28-c362b501dbd3/volumes" Nov 16 14:56:00.644669 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.644618450Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4aec10de69951bafb2f44821a49e72e192c36dc803224766ad8a44f23f22ec9d\"" Nov 16 14:56:00.659540 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.659494879Z" level=info msg="Stopped pod sandbox: 2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a" id=c90d77db-9237-4b3f-8d69-f89cb560a759 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:56:00.706785 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-d641cef8e2832a059829c5346f1aa32449041f32d1099fa54f55db4dc7960f55-merged.mount: Succeeded. Nov 16 14:56:00.707674 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-d641cef8e2832a059829c5346f1aa32449041f32d1099fa54f55db4dc7960f55-merged.mount: Consumed 0 CPU time Nov 16 14:56:00.792982 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.792910538Z" level=info msg="CNI monitoring event \"/etc/kubernetes/cni/net.d/00-multus.conf\": REMOVE" Nov 16 14:56:00.807017 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.806960665Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:56:00.807017 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.807009565Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:56:00.807017 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.807028865Z" level=info msg="CNI monitoring event \"/etc/kubernetes/cni/net.d/00-multus.conf\": CREATE" Nov 16 14:56:00.811454 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:00.811419 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs\") pod \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " Nov 16 14:56:00.811590 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:00.811525 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9nsk\" (UniqueName: \"kubernetes.io/projected/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-kube-api-access-r9nsk\") pod \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\" (UID: \"6754c3dd-53dd-4e4f-8d5f-ce077669c3d4\") " Nov 16 14:56:00.819935 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.819890490Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:56:00.819935 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.819924690Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:56:00.820089 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.819950390Z" level=info msg="CNI monitoring event \"/etc/kubernetes/cni/net.d/00-multus.conf\": WRITE" Nov 16 14:56:00.822929 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-6754c3dd\x2d53dd\x2d4e4f\x2d8d5f\x2dce077669c3d4-volumes-kubernetes.io\x7esecret-metrics\x2dcerts.mount: Succeeded. Nov 16 14:56:00.823900 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-6754c3dd\x2d53dd\x2d4e4f\x2d8d5f\x2dce077669c3d4-volumes-kubernetes.io\x7esecret-metrics\x2dcerts.mount: Consumed 0 CPU time Nov 16 14:56:00.823931 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-6754c3dd\x2d53dd\x2d4e4f\x2d8d5f\x2dce077669c3d4-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dr9nsk.mount: Succeeded. Nov 16 14:56:00.824630 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-6754c3dd\x2d53dd\x2d4e4f\x2d8d5f\x2dce077669c3d4-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dr9nsk.mount: Consumed 0 CPU time Nov 16 14:56:00.828439 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:00.828395 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-kube-api-access-r9nsk" (OuterVolumeSpecName: "kube-api-access-r9nsk") pod "6754c3dd-53dd-4e4f-8d5f-ce077669c3d4" (UID: "6754c3dd-53dd-4e4f-8d5f-ce077669c3d4"). InnerVolumeSpecName "kube-api-access-r9nsk". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:56:00.831320 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.831265612Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:56:00.831320 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.831297212Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:56:00.831320 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.831314912Z" level=info msg="CNI monitoring event \"/etc/kubernetes/cni/net.d/00-multus.conf\": CHMOD" Nov 16 14:56:00.831476 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:00.831329 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6754c3dd-53dd-4e4f-8d5f-ce077669c3d4" (UID: "6754c3dd-53dd-4e4f-8d5f-ce077669c3d4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 14:56:00.909989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:00.909870 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hr6jx" event=&{ID:6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 Type:ContainerDied Data:2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a} Nov 16 14:56:00.909989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:00.909921 1563 scope.go:110] "RemoveContainer" containerID="4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2" Nov 16 14:56:00.910490 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:00.910363 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-sdn/sdn-tqwqj" Nov 16 14:56:00.912242 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:00.912214 1563 reconciler.go:300] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-metrics-certs\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:56:00.912356 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:00.912252 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-r9nsk\" (UniqueName: \"kubernetes.io/projected/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4-kube-api-access-r9nsk\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:56:00.915990 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.915929576Z" level=info msg="Removing container: 4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2" id=dd679e98-503d-422b-aeac-2c5de358f917 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:56:00.918385 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod6754c3dd_53dd_4e4f_8d5f_ce077669c3d4.slice. Nov 16 14:56:00.918448 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod6754c3dd_53dd_4e4f_8d5f_ce077669c3d4.slice: Consumed 14.543s CPU time Nov 16 14:56:00.953382 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.953335349Z" level=info msg="Removed container 4e9a96a2c9ff699aaeb149e4e25cca462a95250ec19455581109dd475e8122a2: openshift-multus/network-metrics-daemon-hr6jx/kube-rbac-proxy" id=dd679e98-503d-422b-aeac-2c5de358f917 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:56:00.953681 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:00.953651 1563 scope.go:110] "RemoveContainer" containerID="a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc" Nov 16 14:56:00.954762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.954722151Z" level=info msg="Removing container: a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc" id=45e035c3-4d92-4385-aa51-4b993e7991a7 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:56:00.989499 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:00.989446719Z" level=info msg="Removed container a19d19ce8fd6bac747d4713c593438ec9df93a01d7359c0218e6961bd003d2dc: openshift-multus/network-metrics-daemon-hr6jx/network-metrics-daemon" id=45e035c3-4d92-4385-aa51-4b993e7991a7 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:56:01.765531 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:01.765486 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-multus/network-metrics-daemon-hr6jx] Nov 16 14:56:01.806685 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:01.806628 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-multus/network-metrics-daemon-hr6jx] Nov 16 14:56:02.016065 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.015904 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-multus/network-metrics-daemon-q8qt5] Nov 16 14:56:02.016065 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.015981 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:56:02.016652 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:02.016087 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="6754c3dd-53dd-4e4f-8d5f-ce077669c3d4" containerName="network-metrics-daemon" Nov 16 14:56:02.016652 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.016102 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6754c3dd-53dd-4e4f-8d5f-ce077669c3d4" containerName="network-metrics-daemon" Nov 16 14:56:02.016652 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:02.016113 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="6754c3dd-53dd-4e4f-8d5f-ce077669c3d4" containerName="kube-rbac-proxy" Nov 16 14:56:02.016652 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.016122 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6754c3dd-53dd-4e4f-8d5f-ce077669c3d4" containerName="kube-rbac-proxy" Nov 16 14:56:02.016652 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.016289 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="6754c3dd-53dd-4e4f-8d5f-ce077669c3d4" containerName="kube-rbac-proxy" Nov 16 14:56:02.016652 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.016304 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="6754c3dd-53dd-4e4f-8d5f-ce077669c3d4" containerName="network-metrics-daemon" Nov 16 14:56:02.027322 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-podbc424839_f613_4159_bb7f_0c1769bf6200.slice. Nov 16 14:56:02.122833 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.122754 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5x95\" (UniqueName: \"kubernetes.io/projected/bc424839-f613-4159-bb7f-0c1769bf6200-kube-api-access-b5x95\") pod \"network-metrics-daemon-q8qt5\" (UID: \"bc424839-f613-4159-bb7f-0c1769bf6200\") " pod="openshift-multus/network-metrics-daemon-q8qt5" Nov 16 14:56:02.122833 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.122839 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc424839-f613-4159-bb7f-0c1769bf6200-metrics-certs\") pod \"network-metrics-daemon-q8qt5\" (UID: \"bc424839-f613-4159-bb7f-0c1769bf6200\") " pod="openshift-multus/network-metrics-daemon-q8qt5" Nov 16 14:56:02.223686 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.223647 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc424839-f613-4159-bb7f-0c1769bf6200-metrics-certs\") pod \"network-metrics-daemon-q8qt5\" (UID: \"bc424839-f613-4159-bb7f-0c1769bf6200\") " pod="openshift-multus/network-metrics-daemon-q8qt5" Nov 16 14:56:02.224008 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.223741 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-b5x95\" (UniqueName: \"kubernetes.io/projected/bc424839-f613-4159-bb7f-0c1769bf6200-kube-api-access-b5x95\") pod \"network-metrics-daemon-q8qt5\" (UID: \"bc424839-f613-4159-bb7f-0c1769bf6200\") " pod="openshift-multus/network-metrics-daemon-q8qt5" Nov 16 14:56:02.227251 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.227220 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc424839-f613-4159-bb7f-0c1769bf6200-metrics-certs\") pod \"network-metrics-daemon-q8qt5\" (UID: \"bc424839-f613-4159-bb7f-0c1769bf6200\") " pod="openshift-multus/network-metrics-daemon-q8qt5" Nov 16 14:56:02.250855 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.250746 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5x95\" (UniqueName: \"kubernetes.io/projected/bc424839-f613-4159-bb7f-0c1769bf6200-kube-api-access-b5x95\") pod \"network-metrics-daemon-q8qt5\" (UID: \"bc424839-f613-4159-bb7f-0c1769bf6200\") " pod="openshift-multus/network-metrics-daemon-q8qt5" Nov 16 14:56:02.340864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.340818 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8qt5" Nov 16 14:56:02.341900 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:02.341730840Z" level=info msg="Running pod sandbox: openshift-multus/network-metrics-daemon-q8qt5/POD" id=a902116c-b49f-4139-b13f-d4fbb9bfd8e9 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:56:02.341900 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:02.341881640Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:56:02.411530 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:02.411466175Z" level=info msg="Got pod network &{Name:network-metrics-daemon-q8qt5 Namespace:openshift-multus ID:eecf8f9867144bc5b05659a0fd8d0821ac112cde8b15aa73366cd1707e234df0 UID:bc424839-f613-4159-bb7f-0c1769bf6200 NetNS:/var/run/netns/027948c7-0682-471c-9980-1b1105a80ec8 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:56:02.411530 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:02.411524675Z" level=info msg="Adding pod openshift-multus_network-metrics-daemon-q8qt5 to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:56:02.456220 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:56:02.475393 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610562.4753] device (veth26d4a1d1): carrier: link connected Nov 16 14:56:02.481673 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth26d4a1d1: link is not ready Nov 16 14:56:02.481768 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth26d4a1d1: link becomes ready Nov 16 14:56:02.481803 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:56:02.489729 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[189350]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:56:02.489925 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[189350]: Could not generate persistent MAC address for veth26d4a1d1: No such file or directory Nov 16 14:56:02.490081 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610562.4900] manager: (veth26d4a1d1): new Veth device (/org/freedesktop/NetworkManager/Devices/82) Nov 16 14:56:02.511560 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610562.5115] manager: (veth26d4a1d1): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/83) Nov 16 14:56:02.517536 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00236|bridge|INFO|bridge br0: added interface veth26d4a1d1 on port 39 Nov 16 14:56:02.518166 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth26d4a1d1 entered promiscuous mode Nov 16 14:56:02.551774 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00237|connmgr|INFO|br0<->unix#535: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:56:02.597296 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00238|connmgr|INFO|br0<->unix#538: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:56:02.597571 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:02.597378 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=6754c3dd-53dd-4e4f-8d5f-ce077669c3d4 path="/var/lib/kubelet/pods/6754c3dd-53dd-4e4f-8d5f-ce077669c3d4/volumes" Nov 16 14:56:04.065122 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:56:04Z [verbose] Add: openshift-multus:network-metrics-daemon-q8qt5:bc424839-f613-4159-bb7f-0c1769bf6200:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/027948c7-0682-471c-9980-1b1105a80ec8"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.38/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:56:04.065122 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:56:04.035041 189333 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-multus", Name:"network-metrics-daemon-q8qt5", UID:"bc424839-f613-4159-bb7f-0c1769bf6200", APIVersion:"v1", ResourceVersion:"103207", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.38/23] from openshift-sdn Nov 16 14:56:04.065953 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:04.065908936Z" level=info msg="Got pod network &{Name:network-metrics-daemon-q8qt5 Namespace:openshift-multus ID:eecf8f9867144bc5b05659a0fd8d0821ac112cde8b15aa73366cd1707e234df0 UID:bc424839-f613-4159-bb7f-0c1769bf6200 NetNS:/var/run/netns/027948c7-0682-471c-9980-1b1105a80ec8 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:56:04.066095 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:04.066058437Z" level=info msg="Checking pod openshift-multus_network-metrics-daemon-q8qt5 for CNI network multus-cni-network (type=multus)" Nov 16 14:56:04.469853 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:04.469802 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-sdn/sdn-tqwqj" Nov 16 14:56:04.539163 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:04.539108 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-multus/network-metrics-daemon-q8qt5] Nov 16 14:56:04.694228 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:56:04.694181 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc424839_f613_4159_bb7f_0c1769bf6200.slice/crio-eecf8f9867144bc5b05659a0fd8d0821ac112cde8b15aa73366cd1707e234df0.scope WatchSource:0}: Error finding container eecf8f9867144bc5b05659a0fd8d0821ac112cde8b15aa73366cd1707e234df0: Status 404 returned error &{%!s(*http.body=&{0xc001d74f48 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:56:04.712174 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:04.712057993Z" level=info msg="Ran pod sandbox eecf8f9867144bc5b05659a0fd8d0821ac112cde8b15aa73366cd1707e234df0 with infra container: openshift-multus/network-metrics-daemon-q8qt5/POD" id=a902116c-b49f-4139-b13f-d4fbb9bfd8e9 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:56:04.713260 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:04.713226503Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:47ccba420402e65f17da159b2d9fa4da8135a8b3956fcc60d45cbfac3aaa4c9b" id=2f3d747d-f004-4a3a-8a2f-3dd712fc9be2 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:56:04.713497 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:04.713467305Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:47ccba420402e65f17da159b2d9fa4da8135a8b3956fcc60d45cbfac3aaa4c9b not found" id=2f3d747d-f004-4a3a-8a2f-3dd712fc9be2 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:56:04.714345 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:04.714270711Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:47ccba420402e65f17da159b2d9fa4da8135a8b3956fcc60d45cbfac3aaa4c9b" id=b1280a8c-0025-489e-9a4b-3b78ede7d7a6 name=/runtime.v1.ImageService/PullImage Nov 16 14:56:04.717720 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:04.717684340Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:47ccba420402e65f17da159b2d9fa4da8135a8b3956fcc60d45cbfac3aaa4c9b\"" Nov 16 14:56:04.876107 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:04.876008052Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4aec10de69951bafb2f44821a49e72e192c36dc803224766ad8a44f23f22ec9d" id=68593a00-00de-4e68-981e-f6b530d10c30 name=/runtime.v1.ImageService/PullImage Nov 16 14:56:04.877311 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:04.877267763Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4aec10de69951bafb2f44821a49e72e192c36dc803224766ad8a44f23f22ec9d" id=18f78b70-9def-4f4b-808b-d06c8bb18934 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:56:04.879217 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:04.879157979Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:a7d17beba3a65a4132b9891950555da0e4eb83ee609968255c5a4930c4e102e0,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4aec10de69951bafb2f44821a49e72e192c36dc803224766ad8a44f23f22ec9d],Size_:459216114,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=18f78b70-9def-4f4b-808b-d06c8bb18934 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:56:04.880550 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:04.880458289Z" level=info msg="Creating container: openshift-dns/node-resolver-27gwx/dns-node-resolver" id=b8af9203-0290-4a8b-a16d-77e97e14e112 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:56:04.880639 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:04.880567290Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:56:04.922727 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:04.922687 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q8qt5" event=&{ID:bc424839-f613-4159-bb7f-0c1769bf6200 Type:ContainerStarted Data:eecf8f9867144bc5b05659a0fd8d0821ac112cde8b15aa73366cd1707e234df0} Nov 16 14:56:04.948345 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-94772e879a2b64e8e4d75de29c7fd0b6891fc7afd7bf5fbd13f439dacc348a9b.scope. Nov 16 14:56:04.979975 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 94772e879a2b64e8e4d75de29c7fd0b6891fc7afd7bf5fbd13f439dacc348a9b. Nov 16 14:56:05.059340 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:05.059285272Z" level=info msg="Created container 94772e879a2b64e8e4d75de29c7fd0b6891fc7afd7bf5fbd13f439dacc348a9b: openshift-dns/node-resolver-27gwx/dns-node-resolver" id=b8af9203-0290-4a8b-a16d-77e97e14e112 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:56:05.068859 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:05.068800551Z" level=info msg="Starting container: 94772e879a2b64e8e4d75de29c7fd0b6891fc7afd7bf5fbd13f439dacc348a9b" id=5e80686a-bed3-4d0c-b4aa-149e2ade2104 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:56:05.083952 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:05.083877276Z" level=info msg="Started container" PID=189490 containerID=94772e879a2b64e8e4d75de29c7fd0b6891fc7afd7bf5fbd13f439dacc348a9b description=openshift-dns/node-resolver-27gwx/dns-node-resolver id=5e80686a-bed3-4d0c-b4aa-149e2ade2104 name=/runtime.v1.RuntimeService/StartContainer sandboxID=e87b2f605404e539c9ac5412ace52d2646b00639ba116cad735f33c9dd795753 Nov 16 14:56:05.927907 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:05.927860 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-27gwx" event=&{ID:8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2 Type:ContainerStarted Data:94772e879a2b64e8e4d75de29c7fd0b6891fc7afd7bf5fbd13f439dacc348a9b} Nov 16 14:56:05.941467 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:05.941423686Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:47ccba420402e65f17da159b2d9fa4da8135a8b3956fcc60d45cbfac3aaa4c9b\"" Nov 16 14:56:08.764573 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:08.764437483Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:47ccba420402e65f17da159b2d9fa4da8135a8b3956fcc60d45cbfac3aaa4c9b" id=b1280a8c-0025-489e-9a4b-3b78ede7d7a6 name=/runtime.v1.ImageService/PullImage Nov 16 14:56:08.765990 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:08.765891496Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:47ccba420402e65f17da159b2d9fa4da8135a8b3956fcc60d45cbfac3aaa4c9b" id=ba039849-d820-4ec6-8277-c3c29ee9f3e9 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:56:08.767520 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:08.767438810Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:241da7bd151e7ebd8806c8137617225a56913e9af0690733b26d30e87f445c07,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:47ccba420402e65f17da159b2d9fa4da8135a8b3956fcc60d45cbfac3aaa4c9b],Size_:370813878,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=ba039849-d820-4ec6-8277-c3c29ee9f3e9 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:56:08.768460 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:08.768424019Z" level=info msg="Creating container: openshift-multus/network-metrics-daemon-q8qt5/network-metrics-daemon" id=010ea795-bb3b-4a43-b7dd-06d549c4d941 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:56:08.768549 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:08.768529220Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:56:08.835703 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-cd6deaa62cd2e859aa700525b50246a76351c27d911f226a626e1bada2b631ca.scope. Nov 16 14:56:08.871525 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container cd6deaa62cd2e859aa700525b50246a76351c27d911f226a626e1bada2b631ca. Nov 16 14:56:08.910517 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:56:08.986385 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:08.986185798Z" level=info msg="Created container cd6deaa62cd2e859aa700525b50246a76351c27d911f226a626e1bada2b631ca: openshift-multus/network-metrics-daemon-q8qt5/network-metrics-daemon" id=010ea795-bb3b-4a43-b7dd-06d549c4d941 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:56:08.986833 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:08.986793403Z" level=info msg="Starting container: cd6deaa62cd2e859aa700525b50246a76351c27d911f226a626e1bada2b631ca" id=b782f8f8-bba7-4ec1-a2d5-abcf8188117f name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:56:08.999247 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:08.999201516Z" level=info msg="Started container" PID=189701 containerID=cd6deaa62cd2e859aa700525b50246a76351c27d911f226a626e1bada2b631ca description=openshift-multus/network-metrics-daemon-q8qt5/network-metrics-daemon id=b782f8f8-bba7-4ec1-a2d5-abcf8188117f name=/runtime.v1.RuntimeService/StartContainer sandboxID=eecf8f9867144bc5b05659a0fd8d0821ac112cde8b15aa73366cd1707e234df0 Nov 16 14:56:09.034902 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:09.034774239Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=a85a792d-eebb-4844-ab6e-150734092c69 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:56:09.035040 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:09.035021841Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=a85a792d-eebb-4844-ab6e-150734092c69 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:56:09.040625 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:09.040585992Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=fda600dd-2583-484a-912a-bb1fb68274b4 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:56:09.040824 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:09.040788094Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=fda600dd-2583-484a-912a-bb1fb68274b4 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:56:09.041731 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:09.041680302Z" level=info msg="Creating container: openshift-multus/network-metrics-daemon-q8qt5/kube-rbac-proxy" id=2462d878-c012-47c8-9c42-2fa7a18ad101 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:56:09.041836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:09.041789203Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:56:09.121934 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-76f47f0bf5f966390a8e3eb618ea25ee4ff6a44639ee2d41d6a11d8d108cbd3b.scope. Nov 16 14:56:09.141998 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 76f47f0bf5f966390a8e3eb618ea25ee4ff6a44639ee2d41d6a11d8d108cbd3b. Nov 16 14:56:09.182350 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:56:09.253085 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:09.252983722Z" level=info msg="Created container 76f47f0bf5f966390a8e3eb618ea25ee4ff6a44639ee2d41d6a11d8d108cbd3b: openshift-multus/network-metrics-daemon-q8qt5/kube-rbac-proxy" id=2462d878-c012-47c8-9c42-2fa7a18ad101 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:56:09.253741 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:09.253665728Z" level=info msg="Starting container: 76f47f0bf5f966390a8e3eb618ea25ee4ff6a44639ee2d41d6a11d8d108cbd3b" id=291f557e-847c-4363-8329-f50ce9d8460e name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:56:09.264689 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:09.264610628Z" level=info msg="Started container" PID=189752 containerID=76f47f0bf5f966390a8e3eb618ea25ee4ff6a44639ee2d41d6a11d8d108cbd3b description=openshift-multus/network-metrics-daemon-q8qt5/kube-rbac-proxy id=291f557e-847c-4363-8329-f50ce9d8460e name=/runtime.v1.RuntimeService/StartContainer sandboxID=eecf8f9867144bc5b05659a0fd8d0821ac112cde8b15aa73366cd1707e234df0 Nov 16 14:56:09.942314 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:09.942239 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q8qt5" event=&{ID:bc424839-f613-4159-bb7f-0c1769bf6200 Type:ContainerStarted Data:76f47f0bf5f966390a8e3eb618ea25ee4ff6a44639ee2d41d6a11d8d108cbd3b} Nov 16 14:56:09.942314 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:09.942317 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q8qt5" event=&{ID:bc424839-f613-4159-bb7f-0c1769bf6200 Type:ContainerStarted Data:cd6deaa62cd2e859aa700525b50246a76351c27d911f226a626e1bada2b631ca} Nov 16 14:56:21.877616 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:21.877546342Z" level=info msg="Stopping pod sandbox: 6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75" id=44d18e7d-fea8-4e6f-8dd6-7bd0c3ea20e1 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:56:21.877616 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:21.877605043Z" level=info msg="Stopped pod sandbox (already stopped): 6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75" id=44d18e7d-fea8-4e6f-8dd6-7bd0c3ea20e1 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:56:21.878102 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:21.877853645Z" level=info msg="Removing pod sandbox: 6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75" id=a47e8a79-c024-4c2c-83d4-c67e20026681 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:56:22.666494 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:22.666359441Z" level=info msg="Removed pod sandbox: 6faa4b2484d1ac39f4d34f1fbbc8103baf1a2eae5be7480794a3faab56c9cc75" id=a47e8a79-c024-4c2c-83d4-c67e20026681 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:56:22.667331 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:22.667276649Z" level=info msg="Stopping pod sandbox: 57035fc21822c56b1677df50b84a40801ec6643b61f910c3ed96ed7ba49c71b8" id=b9ca6635-a051-40eb-bb06-c95460f2e03c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:56:22.667467 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:22.667359250Z" level=info msg="Stopped pod sandbox (already stopped): 57035fc21822c56b1677df50b84a40801ec6643b61f910c3ed96ed7ba49c71b8" id=b9ca6635-a051-40eb-bb06-c95460f2e03c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:56:22.667930 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:22.667863554Z" level=info msg="Removing pod sandbox: 57035fc21822c56b1677df50b84a40801ec6643b61f910c3ed96ed7ba49c71b8" id=57fdc342-41e4-4fe8-8bfc-bd57e8643044 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:56:23.468979 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:23.468889013Z" level=info msg="Removed pod sandbox: 57035fc21822c56b1677df50b84a40801ec6643b61f910c3ed96ed7ba49c71b8" id=57fdc342-41e4-4fe8-8bfc-bd57e8643044 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:56:23.469569 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:23.469516718Z" level=info msg="Stopping pod sandbox: 2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a" id=5391b96c-031c-4768-b80c-7ed43f57480f name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:56:23.469569 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:23.469554319Z" level=info msg="Stopped pod sandbox (already stopped): 2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a" id=5391b96c-031c-4768-b80c-7ed43f57480f name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:56:23.470016 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:23.469921822Z" level=info msg="Removing pod sandbox: 2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a" id=ca21dfde-3efd-424a-8ae6-e118cf2961e2 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:56:23.524100 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:23.524044078Z" level=info msg="Removed pod sandbox: 2045e6fd175c1872a2807cff78a18883986c821ae1986f837ce49a024cf1a83a" id=ca21dfde-3efd-424a-8ae6-e118cf2961e2 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:56:23.524624 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:23.524583783Z" level=info msg="Stopping pod sandbox: 31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427" id=29d464c1-dd2f-4326-a35b-00ba9665f846 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:56:23.524727 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:23.524630083Z" level=info msg="Stopped pod sandbox (already stopped): 31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427" id=29d464c1-dd2f-4326-a35b-00ba9665f846 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:56:23.524955 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:23.524917986Z" level=info msg="Removing pod sandbox: 31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427" id=60a2ef32-81b6-4570-9869-b0a9b8d1feeb name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:56:23.607560 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:23.607506983Z" level=info msg="Removed pod sandbox: 31216a8932fa619b2cf0243de303f525c0b3e5b730ff7bd68ac770a707abf427" id=60a2ef32-81b6-4570-9869-b0a9b8d1feeb name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:56:32.710216 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:32.710038 1563 configmap.go:200] Couldn't get configMap openshift-multus/cni-binary-copy-script: configmap "cni-binary-copy-script" not found Nov 16 14:56:32.711081 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:32.710268 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy podName:13c954d2-615d-4b55-be8a-e712b8fa30a8 nodeName:}" failed. No retries permitted until 2022-11-16 14:56:33.21024591 +0000 UTC m=+10998.147021532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cni-binary-copy" (UniqueName: "kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy") pod "multus-additional-cni-plugins-q722j" (UID: "13c954d2-615d-4b55-be8a-e712b8fa30a8") : configmap "cni-binary-copy-script" not found Nov 16 14:56:33.216573 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:33.216488 1563 configmap.go:200] Couldn't get configMap openshift-multus/cni-binary-copy-script: configmap "cni-binary-copy-script" not found Nov 16 14:56:33.216573 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:33.216584 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy podName:13c954d2-615d-4b55-be8a-e712b8fa30a8 nodeName:}" failed. No retries permitted until 2022-11-16 14:56:34.216563686 +0000 UTC m=+10999.153339308 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cni-binary-copy" (UniqueName: "kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy") pod "multus-additional-cni-plugins-q722j" (UID: "13c954d2-615d-4b55-be8a-e712b8fa30a8") : configmap "cni-binary-copy-script" not found Nov 16 14:56:34.229396 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:34.229337 1563 configmap.go:200] Couldn't get configMap openshift-multus/cni-binary-copy-script: configmap "cni-binary-copy-script" not found Nov 16 14:56:34.229872 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:34.229448 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy podName:13c954d2-615d-4b55-be8a-e712b8fa30a8 nodeName:}" failed. No retries permitted until 2022-11-16 14:56:36.229422842 +0000 UTC m=+11001.166198464 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cni-binary-copy" (UniqueName: "kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy") pod "multus-additional-cni-plugins-q722j" (UID: "13c954d2-615d-4b55-be8a-e712b8fa30a8") : configmap "cni-binary-copy-script" not found Nov 16 14:56:36.246720 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:36.246628 1563 configmap.go:200] Couldn't get configMap openshift-multus/cni-binary-copy-script: configmap "cni-binary-copy-script" not found Nov 16 14:56:36.247235 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:36.246779 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy podName:13c954d2-615d-4b55-be8a-e712b8fa30a8 nodeName:}" failed. No retries permitted until 2022-11-16 14:56:40.246749881 +0000 UTC m=+11005.183525603 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cni-binary-copy" (UniqueName: "kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy") pod "multus-additional-cni-plugins-q722j" (UID: "13c954d2-615d-4b55-be8a-e712b8fa30a8") : configmap "cni-binary-copy-script" not found Nov 16 14:56:40.288164 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:40.288080 1563 configmap.go:200] Couldn't get configMap openshift-multus/cni-binary-copy-script: configmap "cni-binary-copy-script" not found Nov 16 14:56:40.288616 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:40.288192 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy podName:13c954d2-615d-4b55-be8a-e712b8fa30a8 nodeName:}" failed. No retries permitted until 2022-11-16 14:56:48.288168566 +0000 UTC m=+11013.224944288 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cni-binary-copy" (UniqueName: "kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy") pod "multus-additional-cni-plugins-q722j" (UID: "13c954d2-615d-4b55-be8a-e712b8fa30a8") : configmap "cni-binary-copy-script" not found Nov 16 14:56:48.371461 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:48.371365 1563 configmap.go:200] Couldn't get configMap openshift-multus/cni-binary-copy-script: configmap "cni-binary-copy-script" not found Nov 16 14:56:48.371461 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:56:48.371475 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy podName:13c954d2-615d-4b55-be8a-e712b8fa30a8 nodeName:}" failed. No retries permitted until 2022-11-16 14:57:04.371456976 +0000 UTC m=+11029.308232598 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cni-binary-copy" (UniqueName: "kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy") pod "multus-additional-cni-plugins-q722j" (UID: "13c954d2-615d-4b55-be8a-e712b8fa30a8") : configmap "cni-binary-copy-script" not found Nov 16 14:56:59.905090 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:59.904996 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh] Nov 16 14:56:59.905657 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:59.905322 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" podUID=89deb745-c538-46ae-9d77-26643bf01a15 containerName="csi-driver" containerID="cri-o://3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1" gracePeriod=30 Nov 16 14:56:59.905657 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:59.905396 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" podUID=89deb745-c538-46ae-9d77-26643bf01a15 containerName="csi-liveness-probe" containerID="cri-o://c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016" gracePeriod=30 Nov 16 14:56:59.905910 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:59.905849388Z" level=info msg="Stopping container: 3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1 (timeout: 30s)" id=ada5e8ee-1726-4f24-a2b7-73437412b4b3 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:56:59.906249 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:59.905849388Z" level=info msg="Stopping container: c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016 (timeout: 30s)" id=36300612-dcf9-433a-97d4-e9adb17e3302 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:56:59.927046 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016.scope: Succeeded. Nov 16 14:56:59.927701 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016.scope: Consumed 35ms CPU time Nov 16 14:56:59.955542 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016.scope: Succeeded. Nov 16 14:56:59.956341 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016.scope: Consumed 3.942s CPU time Nov 16 14:56:59.956720 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1.scope: Succeeded. Nov 16 14:56:59.957311 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1.scope: Consumed 2.565s CPU time Nov 16 14:56:59.957533 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1.scope: Succeeded. Nov 16 14:56:59.958091 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1.scope: Consumed 32ms CPU time Nov 16 14:56:59.966216 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:59.966175 1563 plugin_watcher.go:215] "Removing socket path from desired state cache" path="/var/lib/kubelet/plugins_registry/disk.csi.azure.com-reg.sock" Nov 16 14:56:59.967648 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:56:59.967586 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" podUID=89deb745-c538-46ae-9d77-26643bf01a15 containerName="csi-node-driver-registrar" containerID="cri-o://83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0" gracePeriod=30 Nov 16 14:56:59.967882 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:56:59.967838961Z" level=info msg="Stopping container: 83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0 (timeout: 30s)" id=a1593cc0-b6b3-401c-99c8-552159fb6ff0 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:56:59.999446 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0.scope: Succeeded. Nov 16 14:57:00.000174 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0.scope: Consumed 416ms CPU time Nov 16 14:57:00.010589 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0.scope: Succeeded. Nov 16 14:57:00.011196 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0.scope: Consumed 34ms CPU time Nov 16 14:57:00.066249 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-e8c7530f62990a0491d9ae617d412e49309fdf2572958626e3ac57e8348928b9-merged.mount: Succeeded. Nov 16 14:57:00.067268 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-e8c7530f62990a0491d9ae617d412e49309fdf2572958626e3ac57e8348928b9-merged.mount: Consumed 0 CPU time Nov 16 14:57:00.105807 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:00.105707 1563 generic.go:296] "Generic (PLEG): container finished" podID=89deb745-c538-46ae-9d77-26643bf01a15 containerID="c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016" exitCode=2 Nov 16 14:57:00.105807 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:00.105742 1563 generic.go:296] "Generic (PLEG): container finished" podID=89deb745-c538-46ae-9d77-26643bf01a15 containerID="3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1" exitCode=2 Nov 16 14:57:00.105807 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:00.105780 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" event=&{ID:89deb745-c538-46ae-9d77-26643bf01a15 Type:ContainerDied Data:c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016} Nov 16 14:57:00.105807 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:00.105808 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" event=&{ID:89deb745-c538-46ae-9d77-26643bf01a15 Type:ContainerDied Data:3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1} Nov 16 14:57:00.213740 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:00.213504 1563 csi_plugin.go:178] kubernetes.io/csi: registrationHandler.DeRegisterPlugin request for plugin disk.csi.azure.com Nov 16 14:57:00.213740 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:00.213544 1563 reconciler.go:143] "OperationExecutor.UnregisterPlugin started" plugin={SocketPath:/var/lib/kubelet/plugins_registry/disk.csi.azure.com-reg.sock Timestamp:2022-11-16 11:59:41.058903096 +0000 UTC m=+385.995678718 Handler:0x7d060a0 Name:disk.csi.azure.com} Nov 16 14:57:00.639175 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-efb351bf8ab890f8474eacf50623edd37225b9aa71139378e9cc51b86b33bae7-merged.mount: Succeeded. Nov 16 14:57:00.640115 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-efb351bf8ab890f8474eacf50623edd37225b9aa71139378e9cc51b86b33bae7-merged.mount: Consumed 0 CPU time Nov 16 14:57:00.645410 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:00.645340418Z" level=info msg="Stopped container c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-liveness-probe" id=36300612-dcf9-433a-97d4-e9adb17e3302 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:57:00.739688 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-fbd1e928d1af25702b3808bb2ce3cce45666e2b40e19ff60397f7abb20f0767c-merged.mount: Succeeded. Nov 16 14:57:00.740735 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-fbd1e928d1af25702b3808bb2ce3cce45666e2b40e19ff60397f7abb20f0767c-merged.mount: Consumed 0 CPU time Nov 16 14:57:00.741041 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:00.740990357Z" level=info msg="Stopped container 3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-driver" id=ada5e8ee-1726-4f24-a2b7-73437412b4b3 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:57:00.875159 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:00.875024313Z" level=info msg="Stopped container 83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-node-driver-registrar" id=a1593cc0-b6b3-401c-99c8-552159fb6ff0 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:57:00.875764 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:00.875724820Z" level=info msg="Stopping pod sandbox: 8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883" id=c6c1195d-0317-42cd-bdf5-4a5254795f3b name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:57:00.939149 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-15dc69bb\x2d3243\x2d405e\x2d8c5a\x2dae64f1dbbd6c.mount: Succeeded. Nov 16 14:57:00.940265 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-15dc69bb\x2d3243\x2d405e\x2d8c5a\x2dae64f1dbbd6c.mount: Consumed 0 CPU time Nov 16 14:57:00.940711 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-15dc69bb\x2d3243\x2d405e\x2d8c5a\x2dae64f1dbbd6c.mount: Succeeded. Nov 16 14:57:00.941634 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-15dc69bb\x2d3243\x2d405e\x2d8c5a\x2dae64f1dbbd6c.mount: Consumed 0 CPU time Nov 16 14:57:00.941676 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-15dc69bb\x2d3243\x2d405e\x2d8c5a\x2dae64f1dbbd6c.mount: Succeeded. Nov 16 14:57:00.942579 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-15dc69bb\x2d3243\x2d405e\x2d8c5a\x2dae64f1dbbd6c.mount: Consumed 0 CPU time Nov 16 14:57:00.951286 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-123c20805d6862f4243e7304d8fd6924ee60ba7596820fd53d19fe6cb694986a-merged.mount: Succeeded. Nov 16 14:57:00.952249 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-123c20805d6862f4243e7304d8fd6924ee60ba7596820fd53d19fe6cb694986a-merged.mount: Consumed 0 CPU time Nov 16 14:57:00.952269 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883-userdata-shm.mount: Succeeded. Nov 16 14:57:00.953156 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883-userdata-shm.mount: Consumed 0 CPU time Nov 16 14:57:01.111712 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.111590 1563 generic.go:296] "Generic (PLEG): container finished" podID=89deb745-c538-46ae-9d77-26643bf01a15 containerID="83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0" exitCode=0 Nov 16 14:57:01.111712 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.111644 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" event=&{ID:89deb745-c538-46ae-9d77-26643bf01a15 Type:ContainerDied Data:83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0} Nov 16 14:57:01.120805 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:01.120718381Z" level=info msg="Stopped pod sandbox: 8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883" id=c6c1195d-0317-42cd-bdf5-4a5254795f3b name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:57:01.205249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.204981 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-registration-dir\") pod \"89deb745-c538-46ae-9d77-26643bf01a15\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " Nov 16 14:57:01.205249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205082 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"host-cloud-config\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-host-cloud-config\") pod \"89deb745-c538-46ae-9d77-26643bf01a15\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " Nov 16 14:57:01.205249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205122 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"merged-cloud-config\" (UniqueName: \"kubernetes.io/empty-dir/89deb745-c538-46ae-9d77-26643bf01a15-merged-cloud-config\") pod \"89deb745-c538-46ae-9d77-26643bf01a15\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " Nov 16 14:57:01.205249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205188 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"scsi-host-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-scsi-host-dir\") pod \"89deb745-c538-46ae-9d77-26643bf01a15\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " Nov 16 14:57:01.205249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205184 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-registration-dir" (OuterVolumeSpecName: "registration-dir") pod "89deb745-c538-46ae-9d77-26643bf01a15" (UID: "89deb745-c538-46ae-9d77-26643bf01a15"). InnerVolumeSpecName "registration-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:57:01.205249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205234 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-host-cloud-config" (OuterVolumeSpecName: "host-cloud-config") pod "89deb745-c538-46ae-9d77-26643bf01a15" (UID: "89deb745-c538-46ae-9d77-26643bf01a15"). InnerVolumeSpecName "host-cloud-config". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:57:01.205249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205255 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89deb745-c538-46ae-9d77-26643bf01a15-non-standard-root-system-trust-ca-bundle\") pod \"89deb745-c538-46ae-9d77-26643bf01a15\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " Nov 16 14:57:01.205662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205314 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"sys-devices-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-sys-devices-dir\") pod \"89deb745-c538-46ae-9d77-26643bf01a15\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " Nov 16 14:57:01.205662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205366 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-mountpoint-dir\") pod \"89deb745-c538-46ae-9d77-26643bf01a15\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " Nov 16 14:57:01.205662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205360 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-scsi-host-dir" (OuterVolumeSpecName: "scsi-host-dir") pod "89deb745-c538-46ae-9d77-26643bf01a15" (UID: "89deb745-c538-46ae-9d77-26643bf01a15"). InnerVolumeSpecName "scsi-host-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:57:01.205662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205389 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-sys-devices-dir" (OuterVolumeSpecName: "sys-devices-dir") pod "89deb745-c538-46ae-9d77-26643bf01a15" (UID: "89deb745-c538-46ae-9d77-26643bf01a15"). InnerVolumeSpecName "sys-devices-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:57:01.205662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205392 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnggp\" (UniqueName: \"kubernetes.io/projected/89deb745-c538-46ae-9d77-26643bf01a15-kube-api-access-mnggp\") pod \"89deb745-c538-46ae-9d77-26643bf01a15\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " Nov 16 14:57:01.205662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205424 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-mountpoint-dir" (OuterVolumeSpecName: "mountpoint-dir") pod "89deb745-c538-46ae-9d77-26643bf01a15" (UID: "89deb745-c538-46ae-9d77-26643bf01a15"). InnerVolumeSpecName "mountpoint-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:57:01.205662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205473 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-device-dir\") pod \"89deb745-c538-46ae-9d77-26643bf01a15\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " Nov 16 14:57:01.205662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:57:01.205464 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/89deb745-c538-46ae-9d77-26643bf01a15/volumes/kubernetes.io~empty-dir/merged-cloud-config: clearQuota called, but quotas disabled Nov 16 14:57:01.205662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:57:01.205523 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/89deb745-c538-46ae-9d77-26643bf01a15/volumes/kubernetes.io~configmap/non-standard-root-system-trust-ca-bundle: clearQuota called, but quotas disabled Nov 16 14:57:01.205662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205533 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-socket-dir" (OuterVolumeSpecName: "socket-dir") pod "89deb745-c538-46ae-9d77-26643bf01a15" (UID: "89deb745-c538-46ae-9d77-26643bf01a15"). InnerVolumeSpecName "socket-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:57:01.205662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205510 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-device-dir" (OuterVolumeSpecName: "device-dir") pod "89deb745-c538-46ae-9d77-26643bf01a15" (UID: "89deb745-c538-46ae-9d77-26643bf01a15"). InnerVolumeSpecName "device-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:57:01.205662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205511 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-socket-dir\") pod \"89deb745-c538-46ae-9d77-26643bf01a15\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " Nov 16 14:57:01.205662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205643 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"msi\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-msi\") pod \"89deb745-c538-46ae-9d77-26643bf01a15\" (UID: \"89deb745-c538-46ae-9d77-26643bf01a15\") " Nov 16 14:57:01.206051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205679 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89deb745-c538-46ae-9d77-26643bf01a15-merged-cloud-config" (OuterVolumeSpecName: "merged-cloud-config") pod "89deb745-c538-46ae-9d77-26643bf01a15" (UID: "89deb745-c538-46ae-9d77-26643bf01a15"). InnerVolumeSpecName "merged-cloud-config". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Nov 16 14:57:01.206051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205716 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-msi" (OuterVolumeSpecName: "msi") pod "89deb745-c538-46ae-9d77-26643bf01a15" (UID: "89deb745-c538-46ae-9d77-26643bf01a15"). InnerVolumeSpecName "msi". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:57:01.206051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205760 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89deb745-c538-46ae-9d77-26643bf01a15-non-standard-root-system-trust-ca-bundle" (OuterVolumeSpecName: "non-standard-root-system-trust-ca-bundle") pod "89deb745-c538-46ae-9d77-26643bf01a15" (UID: "89deb745-c538-46ae-9d77-26643bf01a15"). InnerVolumeSpecName "non-standard-root-system-trust-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 14:57:01.206051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205790 1563 reconciler.go:300] "Volume detached for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-registration-dir\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:57:01.206051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205811 1563 reconciler.go:300] "Volume detached for volume \"host-cloud-config\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-host-cloud-config\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:57:01.206051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205826 1563 reconciler.go:300] "Volume detached for volume \"merged-cloud-config\" (UniqueName: \"kubernetes.io/empty-dir/89deb745-c538-46ae-9d77-26643bf01a15-merged-cloud-config\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:57:01.206051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205841 1563 reconciler.go:300] "Volume detached for volume \"scsi-host-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-scsi-host-dir\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:57:01.206051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205857 1563 reconciler.go:300] "Volume detached for volume \"sys-devices-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-sys-devices-dir\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:57:01.206051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205872 1563 reconciler.go:300] "Volume detached for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-mountpoint-dir\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:57:01.206051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205887 1563 reconciler.go:300] "Volume detached for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-socket-dir\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:57:01.206051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205903 1563 reconciler.go:300] "Volume detached for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-device-dir\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:57:01.206051 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.205916 1563 reconciler.go:300] "Volume detached for volume \"msi\" (UniqueName: \"kubernetes.io/host-path/89deb745-c538-46ae-9d77-26643bf01a15-msi\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:57:01.216622 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-89deb745\x2dc538\x2d46ae\x2d9d77\x2d26643bf01a15-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dmnggp.mount: Succeeded. Nov 16 14:57:01.217580 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-89deb745\x2dc538\x2d46ae\x2d9d77\x2d26643bf01a15-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dmnggp.mount: Consumed 0 CPU time Nov 16 14:57:01.229101 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.229021 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89deb745-c538-46ae-9d77-26643bf01a15-kube-api-access-mnggp" (OuterVolumeSpecName: "kube-api-access-mnggp") pod "89deb745-c538-46ae-9d77-26643bf01a15" (UID: "89deb745-c538-46ae-9d77-26643bf01a15"). InnerVolumeSpecName "kube-api-access-mnggp". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:57:01.306654 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.306557 1563 reconciler.go:300] "Volume detached for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89deb745-c538-46ae-9d77-26643bf01a15-non-standard-root-system-trust-ca-bundle\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:57:01.306654 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:01.306629 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-mnggp\" (UniqueName: \"kubernetes.io/projected/89deb745-c538-46ae-9d77-26643bf01a15-kube-api-access-mnggp\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:57:02.117876 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.117791 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh" event=&{ID:89deb745-c538-46ae-9d77-26643bf01a15 Type:ContainerDied Data:8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883} Nov 16 14:57:02.117876 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.117886 1563 scope.go:110] "RemoveContainer" containerID="c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016" Nov 16 14:57:02.118957 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.118913220Z" level=info msg="Removing container: c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016" id=a0f65dc9-9421-4cd8-a962-f8e1def37e8b name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:57:02.128061 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod89deb745_c538_46ae_9d77_26643bf01a15.slice. Nov 16 14:57:02.128193 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod89deb745_c538_46ae_9d77_26643bf01a15.slice: Consumed 7.089s CPU time Nov 16 14:57:02.153984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.153945 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh] Nov 16 14:57:02.156089 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.156040323Z" level=info msg="Removed container c373e898882f1c34675f0db60045914bf95531d61a0b9b80b1366281dc74b016: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-liveness-probe" id=a0f65dc9-9421-4cd8-a962-f8e1def37e8b name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:57:02.156516 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.156388 1563 scope.go:110] "RemoveContainer" containerID="83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0" Nov 16 14:57:02.157429 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.157395038Z" level=info msg="Removing container: 83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0" id=11671bd0-8671-4e9d-b321-829ca491d75c name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:57:02.164101 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.164021 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh] Nov 16 14:57:02.193732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.193669332Z" level=info msg="Removed container 83e388507ce9d35a1346947a7b718770b3dff11544e548dadcaa010c2c2c1ed0: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-node-driver-registrar" id=11671bd0-8671-4e9d-b321-829ca491d75c name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:57:02.194110 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.194082 1563 scope.go:110] "RemoveContainer" containerID="3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1" Nov 16 14:57:02.195256 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.195222349Z" level=info msg="Removing container: 3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1" id=5da98f53-2f84-4765-9243-998603f6d6bb name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:57:02.231593 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.231541543Z" level=info msg="Removed container 3534db91edb1c4a8425566fe5e9d4608e297d3030944f5e1ed7eac7f36e99db1: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/csi-driver" id=5da98f53-2f84-4765-9243-998603f6d6bb name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:57:02.231917 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.231859 1563 scope.go:110] "RemoveContainer" containerID="6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82" Nov 16 14:57:02.233261 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.233061959Z" level=info msg="Removing container: 6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82" id=e9947b90-8c75-4c1e-8609-a93295761d27 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:57:02.235685 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.235658 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6] Nov 16 14:57:02.235788 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.235721 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:57:02.235843 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:57:02.235825 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="89deb745-c538-46ae-9d77-26643bf01a15" containerName="csi-node-driver-registrar" Nov 16 14:57:02.235843 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.235840 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="89deb745-c538-46ae-9d77-26643bf01a15" containerName="csi-node-driver-registrar" Nov 16 14:57:02.235927 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:57:02.235851 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="89deb745-c538-46ae-9d77-26643bf01a15" containerName="azure-inject-credentials" Nov 16 14:57:02.235927 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.235860 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="89deb745-c538-46ae-9d77-26643bf01a15" containerName="azure-inject-credentials" Nov 16 14:57:02.235927 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:57:02.235873 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="89deb745-c538-46ae-9d77-26643bf01a15" containerName="csi-driver" Nov 16 14:57:02.235927 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.235881 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="89deb745-c538-46ae-9d77-26643bf01a15" containerName="csi-driver" Nov 16 14:57:02.235927 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:57:02.235894 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="89deb745-c538-46ae-9d77-26643bf01a15" containerName="csi-liveness-probe" Nov 16 14:57:02.235927 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.235902 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="89deb745-c538-46ae-9d77-26643bf01a15" containerName="csi-liveness-probe" Nov 16 14:57:02.236120 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.235975 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="89deb745-c538-46ae-9d77-26643bf01a15" containerName="csi-driver" Nov 16 14:57:02.236120 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.235992 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="89deb745-c538-46ae-9d77-26643bf01a15" containerName="csi-node-driver-registrar" Nov 16 14:57:02.236120 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.236004 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="89deb745-c538-46ae-9d77-26643bf01a15" containerName="csi-liveness-probe" Nov 16 14:57:02.247651 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod062fe84e_408b_4970_94b8_5df0ae82ba87.slice. Nov 16 14:57:02.261471 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0e8ae870dc107fe9d8a200b3abbcee5f7f00c283e3b53ed8033a06e8abc06157-merged.mount: Succeeded. Nov 16 14:57:02.262446 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0e8ae870dc107fe9d8a200b3abbcee5f7f00c283e3b53ed8033a06e8abc06157-merged.mount: Consumed 0 CPU time Nov 16 14:57:02.315228 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.315188 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prk5v\" (UniqueName: \"kubernetes.io/projected/062fe84e-408b-4970-94b8-5df0ae82ba87-kube-api-access-prk5v\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.315468 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.315248 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"merged-cloud-config\" (UniqueName: \"kubernetes.io/empty-dir/062fe84e-408b-4970-94b8-5df0ae82ba87-merged-cloud-config\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.315468 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.315332 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"msi\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-msi\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.315468 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.315384 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-socket-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.315468 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.315423 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/062fe84e-408b-4970-94b8-5df0ae82ba87-non-standard-root-system-trust-ca-bundle\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.315468 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.315456 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-device-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.315663 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.315565 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-registration-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.315737 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.315661 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-devices-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-sys-devices-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.315737 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.315733 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-mountpoint-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.315833 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.315794 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scsi-host-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-scsi-host-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.315872 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.315836 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cloud-config\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-host-cloud-config\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417281 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417124 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"scsi-host-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-scsi-host-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417281 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417226 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"host-cloud-config\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-host-cloud-config\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417281 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417273 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-prk5v\" (UniqueName: \"kubernetes.io/projected/062fe84e-408b-4970-94b8-5df0ae82ba87-kube-api-access-prk5v\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417281 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417274 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"scsi-host-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-scsi-host-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417593 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417323 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"merged-cloud-config\" (UniqueName: \"kubernetes.io/empty-dir/062fe84e-408b-4970-94b8-5df0ae82ba87-merged-cloud-config\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417593 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417323 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"host-cloud-config\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-host-cloud-config\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417593 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417429 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"msi\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-msi\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417593 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417466 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-socket-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417593 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417515 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/062fe84e-408b-4970-94b8-5df0ae82ba87-non-standard-root-system-trust-ca-bundle\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417593 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417561 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-device-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417598 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-registration-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417608 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-socket-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417560 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"msi\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-msi\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417631 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"sys-devices-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-sys-devices-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417684 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"sys-devices-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-sys-devices-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417689 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-registration-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417666 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"merged-cloud-config\" (UniqueName: \"kubernetes.io/empty-dir/062fe84e-408b-4970-94b8-5df0ae82ba87-merged-cloud-config\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417738 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-mountpoint-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417691 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-device-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.417838 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.417825 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/062fe84e-408b-4970-94b8-5df0ae82ba87-mountpoint-dir\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.418500 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.418472 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/062fe84e-408b-4970-94b8-5df0ae82ba87-non-standard-root-system-trust-ca-bundle\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.441781 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.441687 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-prk5v\" (UniqueName: \"kubernetes.io/projected/062fe84e-408b-4970-94b8-5df0ae82ba87-kube-api-access-prk5v\") pod \"azure-disk-csi-driver-node-p82p6\" (UID: \"062fe84e-408b-4970-94b8-5df0ae82ba87\") " pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.542080 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.541994414Z" level=info msg="Removed container 6558798c358482fdcbc3becd4c2879b1c35c0e7cfe27b17b91981769b1af7c82: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-f55gh/azure-inject-credentials" id=e9947b90-8c75-4c1e-8609-a93295761d27 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:57:02.565362 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.565264 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" Nov 16 14:57:02.565947 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.565878574Z" level=info msg="Running pod sandbox: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/POD" id=cca4f089-7d5d-4bbc-a104-1496e6f2b8af name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:57:02.566045 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.565964674Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:57:02.599818 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:02.599719 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=89deb745-c538-46ae-9d77-26643bf01a15 path="/var/lib/kubelet/pods/89deb745-c538-46ae-9d77-26643bf01a15/volumes" Nov 16 14:57:02.629967 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.629879269Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=cca4f089-7d5d-4bbc-a104-1496e6f2b8af name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:57:02.635191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:57:02.635146 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod062fe84e_408b_4970_94b8_5df0ae82ba87.slice/crio-c0c185f3a19a6a6d7f207a0845de00e3bb86afb4eaf336e2ec7a49ba9cde7f12.scope WatchSource:0}: Error finding container c0c185f3a19a6a6d7f207a0845de00e3bb86afb4eaf336e2ec7a49ba9cde7f12: Status 404 returned error &{%!s(*http.body=&{0xc002bc8ae0 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:57:02.651439 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.651387002Z" level=info msg="Ran pod sandbox c0c185f3a19a6a6d7f207a0845de00e3bb86afb4eaf336e2ec7a49ba9cde7f12 with infra container: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/POD" id=cca4f089-7d5d-4bbc-a104-1496e6f2b8af name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:57:02.652521 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.652479614Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:949e941fd91b2355bf6b755c5f036d170af7e07f5396d9a90177f5871fcbf6ba" id=48f9d7e1-750b-4acd-9f9d-b8b7bbbe277a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:02.652769 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.652735217Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5a4effe21da47ccc0ff77199890d3c1763c74455a1b0c52b3ebadb60c2b341ae,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:949e941fd91b2355bf6b755c5f036d170af7e07f5396d9a90177f5871fcbf6ba],Size_:438790711,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=48f9d7e1-750b-4acd-9f9d-b8b7bbbe277a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:02.653646 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.653603826Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:949e941fd91b2355bf6b755c5f036d170af7e07f5396d9a90177f5871fcbf6ba" id=528453d2-cf35-4a37-b8d5-5eabdb6aa118 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:02.653854 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.653810528Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5a4effe21da47ccc0ff77199890d3c1763c74455a1b0c52b3ebadb60c2b341ae,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:949e941fd91b2355bf6b755c5f036d170af7e07f5396d9a90177f5871fcbf6ba],Size_:438790711,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=528453d2-cf35-4a37-b8d5-5eabdb6aa118 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:02.654687 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.654645937Z" level=info msg="Creating container: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/azure-inject-credentials" id=c0a673fe-3d14-4018-86bf-c12d2e03bca4 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:57:02.654773 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.654749839Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:57:02.719295 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-85e47872a8d158098bc51ab2bf12cbd1e9121acbb0ee2d34c56ba5bf2fd2b685.scope. Nov 16 14:57:02.741448 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 85e47872a8d158098bc51ab2bf12cbd1e9121acbb0ee2d34c56ba5bf2fd2b685. Nov 16 14:57:02.777534 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:57:02.838848 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.838740163Z" level=info msg="Created container 85e47872a8d158098bc51ab2bf12cbd1e9121acbb0ee2d34c56ba5bf2fd2b685: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/azure-inject-credentials" id=c0a673fe-3d14-4018-86bf-c12d2e03bca4 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:57:02.839379 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.839342447Z" level=info msg="Starting container: 85e47872a8d158098bc51ab2bf12cbd1e9121acbb0ee2d34c56ba5bf2fd2b685" id=50b5e5b4-2ab1-4ad1-8364-d5c817c23dd8 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:57:02.852272 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:02.852224005Z" level=info msg="Started container" PID=191177 containerID=85e47872a8d158098bc51ab2bf12cbd1e9121acbb0ee2d34c56ba5bf2fd2b685 description=openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/azure-inject-credentials id=50b5e5b4-2ab1-4ad1-8364-d5c817c23dd8 name=/runtime.v1.RuntimeService/StartContainer sandboxID=c0c185f3a19a6a6d7f207a0845de00e3bb86afb4eaf336e2ec7a49ba9cde7f12 Nov 16 14:57:02.871180 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-85e47872a8d158098bc51ab2bf12cbd1e9121acbb0ee2d34c56ba5bf2fd2b685.scope: Succeeded. Nov 16 14:57:02.871853 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-85e47872a8d158098bc51ab2bf12cbd1e9121acbb0ee2d34c56ba5bf2fd2b685.scope: Consumed 33ms CPU time Nov 16 14:57:02.872139 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-85e47872a8d158098bc51ab2bf12cbd1e9121acbb0ee2d34c56ba5bf2fd2b685.scope: Succeeded. Nov 16 14:57:02.872741 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-85e47872a8d158098bc51ab2bf12cbd1e9121acbb0ee2d34c56ba5bf2fd2b685.scope: Consumed 41ms CPU time Nov 16 14:57:03.123607 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:03.123575 1563 generic.go:296] "Generic (PLEG): container finished" podID=062fe84e-408b-4970-94b8-5df0ae82ba87 containerID="85e47872a8d158098bc51ab2bf12cbd1e9121acbb0ee2d34c56ba5bf2fd2b685" exitCode=0 Nov 16 14:57:03.123607 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:03.123627 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" event=&{ID:062fe84e-408b-4970-94b8-5df0ae82ba87 Type:ContainerDied Data:85e47872a8d158098bc51ab2bf12cbd1e9121acbb0ee2d34c56ba5bf2fd2b685} Nov 16 14:57:03.124095 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:03.123672 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" event=&{ID:062fe84e-408b-4970-94b8-5df0ae82ba87 Type:ContainerStarted Data:c0c185f3a19a6a6d7f207a0845de00e3bb86afb4eaf336e2ec7a49ba9cde7f12} Nov 16 14:57:04.128605 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:04.128527236Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:82cbee841c604fb8f6b1cfeceb5aa0a216016d94b54d5d0635877c621f606372" id=7e8f9b60-6fdd-48df-8815-49e34af72405 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:04.129181 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:04.128839028Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:82cbee841c604fb8f6b1cfeceb5aa0a216016d94b54d5d0635877c621f606372 not found" id=7e8f9b60-6fdd-48df-8815-49e34af72405 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:04.129730 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:04.129661506Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:82cbee841c604fb8f6b1cfeceb5aa0a216016d94b54d5d0635877c621f606372" id=9311a482-e05d-4b52-a5cf-35f8e874fd74 name=/runtime.v1.ImageService/PullImage Nov 16 14:57:04.132787 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:04.132751324Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:82cbee841c604fb8f6b1cfeceb5aa0a216016d94b54d5d0635877c621f606372\"" Nov 16 14:57:04.433178 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:57:04.432964 1563 configmap.go:200] Couldn't get configMap openshift-multus/cni-binary-copy-script: configmap "cni-binary-copy-script" not found Nov 16 14:57:04.433178 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:57:04.433057 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy podName:13c954d2-615d-4b55-be8a-e712b8fa30a8 nodeName:}" failed. No retries permitted until 2022-11-16 14:57:36.433036617 +0000 UTC m=+11061.369812239 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cni-binary-copy" (UniqueName: "kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy") pod "multus-additional-cni-plugins-q722j" (UID: "13c954d2-615d-4b55-be8a-e712b8fa30a8") : configmap "cni-binary-copy-script" not found Nov 16 14:57:05.378873 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:05.378822454Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:82cbee841c604fb8f6b1cfeceb5aa0a216016d94b54d5d0635877c621f606372\"" Nov 16 14:57:09.080397 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.080308843Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:82cbee841c604fb8f6b1cfeceb5aa0a216016d94b54d5d0635877c621f606372" id=9311a482-e05d-4b52-a5cf-35f8e874fd74 name=/runtime.v1.ImageService/PullImage Nov 16 14:57:09.085881 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.085788387Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:82cbee841c604fb8f6b1cfeceb5aa0a216016d94b54d5d0635877c621f606372" id=930abe10-a0e0-42a5-a178-61d6d5d0b481 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:09.087532 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.087493501Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:153c8a8a29f2ad7f9a75856dc5a737d6420b1f43d8f28ec8323c5fcd2289ab69,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:82cbee841c604fb8f6b1cfeceb5aa0a216016d94b54d5d0635877c621f606372],Size_:425321598,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=930abe10-a0e0-42a5-a178-61d6d5d0b481 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:09.089062 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.088978313Z" level=info msg="Creating container: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/csi-driver" id=dfd3342a-2af7-46f2-9a10-59a9c136c151 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:57:09.089260 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.089176415Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:57:09.147969 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-f56d510a0e77e58bbea342ff69dff2620fe875a455bd2aa2f588a0e6fd5ef9e1.scope. Nov 16 14:57:09.177492 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container f56d510a0e77e58bbea342ff69dff2620fe875a455bd2aa2f588a0e6fd5ef9e1. Nov 16 14:57:09.247414 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.247325792Z" level=info msg="Created container f56d510a0e77e58bbea342ff69dff2620fe875a455bd2aa2f588a0e6fd5ef9e1: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/csi-driver" id=dfd3342a-2af7-46f2-9a10-59a9c136c151 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:57:09.248271 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.248234899Z" level=info msg="Starting container: f56d510a0e77e58bbea342ff69dff2620fe875a455bd2aa2f588a0e6fd5ef9e1" id=4732ed87-9e17-4071-8afb-36aa37c6b7ca name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:57:09.259862 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.259811193Z" level=info msg="Started container" PID=191456 containerID=f56d510a0e77e58bbea342ff69dff2620fe875a455bd2aa2f588a0e6fd5ef9e1 description=openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/csi-driver id=4732ed87-9e17-4071-8afb-36aa37c6b7ca name=/runtime.v1.RuntimeService/StartContainer sandboxID=c0c185f3a19a6a6d7f207a0845de00e3bb86afb4eaf336e2ec7a49ba9cde7f12 Nov 16 14:57:09.285410 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.285344599Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f430448d09b0fbb6b90291fa808dbf00e081eb36196b23a459eab69fbeca761" id=3c30e551-0c4a-4c40-bfea-cc8638362e20 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:09.285644 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.285613401Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:96126ea68fc9819028bfd7e1923225924fa5cdbe2aee3ad53fb86f04f8e27da2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f430448d09b0fbb6b90291fa808dbf00e081eb36196b23a459eab69fbeca761],Size_:350900510,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=3c30e551-0c4a-4c40-bfea-cc8638362e20 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:09.286660 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.286609309Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f430448d09b0fbb6b90291fa808dbf00e081eb36196b23a459eab69fbeca761" id=8247b663-b00b-4f65-87b5-79c0b0c56b2f name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:09.286893 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.286858911Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:96126ea68fc9819028bfd7e1923225924fa5cdbe2aee3ad53fb86f04f8e27da2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f430448d09b0fbb6b90291fa808dbf00e081eb36196b23a459eab69fbeca761],Size_:350900510,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=8247b663-b00b-4f65-87b5-79c0b0c56b2f name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:09.288045 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.287988720Z" level=info msg="Creating container: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/csi-node-driver-registrar" id=7938fdeb-1d17-43a4-8e11-23ce3ea54344 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:57:09.288176 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.288158721Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:57:09.354557 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-731d7b6a8257275a7776b1439b86dc22039ab0347d4d7064b6c42715da0321d9.scope. Nov 16 14:57:09.377249 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 731d7b6a8257275a7776b1439b86dc22039ab0347d4d7064b6c42715da0321d9. Nov 16 14:57:09.467999 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.467912273Z" level=info msg="Created container 731d7b6a8257275a7776b1439b86dc22039ab0347d4d7064b6c42715da0321d9: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/csi-node-driver-registrar" id=7938fdeb-1d17-43a4-8e11-23ce3ea54344 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:57:09.468529 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.468489678Z" level=info msg="Starting container: 731d7b6a8257275a7776b1439b86dc22039ab0347d4d7064b6c42715da0321d9" id=ce55cb19-33ed-4f8b-9697-e2bdb10c24e9 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:57:09.482267 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.482218989Z" level=info msg="Started container" PID=191504 containerID=731d7b6a8257275a7776b1439b86dc22039ab0347d4d7064b6c42715da0321d9 description=openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/csi-node-driver-registrar id=ce55cb19-33ed-4f8b-9697-e2bdb10c24e9 name=/runtime.v1.RuntimeService/StartContainer sandboxID=c0c185f3a19a6a6d7f207a0845de00e3bb86afb4eaf336e2ec7a49ba9cde7f12 Nov 16 14:57:09.492531 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:09.492431 1563 plugin_watcher.go:203] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/disk.csi.azure.com-reg.sock" Nov 16 14:57:09.503600 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.503555161Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6976cb9aa79c0ab33783342f2b5a6066cda4c177d0205c73b8f63e6a0641338e" id=ac078ff2-73e1-4fe9-89d0-28369b9156d1 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:09.503839 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.503802263Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:e00c8045676e7d065c67da533fd2ca0aff54a5dc7f41c8b88f6b051860907dad,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6976cb9aa79c0ab33783342f2b5a6066cda4c177d0205c73b8f63e6a0641338e],Size_:348932379,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=ac078ff2-73e1-4fe9-89d0-28369b9156d1 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:09.504952 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.504848971Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6976cb9aa79c0ab33783342f2b5a6066cda4c177d0205c73b8f63e6a0641338e" id=58796bde-fb59-48e2-a6cc-69cc5449eafc name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:09.505110 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.505079373Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:e00c8045676e7d065c67da533fd2ca0aff54a5dc7f41c8b88f6b051860907dad,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6976cb9aa79c0ab33783342f2b5a6066cda4c177d0205c73b8f63e6a0641338e],Size_:348932379,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=58796bde-fb59-48e2-a6cc-69cc5449eafc name=/runtime.v1.ImageService/ImageStatus Nov 16 14:57:09.506086 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.505970980Z" level=info msg="Creating container: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/csi-liveness-probe" id=7e0edfdc-6632-4309-ae96-5d6d81969a31 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:57:09.506193 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.506140882Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:57:09.560064 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-b0c90c7d29fe0493a276cfffb8510ad2e1a938ead0657a8666e87faa05bbad0d.scope. Nov 16 14:57:09.582574 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container b0c90c7d29fe0493a276cfffb8510ad2e1a938ead0657a8666e87faa05bbad0d. Nov 16 14:57:09.620054 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:57:09.683373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.683286712Z" level=info msg="Created container b0c90c7d29fe0493a276cfffb8510ad2e1a938ead0657a8666e87faa05bbad0d: openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/csi-liveness-probe" id=7e0edfdc-6632-4309-ae96-5d6d81969a31 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:57:09.684102 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.684045918Z" level=info msg="Starting container: b0c90c7d29fe0493a276cfffb8510ad2e1a938ead0657a8666e87faa05bbad0d" id=c12a85d9-be7e-4e2f-b102-f38163973c21 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:57:09.694980 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:09.694938806Z" level=info msg="Started container" PID=191573 containerID=b0c90c7d29fe0493a276cfffb8510ad2e1a938ead0657a8666e87faa05bbad0d description=openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6/csi-liveness-probe id=c12a85d9-be7e-4e2f-b102-f38163973c21 name=/runtime.v1.RuntimeService/StartContainer sandboxID=c0c185f3a19a6a6d7f207a0845de00e3bb86afb4eaf336e2ec7a49ba9cde7f12 Nov 16 14:57:10.155412 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:10.155315 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" event=&{ID:062fe84e-408b-4970-94b8-5df0ae82ba87 Type:ContainerStarted Data:b0c90c7d29fe0493a276cfffb8510ad2e1a938ead0657a8666e87faa05bbad0d} Nov 16 14:57:10.155412 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:10.155398 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" event=&{ID:062fe84e-408b-4970-94b8-5df0ae82ba87 Type:ContainerStarted Data:731d7b6a8257275a7776b1439b86dc22039ab0347d4d7064b6c42715da0321d9} Nov 16 14:57:10.155412 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:10.155416 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/azure-disk-csi-driver-node-p82p6" event=&{ID:062fe84e-408b-4970-94b8-5df0ae82ba87 Type:ContainerStarted Data:f56d510a0e77e58bbea342ff69dff2620fe875a455bd2aa2f588a0e6fd5ef9e1} Nov 16 14:57:10.217985 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:10.217854 1563 reconciler.go:160] "OperationExecutor.RegisterPlugin started" plugin={SocketPath:/var/lib/kubelet/plugins_registry/disk.csi.azure.com-reg.sock Timestamp:2022-11-16 14:57:09.492464271 +0000 UTC m=+11034.429239893 Handler: Name:} Nov 16 14:57:10.219595 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:10.219529 1563 csi_plugin.go:99] kubernetes.io/csi: Trying to validate a new CSI Driver with name: disk.csi.azure.com endpoint: /var/lib/kubelet/plugins/disk.csi.azure.com/csi.sock versions: 1.0.0 Nov 16 14:57:10.219595 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:57:10.219603 1563 csi_plugin.go:112] kubernetes.io/csi: Register new plugin with name: disk.csi.azure.com at endpoint: /var/lib/kubelet/plugins/disk.csi.azure.com/csi.sock Nov 16 14:57:23.613010 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:23.612943735Z" level=info msg="Stopping pod sandbox: 8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883" id=da455731-e28b-442a-9182-dfd49372172f name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:57:23.613010 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:23.613009435Z" level=info msg="Stopped pod sandbox (already stopped): 8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883" id=da455731-e28b-442a-9182-dfd49372172f name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:57:23.613517 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:23.613252137Z" level=info msg="Removing pod sandbox: 8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883" id=09e38daf-3c61-4691-99a6-b61a966bf378 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:57:23.764730 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:57:23.764665658Z" level=info msg="Removed pod sandbox: 8c5b7998b74e8fc3ed0c30dca33dad3ada675f77c50285df394aea21f73f9883" id=09e38daf-3c61-4691-99a6-b61a966bf378 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:57:36.457773 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:57:36.457691 1563 configmap.go:200] Couldn't get configMap openshift-multus/cni-binary-copy-script: configmap "cni-binary-copy-script" not found Nov 16 14:57:36.457773 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:57:36.457791 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy podName:13c954d2-615d-4b55-be8a-e712b8fa30a8 nodeName:}" failed. No retries permitted until 2022-11-16 14:58:40.457770663 +0000 UTC m=+11125.394546285 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cni-binary-copy" (UniqueName: "kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy") pod "multus-additional-cni-plugins-q722j" (UID: "13c954d2-615d-4b55-be8a-e712b8fa30a8") : configmap "cni-binary-copy-script" not found Nov 16 14:57:38.889661 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.4UR6KF.mount: Succeeded. Nov 16 14:57:38.889710 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.1aX3ly.mount: Succeeded. Nov 16 14:57:44.471184 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b-runc.B0PuyH.mount: Succeeded. Nov 16 14:58:05.169917 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:05.169824 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-multus/multus-additional-cni-plugins-q722j] Nov 16 14:58:05.170453 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:05.170255 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-multus/multus-additional-cni-plugins-q722j" podUID=13c954d2-615d-4b55-be8a-e712b8fa30a8 containerName="kube-multus-additional-cni-plugins" containerID="cri-o://c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e" gracePeriod=30 Nov 16 14:58:05.170691 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:05.170639168Z" level=info msg="Stopping container: c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e (timeout: 30s)" id=73d39c79-496b-4cba-bf71-04d847d0a45e name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:58:05.192103 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e.scope: Succeeded. Nov 16 14:58:05.192890 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e.scope: Consumed 34ms CPU time Nov 16 14:58:05.194989 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e.scope: Succeeded. Nov 16 14:58:05.195652 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e.scope: Consumed 33ms CPU time Nov 16 14:58:05.331901 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-b7a2b6a4aa55551cf038e323676ea9527628402a9f7fe663ff656054e27bcaa9-merged.mount: Succeeded. Nov 16 14:58:05.332766 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-b7a2b6a4aa55551cf038e323676ea9527628402a9f7fe663ff656054e27bcaa9-merged.mount: Consumed 0 CPU time Nov 16 14:58:05.340212 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:05.340183 1563 generic.go:296] "Generic (PLEG): container finished" podID=13c954d2-615d-4b55-be8a-e712b8fa30a8 containerID="c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e" exitCode=143 Nov 16 14:58:05.340361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:05.340235 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q722j" event=&{ID:13c954d2-615d-4b55-be8a-e712b8fa30a8 Type:ContainerDied Data:c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e} Nov 16 14:58:05.694669 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:05.694605201Z" level=info msg="Stopped container c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e: openshift-multus/multus-additional-cni-plugins-q722j/kube-multus-additional-cni-plugins" id=73d39c79-496b-4cba-bf71-04d847d0a45e name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:58:05.695688 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:05.695595710Z" level=info msg="Stopping pod sandbox: 8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3" id=4bbda279-2fb0-448b-8030-d11c63fe0d8a name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:58:05.705382 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-30cac234\x2d6a80\x2d40a0\x2d9562\x2d95fc50b00e8c.mount: Succeeded. Nov 16 14:58:05.706276 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-30cac234\x2d6a80\x2d40a0\x2d9562\x2d95fc50b00e8c.mount: Consumed 0 CPU time Nov 16 14:58:05.724996 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-30cac234\x2d6a80\x2d40a0\x2d9562\x2d95fc50b00e8c.mount: Succeeded. Nov 16 14:58:05.725874 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-30cac234\x2d6a80\x2d40a0\x2d9562\x2d95fc50b00e8c.mount: Consumed 0 CPU time Nov 16 14:58:05.751014 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-30cac234\x2d6a80\x2d40a0\x2d9562\x2d95fc50b00e8c.mount: Succeeded. Nov 16 14:58:05.751876 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-30cac234\x2d6a80\x2d40a0\x2d9562\x2d95fc50b00e8c.mount: Consumed 0 CPU time Nov 16 14:58:05.773913 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3-userdata-shm.mount: Succeeded. Nov 16 14:58:05.775366 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3-userdata-shm.mount: Consumed 0 CPU time Nov 16 14:58:05.783952 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-aa41ec2ae9aaefb1cbf126992ec7345fb7b810d588f0e70beb925bc0dc101560-merged.mount: Succeeded. Nov 16 14:58:05.784786 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-aa41ec2ae9aaefb1cbf126992ec7345fb7b810d588f0e70beb925bc0dc101560-merged.mount: Consumed 0 CPU time Nov 16 14:58:05.916294 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:05.916233234Z" level=info msg="Stopped pod sandbox: 8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3" id=4bbda279-2fb0-448b-8030-d11c63fe0d8a name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:58:06.063959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.063700 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-system-cni-dir\") pod \"13c954d2-615d-4b55-be8a-e712b8fa30a8\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " Nov 16 14:58:06.063959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.063782 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm8zx\" (UniqueName: \"kubernetes.io/projected/13c954d2-615d-4b55-be8a-e712b8fa30a8-kube-api-access-jm8zx\") pod \"13c954d2-615d-4b55-be8a-e712b8fa30a8\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " Nov 16 14:58:06.063959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.063836 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-system-cni-dir" (OuterVolumeSpecName: "system-cni-dir") pod "13c954d2-615d-4b55-be8a-e712b8fa30a8" (UID: "13c954d2-615d-4b55-be8a-e712b8fa30a8"). InnerVolumeSpecName "system-cni-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:58:06.063959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.063853 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-cnibin\") pod \"13c954d2-615d-4b55-be8a-e712b8fa30a8\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " Nov 16 14:58:06.063959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.063916 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-cnibin" (OuterVolumeSpecName: "cnibin") pod "13c954d2-615d-4b55-be8a-e712b8fa30a8" (UID: "13c954d2-615d-4b55-be8a-e712b8fa30a8"). InnerVolumeSpecName "cnibin". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:58:06.064372 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.063965 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy\") pod \"13c954d2-615d-4b55-be8a-e712b8fa30a8\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " Nov 16 14:58:06.064372 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.064001 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-os-release\") pod \"13c954d2-615d-4b55-be8a-e712b8fa30a8\" (UID: \"13c954d2-615d-4b55-be8a-e712b8fa30a8\") " Nov 16 14:58:06.064372 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.064221 1563 reconciler.go:300] "Volume detached for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-cnibin\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:58:06.064372 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.064243 1563 reconciler.go:300] "Volume detached for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-system-cni-dir\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:58:06.064372 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:58:06.064241 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/13c954d2-615d-4b55-be8a-e712b8fa30a8/volumes/kubernetes.io~configmap/cni-binary-copy: clearQuota called, but quotas disabled Nov 16 14:58:06.064372 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.064263 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-os-release" (OuterVolumeSpecName: "os-release") pod "13c954d2-615d-4b55-be8a-e712b8fa30a8" (UID: "13c954d2-615d-4b55-be8a-e712b8fa30a8"). InnerVolumeSpecName "os-release". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 14:58:06.064545 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.064436 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy" (OuterVolumeSpecName: "cni-binary-copy") pod "13c954d2-615d-4b55-be8a-e712b8fa30a8" (UID: "13c954d2-615d-4b55-be8a-e712b8fa30a8"). InnerVolumeSpecName "cni-binary-copy". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 14:58:06.084434 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.084348 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13c954d2-615d-4b55-be8a-e712b8fa30a8-kube-api-access-jm8zx" (OuterVolumeSpecName: "kube-api-access-jm8zx") pod "13c954d2-615d-4b55-be8a-e712b8fa30a8" (UID: "13c954d2-615d-4b55-be8a-e712b8fa30a8"). InnerVolumeSpecName "kube-api-access-jm8zx". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:58:06.166662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.166555 1563 reconciler.go:300] "Volume detached for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13c954d2-615d-4b55-be8a-e712b8fa30a8-cni-binary-copy\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:58:06.166662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.166612 1563 reconciler.go:300] "Volume detached for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13c954d2-615d-4b55-be8a-e712b8fa30a8-os-release\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:58:06.166662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.166641 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-jm8zx\" (UniqueName: \"kubernetes.io/projected/13c954d2-615d-4b55-be8a-e712b8fa30a8-kube-api-access-jm8zx\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:58:06.347692 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.347579 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q722j" event=&{ID:13c954d2-615d-4b55-be8a-e712b8fa30a8 Type:ContainerDied Data:8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3} Nov 16 14:58:06.347692 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.347640 1563 scope.go:110] "RemoveContainer" containerID="c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e" Nov 16 14:58:06.349974 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:06.349922621Z" level=info msg="Removing container: c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e" id=68ef9603-f460-4889-8da3-a224e3f1c6e2 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:06.361845 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod13c954d2_615d_4b55_be8a_e712b8fa30a8.slice. Nov 16 14:58:06.361929 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod13c954d2_615d_4b55_be8a_e712b8fa30a8.slice: Consumed 593ms CPU time Nov 16 14:58:06.385020 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:06.384932811Z" level=info msg="Removed container c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e: openshift-multus/multus-additional-cni-plugins-q722j/kube-multus-additional-cni-plugins" id=68ef9603-f460-4889-8da3-a224e3f1c6e2 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:06.385276 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.385247 1563 scope.go:110] "RemoveContainer" containerID="bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554" Nov 16 14:58:06.386455 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:06.386419023Z" level=info msg="Removing container: bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554" id=f822bbe9-deb5-43b5-8861-7b34a6f50887 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:06.393330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.393288 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-multus/multus-additional-cni-plugins-q722j] Nov 16 14:58:06.407987 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.407908 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-multus/multus-additional-cni-plugins-q722j] Nov 16 14:58:06.440465 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.440422 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-multus/multus-additional-cni-plugins-vw2s6] Nov 16 14:58:06.440674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.440495 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:58:06.440674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:58:06.440592 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="bond-cni-plugin" Nov 16 14:58:06.440674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.440606 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="bond-cni-plugin" Nov 16 14:58:06.440674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:58:06.440620 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="whereabouts-cni" Nov 16 14:58:06.440674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.440628 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="whereabouts-cni" Nov 16 14:58:06.440674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:58:06.440638 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="whereabouts-cni-bincopy" Nov 16 14:58:06.440674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.440647 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="whereabouts-cni-bincopy" Nov 16 14:58:06.440674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:58:06.440660 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="kube-multus-additional-cni-plugins" Nov 16 14:58:06.440674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.440668 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="kube-multus-additional-cni-plugins" Nov 16 14:58:06.440674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:58:06.440679 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="egress-router-binary-copy" Nov 16 14:58:06.440972 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.440687 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="egress-router-binary-copy" Nov 16 14:58:06.440972 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:58:06.440698 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="cni-plugins" Nov 16 14:58:06.440972 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.440706 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="cni-plugins" Nov 16 14:58:06.440972 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:58:06.440727 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="routeoverride-cni" Nov 16 14:58:06.440972 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.440735 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="routeoverride-cni" Nov 16 14:58:06.440972 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.440807 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="13c954d2-615d-4b55-be8a-e712b8fa30a8" containerName="kube-multus-additional-cni-plugins" Nov 16 14:58:06.451723 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod57197822_83e2_43fe_a3d3_1a63580e3155.slice. Nov 16 14:58:06.571026 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.570966 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/57197822-83e2-43fe-a3d3-1a63580e3155-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.571026 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.571038 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmsx7\" (UniqueName: \"kubernetes.io/projected/57197822-83e2-43fe-a3d3-1a63580e3155-kube-api-access-dmsx7\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.571386 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.571204 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/57197822-83e2-43fe-a3d3-1a63580e3155-cnibin\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.571386 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.571328 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57197822-83e2-43fe-a3d3-1a63580e3155-system-cni-dir\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.571469 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.571393 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/57197822-83e2-43fe-a3d3-1a63580e3155-os-release\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.571469 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.571437 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/57197822-83e2-43fe-a3d3-1a63580e3155-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.592814 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.592781 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-multus/multus-additional-cni-plugins-q722j" podUID=13c954d2-615d-4b55-be8a-e712b8fa30a8 containerName="kube-multus-additional-cni-plugins" containerID="cri-o://c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e" gracePeriod=1 Nov 16 14:58:06.593075 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:06.593023432Z" level=info msg="Stopping container: c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e (timeout: 1s)" id=ebc990bd-a13a-4dda-8783-9b1431926a1c name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:58:06.593336 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:58:06.593297 1563 remote_runtime.go:479] "StopContainer from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e\": container with ID starting with c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e not found: ID does not exist" containerID="c597d6091ee83f740d4c05569fe6da50bcc0ffd4eac4f889f1aafe27e4be119e" Nov 16 14:58:06.593557 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:06.593518636Z" level=info msg="Stopping pod sandbox: 8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3" id=1d28a8cd-9a41-40f1-8b01-e881bf1eeffe name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:58:06.593624 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:06.593561936Z" level=info msg="Stopped pod sandbox (already stopped): 8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3" id=1d28a8cd-9a41-40f1-8b01-e881bf1eeffe name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:58:06.595255 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.595224 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=13c954d2-615d-4b55-be8a-e712b8fa30a8 path="/var/lib/kubelet/pods/13c954d2-615d-4b55-be8a-e712b8fa30a8/volumes" Nov 16 14:58:06.605022 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:06.604875630Z" level=info msg="Removed container bf338362250c2879c8f4aa563b64992a9791c4114524451de642412e7cb19554: openshift-multus/multus-additional-cni-plugins-q722j/whereabouts-cni" id=f822bbe9-deb5-43b5-8861-7b34a6f50887 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:06.605120 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.605096 1563 scope.go:110] "RemoveContainer" containerID="ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635" Nov 16 14:58:06.606158 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:06.606103040Z" level=info msg="Removing container: ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635" id=ad2b0d65-5696-4474-b1ae-56ce721d5e0d name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:06.672148 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.672046 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/57197822-83e2-43fe-a3d3-1a63580e3155-os-release\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.672148 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.672157 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/57197822-83e2-43fe-a3d3-1a63580e3155-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.672447 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.672211 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/57197822-83e2-43fe-a3d3-1a63580e3155-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.672447 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.672209 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/57197822-83e2-43fe-a3d3-1a63580e3155-os-release\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.672447 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.672248 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-dmsx7\" (UniqueName: \"kubernetes.io/projected/57197822-83e2-43fe-a3d3-1a63580e3155-kube-api-access-dmsx7\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.672447 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.672285 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/57197822-83e2-43fe-a3d3-1a63580e3155-cnibin\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.672447 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.672334 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57197822-83e2-43fe-a3d3-1a63580e3155-system-cni-dir\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.672447 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.672374 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/57197822-83e2-43fe-a3d3-1a63580e3155-cnibin\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.672447 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.672410 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/57197822-83e2-43fe-a3d3-1a63580e3155-system-cni-dir\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.672447 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.672443 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/57197822-83e2-43fe-a3d3-1a63580e3155-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.672878 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.672850 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/57197822-83e2-43fe-a3d3-1a63580e3155-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.699520 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.699439 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmsx7\" (UniqueName: \"kubernetes.io/projected/57197822-83e2-43fe-a3d3-1a63580e3155-kube-api-access-dmsx7\") pod \"multus-additional-cni-plugins-vw2s6\" (UID: \"57197822-83e2-43fe-a3d3-1a63580e3155\") " pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.705317 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-dddef48eab46e0ed01e5dfd2dfa8cdefc64dcf3bfefda444531a59c6f8151fd2-merged.mount: Succeeded. Nov 16 14:58:06.706351 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-dddef48eab46e0ed01e5dfd2dfa8cdefc64dcf3bfefda444531a59c6f8151fd2-merged.mount: Consumed 0 CPU time Nov 16 14:58:06.706371 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-f7dcbc3a92ec4883447246f2732b25776df4fac9c354dd792532a910b0c9b88b-merged.mount: Succeeded. Nov 16 14:58:06.707078 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-f7dcbc3a92ec4883447246f2732b25776df4fac9c354dd792532a910b0c9b88b-merged.mount: Consumed 0 CPU time Nov 16 14:58:06.707102 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-13c954d2\x2d615d\x2d4b55\x2dbe8a\x2de712b8fa30a8-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2djm8zx.mount: Succeeded. Nov 16 14:58:06.707785 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-13c954d2\x2d615d\x2d4b55\x2dbe8a\x2de712b8fa30a8-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2djm8zx.mount: Consumed 0 CPU time Nov 16 14:58:06.763657 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.763552 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vw2s6" Nov 16 14:58:06.764505 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:06.764374149Z" level=info msg="Running pod sandbox: openshift-multus/multus-additional-cni-plugins-vw2s6/POD" id=1cb23724-5431-4dde-acc8-f7654b98705f name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:58:06.764505 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:06.764486850Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:58:06.795750 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:06.795659308Z" level=info msg="Removed container ca854f85e0dfa5823bdaed0cc1d2218baad9854bf996af397d24e743c6513635: openshift-multus/multus-additional-cni-plugins-q722j/whereabouts-cni-bincopy" id=ad2b0d65-5696-4474-b1ae-56ce721d5e0d name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:06.796076 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:06.796036 1563 scope.go:110] "RemoveContainer" containerID="ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6" Nov 16 14:58:06.797197 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:06.797159820Z" level=info msg="Removing container: ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6" id=7183a76c-7a95-4c76-8b41-605b36d564d6 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:06.849917 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-3c4a61d64d3f36ecf62f5452681a779c10b89f2e9472af2ef8baee13a42b5ca2-merged.mount: Succeeded. Nov 16 14:58:06.850767 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-3c4a61d64d3f36ecf62f5452681a779c10b89f2e9472af2ef8baee13a42b5ca2-merged.mount: Consumed 0 CPU time Nov 16 14:58:07.043359 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:07.043207617Z" level=info msg="Removed container ea85273ec908acf703a7a644979ad8670b5b5dbd7a03936f98ca27e2aec657c6: openshift-multus/multus-additional-cni-plugins-q722j/routeoverride-cni" id=7183a76c-7a95-4c76-8b41-605b36d564d6 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:07.043599 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:07.043489 1563 scope.go:110] "RemoveContainer" containerID="4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6" Nov 16 14:58:07.044549 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:07.044512326Z" level=info msg="Removing container: 4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6" id=b75c2c90-976c-4525-a197-443a2b00d306 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:07.064384 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:07.064336071Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=1cb23724-5431-4dde-acc8-f7654b98705f name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:58:07.078921 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-448e9c49f4de790f92a9be3fdb5b6a3f004f8db19ff53ccdd5b475e477addf40-merged.mount: Succeeded. Nov 16 14:58:07.079683 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-448e9c49f4de790f92a9be3fdb5b6a3f004f8db19ff53ccdd5b475e477addf40-merged.mount: Consumed 0 CPU time Nov 16 14:58:07.195732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:07.195649627Z" level=info msg="Removed container 4b46499d72631b2111dcbe20b33593eb588bc28d9f667c634978c83fa554d1e6: openshift-multus/multus-additional-cni-plugins-q722j/bond-cni-plugin" id=b75c2c90-976c-4525-a197-443a2b00d306 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:07.196151 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:07.196084 1563 scope.go:110] "RemoveContainer" containerID="7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632" Nov 16 14:58:07.197295 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:07.197257739Z" level=info msg="Removing container: 7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632" id=65efd5b7-3028-466a-bd16-14c5f4110fa5 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:07.199437 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:58:07.199400 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57197822_83e2_43fe_a3d3_1a63580e3155.slice/crio-99dbbd3c144110014f41261f1b3b4a57124d397e3ff49838d099ed63cff0cb80.scope WatchSource:0}: Error finding container 99dbbd3c144110014f41261f1b3b4a57124d397e3ff49838d099ed63cff0cb80: Status 404 returned error &{%!s(*http.body=&{0xc0010b8858 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:58:07.211236 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:07.211180440Z" level=info msg="Ran pod sandbox 99dbbd3c144110014f41261f1b3b4a57124d397e3ff49838d099ed63cff0cb80 with infra container: openshift-multus/multus-additional-cni-plugins-vw2s6/POD" id=1cb23724-5431-4dde-acc8-f7654b98705f name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:58:07.212417 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:07.212380349Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8adaa53f43c5257d1d4a02f2b880bd63430908f2286bc97f323f42a7cf9166bd" id=22d74da1-511f-4bd5-96a7-65be1139891d name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:07.212836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:07.212783452Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8adaa53f43c5257d1d4a02f2b880bd63430908f2286bc97f323f42a7cf9166bd not found" id=22d74da1-511f-4bd5-96a7-65be1139891d name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:07.217041 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:07.213503857Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8adaa53f43c5257d1d4a02f2b880bd63430908f2286bc97f323f42a7cf9166bd" id=dc673956-a36d-4443-a54d-fd1098539e1e name=/runtime.v1.ImageService/PullImage Nov 16 14:58:07.327422 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:07.327264786Z" level=info msg="Removed container 7b7523cd5e6cc14adaf1cba7124fb2639ecb206ddab1a9e2ee183db679926632: openshift-multus/multus-additional-cni-plugins-q722j/cni-plugins" id=65efd5b7-3028-466a-bd16-14c5f4110fa5 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:07.327654 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:07.327574 1563 scope.go:110] "RemoveContainer" containerID="4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470" Nov 16 14:58:07.328700 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:07.328662496Z" level=info msg="Removing container: 4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470" id=acf27bb0-d885-46dd-ade6-c2676b044f3f name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:07.331160 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:07.331102614Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8adaa53f43c5257d1d4a02f2b880bd63430908f2286bc97f323f42a7cf9166bd\"" Nov 16 14:58:07.352523 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:07.352444 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2s6" event=&{ID:57197822-83e2-43fe-a3d3-1a63580e3155 Type:ContainerStarted Data:99dbbd3c144110014f41261f1b3b4a57124d397e3ff49838d099ed63cff0cb80} Nov 16 14:58:07.428001 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:07.427911119Z" level=info msg="Removed container 4c72318e52ae411bdd2b98e5f3cfe3e19e2478620c571dc9618699f64969b470: openshift-multus/multus-additional-cni-plugins-q722j/egress-router-binary-copy" id=acf27bb0-d885-46dd-ade6-c2676b044f3f name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:58:07.708162 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-5f24b11d200a0e5782e01a1fa22205bb2ddd31b22365ef54b9ab36a5c77fb4c9-merged.mount: Succeeded. Nov 16 14:58:07.709087 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-5f24b11d200a0e5782e01a1fa22205bb2ddd31b22365ef54b9ab36a5c77fb4c9-merged.mount: Consumed 0 CPU time Nov 16 14:58:07.709106 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-13e4fbd5444db91ac3560334cbe16888f2dd14f25ffaae7625c694172ef8d4e2-merged.mount: Succeeded. Nov 16 14:58:07.709784 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-13e4fbd5444db91ac3560334cbe16888f2dd14f25ffaae7625c694172ef8d4e2-merged.mount: Consumed 0 CPU time Nov 16 14:58:08.583668 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:08.583570738Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8adaa53f43c5257d1d4a02f2b880bd63430908f2286bc97f323f42a7cf9166bd\"" Nov 16 14:58:12.322991 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.322874978Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8adaa53f43c5257d1d4a02f2b880bd63430908f2286bc97f323f42a7cf9166bd" id=dc673956-a36d-4443-a54d-fd1098539e1e name=/runtime.v1.ImageService/PullImage Nov 16 14:58:12.324313 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.324225388Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8adaa53f43c5257d1d4a02f2b880bd63430908f2286bc97f323f42a7cf9166bd" id=2131bbbe-97a3-47f2-aad7-cdabfeb93e0a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:12.326142 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.326041301Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9bd34ae7bd272a38f52b6bec74d092104a47bda982b4e2c3a4dce3579113402d,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8adaa53f43c5257d1d4a02f2b880bd63430908f2286bc97f323f42a7cf9166bd],Size_:424663494,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=2131bbbe-97a3-47f2-aad7-cdabfeb93e0a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:12.327161 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.327060709Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-vw2s6/egress-router-binary-copy" id=4548cf43-c986-456d-ac06-44b366a28537 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:12.327252 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.327203810Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:58:12.399467 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-da9c4b637190ad8fd06805def33cdfc23029626686a3a8b426d6e9cb333ed7a0.scope. Nov 16 14:58:12.432256 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container da9c4b637190ad8fd06805def33cdfc23029626686a3a8b426d6e9cb333ed7a0. Nov 16 14:58:12.472367 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:58:12.533334 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.533267811Z" level=info msg="Created container da9c4b637190ad8fd06805def33cdfc23029626686a3a8b426d6e9cb333ed7a0: openshift-multus/multus-additional-cni-plugins-vw2s6/egress-router-binary-copy" id=4548cf43-c986-456d-ac06-44b366a28537 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:12.533981 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.533945216Z" level=info msg="Starting container: da9c4b637190ad8fd06805def33cdfc23029626686a3a8b426d6e9cb333ed7a0" id=43908db9-fc43-49eb-8d86-07cf5590a741 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:58:12.547075 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.547007711Z" level=info msg="Started container" PID=193126 containerID=da9c4b637190ad8fd06805def33cdfc23029626686a3a8b426d6e9cb333ed7a0 description=openshift-multus/multus-additional-cni-plugins-vw2s6/egress-router-binary-copy id=43908db9-fc43-49eb-8d86-07cf5590a741 name=/runtime.v1.RuntimeService/StartContainer sandboxID=99dbbd3c144110014f41261f1b3b4a57124d397e3ff49838d099ed63cff0cb80 Nov 16 14:58:12.554281 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.554212363Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/upgrade_2b5c60c9-ba51-4fb0-af34-7331c24f9511\": CREATE" Nov 16 14:58:12.566931 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.566869755Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:12.566931 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.566921356Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:12.581898 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.581717064Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/egress-router\": CREATE" Nov 16 14:58:12.595105 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.595028961Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:12.595105 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.595062961Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:12.595105 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:12.595080361Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/upgrade_2b5c60c9-ba51-4fb0-af34-7331c24f9511\": REMOVE" Nov 16 14:58:12.606148 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-da9c4b637190ad8fd06805def33cdfc23029626686a3a8b426d6e9cb333ed7a0.scope: Succeeded. Nov 16 14:58:12.606806 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-da9c4b637190ad8fd06805def33cdfc23029626686a3a8b426d6e9cb333ed7a0.scope: Consumed 69ms CPU time Nov 16 14:58:12.615842 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-da9c4b637190ad8fd06805def33cdfc23029626686a3a8b426d6e9cb333ed7a0.scope: Succeeded. Nov 16 14:58:12.617084 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-da9c4b637190ad8fd06805def33cdfc23029626686a3a8b426d6e9cb333ed7a0.scope: Consumed 44ms CPU time Nov 16 14:58:13.374256 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:13.374213 1563 generic.go:296] "Generic (PLEG): container finished" podID=57197822-83e2-43fe-a3d3-1a63580e3155 containerID="da9c4b637190ad8fd06805def33cdfc23029626686a3a8b426d6e9cb333ed7a0" exitCode=0 Nov 16 14:58:13.374256 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:13.374274 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2s6" event=&{ID:57197822-83e2-43fe-a3d3-1a63580e3155 Type:ContainerDied Data:da9c4b637190ad8fd06805def33cdfc23029626686a3a8b426d6e9cb333ed7a0} Nov 16 14:58:13.375253 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:13.375199044Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5dae540d3f6e8c54e384ecec150e702b2ed868301066463c8710501b88b125a8" id=e1f96239-4fa9-4b3b-934d-33f3e77b3784 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:13.375547 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:13.375485846Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5dae540d3f6e8c54e384ecec150e702b2ed868301066463c8710501b88b125a8 not found" id=e1f96239-4fa9-4b3b-934d-33f3e77b3784 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:13.376373 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:13.376331052Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5dae540d3f6e8c54e384ecec150e702b2ed868301066463c8710501b88b125a8" id=2bd58aa4-bcc8-443d-9390-51f9aa246840 name=/runtime.v1.ImageService/PullImage Nov 16 14:58:13.379375 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:13.379342574Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5dae540d3f6e8c54e384ecec150e702b2ed868301066463c8710501b88b125a8\"" Nov 16 14:58:14.642020 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:14.641913272Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5dae540d3f6e8c54e384ecec150e702b2ed868301066463c8710501b88b125a8\"" Nov 16 14:58:20.496920 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:20.496748211Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=4108789a-32f0-4aa8-955f-3e3b143f877a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:20.718772 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:20.497026613Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=4108789a-32f0-4aa8-955f-3e3b143f877a name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:20.981378 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:20.981276485Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5dae540d3f6e8c54e384ecec150e702b2ed868301066463c8710501b88b125a8" id=2bd58aa4-bcc8-443d-9390-51f9aa246840 name=/runtime.v1.ImageService/PullImage Nov 16 14:58:20.988652 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:20.988608340Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5dae540d3f6e8c54e384ecec150e702b2ed868301066463c8710501b88b125a8" id=522f4a74-8eb4-465e-b8af-bf543a8efe61 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:20.990251 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:20.990206352Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:1f7988a3b58f5d9ce404b3aa94826059e635b0084864c20687964016ae7042d9,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5dae540d3f6e8c54e384ecec150e702b2ed868301066463c8710501b88b125a8],Size_:553037419,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=522f4a74-8eb4-465e-b8af-bf543a8efe61 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:20.991294 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:20.991256360Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-vw2s6/cni-plugins" id=b6f55a9f-bc85-4f85-b92d-23c988825395 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:20.991388 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:20.991363061Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:58:21.055869 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-01092b85c12244521e5fff43667530b60f56e605201a51ef24e9e5afe133d000.scope. Nov 16 14:58:21.093010 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 01092b85c12244521e5fff43667530b60f56e605201a51ef24e9e5afe133d000. Nov 16 14:58:21.128321 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:58:21.210828 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.210719324Z" level=info msg="Created container 01092b85c12244521e5fff43667530b60f56e605201a51ef24e9e5afe133d000: openshift-multus/multus-additional-cni-plugins-vw2s6/cni-plugins" id=b6f55a9f-bc85-4f85-b92d-23c988825395 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:21.211478 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.211388429Z" level=info msg="Starting container: 01092b85c12244521e5fff43667530b60f56e605201a51ef24e9e5afe133d000" id=02f251d4-4ebd-4cfd-a066-98defdf7ff17 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:58:21.226054 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.225964640Z" level=info msg="Started container" PID=193398 containerID=01092b85c12244521e5fff43667530b60f56e605201a51ef24e9e5afe133d000 description=openshift-multus/multus-additional-cni-plugins-vw2s6/cni-plugins id=02f251d4-4ebd-4cfd-a066-98defdf7ff17 name=/runtime.v1.RuntimeService/StartContainer sandboxID=99dbbd3c144110014f41261f1b3b4a57124d397e3ff49838d099ed63cff0cb80 Nov 16 14:58:21.236505 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.236357519Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/upgrade_0770e5dd-9fe2-4595-835f-b45938ce843e\": CREATE" Nov 16 14:58:21.250116 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.250039922Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.250116 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.250089923Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.289517 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.289407121Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/bandwidth\": CREATE" Nov 16 14:58:21.302339 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.302287219Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.302339 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.302330319Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.302339 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.302348319Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/bridge\": CREATE" Nov 16 14:58:21.316877 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-01092b85c12244521e5fff43667530b60f56e605201a51ef24e9e5afe133d000.scope: Succeeded. Nov 16 14:58:21.317600 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-01092b85c12244521e5fff43667530b60f56e605201a51ef24e9e5afe133d000.scope: Consumed 105ms CPU time Nov 16 14:58:21.319190 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.319143746Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.319190 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.319182747Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.319325 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.319208847Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/dhcp\": CREATE" Nov 16 14:58:21.326655 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-01092b85c12244521e5fff43667530b60f56e605201a51ef24e9e5afe133d000.scope: Succeeded. Nov 16 14:58:21.327309 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-01092b85c12244521e5fff43667530b60f56e605201a51ef24e9e5afe133d000.scope: Consumed 49ms CPU time Nov 16 14:58:21.332571 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.332525248Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.332571 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.332569948Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.332729 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.332587548Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/firewall\": CREATE" Nov 16 14:58:21.346136 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.346030550Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.346136 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.346088351Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.346136 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.346111851Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/host-device\": CREATE" Nov 16 14:58:21.357814 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.357755639Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.357814 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.357793239Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.357814 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.357809440Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/host-local\": CREATE" Nov 16 14:58:21.371664 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.371550144Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.371664 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.371610544Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.371664 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.371628544Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/ipvlan\": CREATE" Nov 16 14:58:21.385980 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.385918353Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.385980 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.385968553Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.385980 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.385992953Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/loopback\": CREATE" Nov 16 14:58:21.400116 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.400057160Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.400116 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.400097660Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.400116 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.400115160Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/macvlan\": CREATE" Nov 16 14:58:21.400852 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:21.400746 1563 generic.go:296] "Generic (PLEG): container finished" podID=57197822-83e2-43fe-a3d3-1a63580e3155 containerID="01092b85c12244521e5fff43667530b60f56e605201a51ef24e9e5afe133d000" exitCode=0 Nov 16 14:58:21.400852 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:21.400802 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2s6" event=&{ID:57197822-83e2-43fe-a3d3-1a63580e3155 Type:ContainerDied Data:01092b85c12244521e5fff43667530b60f56e605201a51ef24e9e5afe133d000} Nov 16 14:58:21.401604 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.401565771Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0c130954dbb465f1c7ab3095bbe964f865cf29adede61a0a48511eb15ededd8" id=90382f3d-383d-45d1-a763-480f9de672b2 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:21.401863 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.401832673Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0c130954dbb465f1c7ab3095bbe964f865cf29adede61a0a48511eb15ededd8 not found" id=90382f3d-383d-45d1-a763-480f9de672b2 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:21.402592 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.402547279Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0c130954dbb465f1c7ab3095bbe964f865cf29adede61a0a48511eb15ededd8" id=f50278f8-3cc7-4059-9a78-072c7cce12c3 name=/runtime.v1.ImageService/PullImage Nov 16 14:58:21.405485 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.405444001Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0c130954dbb465f1c7ab3095bbe964f865cf29adede61a0a48511eb15ededd8\"" Nov 16 14:58:21.414277 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.414235767Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.414277 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.414270068Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.414440 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.414289568Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/portmap\": CREATE" Nov 16 14:58:21.427993 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.427944971Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.427993 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.427986272Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.428215 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.428015772Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/ptp\": CREATE" Nov 16 14:58:21.441353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.441302073Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.441353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.441343973Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.441353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.441364473Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/sbr\": CREATE" Nov 16 14:58:21.454066 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.454014069Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.454066 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.454051169Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.454066 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.454073769Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/static\": CREATE" Nov 16 14:58:21.466358 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.466311262Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.466358 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.466350962Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.466358 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.466376163Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/tuning\": CREATE" Nov 16 14:58:21.479183 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.479093759Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.479183 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.479146159Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.479183 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.479164360Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/vlan\": CREATE" Nov 16 14:58:21.489989 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.489874241Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.489989 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.489911741Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.489989 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.489934241Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/vrf\": CREATE" Nov 16 14:58:21.503326 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.503285743Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:21.503326 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.503324743Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:21.503777 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:21.503351443Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/upgrade_0770e5dd-9fe2-4595-835f-b45938ce843e\": REMOVE" Nov 16 14:58:22.672762 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:22.672639308Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0c130954dbb465f1c7ab3095bbe964f865cf29adede61a0a48511eb15ededd8\"" Nov 16 14:58:23.769829 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:23.769775708Z" level=info msg="Stopping pod sandbox: 8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3" id=02ea1631-5628-4d3c-93aa-a44168fb8b5c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:58:23.769829 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:23.769828509Z" level=info msg="Stopped pod sandbox (already stopped): 8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3" id=02ea1631-5628-4d3c-93aa-a44168fb8b5c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:58:23.770388 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:23.770122211Z" level=info msg="Removing pod sandbox: 8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3" id=12a2a034-0181-4fde-a16a-0d342fab36a0 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:58:23.826539 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:23.826378218Z" level=info msg="Removed pod sandbox: 8dea0f5a469cea07b9d6312e031ea1e37d638c34906bbbc2ef4df2a9fd4deab3" id=12a2a034-0181-4fde-a16a-0d342fab36a0 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:58:25.128622 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.128523353Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0c130954dbb465f1c7ab3095bbe964f865cf29adede61a0a48511eb15ededd8" id=f50278f8-3cc7-4059-9a78-072c7cce12c3 name=/runtime.v1.ImageService/PullImage Nov 16 14:58:25.131997 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.131657876Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0c130954dbb465f1c7ab3095bbe964f865cf29adede61a0a48511eb15ededd8" id=ab609251-8225-4d90-9c8c-d3ad622626a6 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:25.136981 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.133558890Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:4636afddb07954953b980444f4ea48854447719913094a05af4cd76bb50cf838,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0c130954dbb465f1c7ab3095bbe964f865cf29adede61a0a48511eb15ededd8],Size_:337484529,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=ab609251-8225-4d90-9c8c-d3ad622626a6 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:25.136981 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.134683298Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-vw2s6/bond-cni-plugin" id=20f67097-3f1e-41f1-b97b-612e30e65633 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:25.136981 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.134783599Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:58:25.211975 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-8a36ed77c0b4a188dd4f685c8133a390fa799f0c800126e258407835580581db.scope. Nov 16 14:58:25.242707 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 8a36ed77c0b4a188dd4f685c8133a390fa799f0c800126e258407835580581db. Nov 16 14:58:25.277752 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:58:25.350846 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.350720363Z" level=info msg="Created container 8a36ed77c0b4a188dd4f685c8133a390fa799f0c800126e258407835580581db: openshift-multus/multus-additional-cni-plugins-vw2s6/bond-cni-plugin" id=20f67097-3f1e-41f1-b97b-612e30e65633 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:25.351478 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.351380468Z" level=info msg="Starting container: 8a36ed77c0b4a188dd4f685c8133a390fa799f0c800126e258407835580581db" id=81045170-294a-4f3a-a083-9eaaba9a483e name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:58:25.364108 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.364028660Z" level=info msg="Started container" PID=193715 containerID=8a36ed77c0b4a188dd4f685c8133a390fa799f0c800126e258407835580581db description=openshift-multus/multus-additional-cni-plugins-vw2s6/bond-cni-plugin id=81045170-294a-4f3a-a083-9eaaba9a483e name=/runtime.v1.RuntimeService/StartContainer sandboxID=99dbbd3c144110014f41261f1b3b4a57124d397e3ff49838d099ed63cff0cb80 Nov 16 14:58:25.373868 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.373796831Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/upgrade_62d05b01-ad50-4e31-9797-ad1a3e90a7b2\": CREATE" Nov 16 14:58:25.386991 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.386851025Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:25.386991 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.386893026Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:25.386991 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.386922126Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/bond\": CREATE" Nov 16 14:58:25.397311 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.397269801Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:25.397311 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.397304901Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:25.397449 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:25.397322501Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/upgrade_62d05b01-ad50-4e31-9797-ad1a3e90a7b2\": REMOVE" Nov 16 14:58:25.405287 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-8a36ed77c0b4a188dd4f685c8133a390fa799f0c800126e258407835580581db.scope: Succeeded. Nov 16 14:58:25.406058 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-8a36ed77c0b4a188dd4f685c8133a390fa799f0c800126e258407835580581db.scope: Consumed 53ms CPU time Nov 16 14:58:25.414953 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:25.414917 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2s6" event=&{ID:57197822-83e2-43fe-a3d3-1a63580e3155 Type:ContainerStarted Data:8a36ed77c0b4a188dd4f685c8133a390fa799f0c800126e258407835580581db} Nov 16 14:58:25.417307 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-8a36ed77c0b4a188dd4f685c8133a390fa799f0c800126e258407835580581db.scope: Succeeded. Nov 16 14:58:25.418435 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-8a36ed77c0b4a188dd4f685c8133a390fa799f0c800126e258407835580581db.scope: Consumed 38ms CPU time Nov 16 14:58:26.422989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:26.422949 1563 generic.go:296] "Generic (PLEG): container finished" podID=57197822-83e2-43fe-a3d3-1a63580e3155 containerID="8a36ed77c0b4a188dd4f685c8133a390fa799f0c800126e258407835580581db" exitCode=0 Nov 16 14:58:26.422989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:26.423008 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2s6" event=&{ID:57197822-83e2-43fe-a3d3-1a63580e3155 Type:ContainerDied Data:8a36ed77c0b4a188dd4f685c8133a390fa799f0c800126e258407835580581db} Nov 16 14:58:26.424246 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:26.424179841Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a189a80a8dd46629637615857b83d8a03bf359e328678619ad96f9a4e15a3e25" id=b0cce6a6-7a1a-4861-a0b4-ab4ab116c91e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:26.424573 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:26.424494844Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a189a80a8dd46629637615857b83d8a03bf359e328678619ad96f9a4e15a3e25 not found" id=b0cce6a6-7a1a-4861-a0b4-ab4ab116c91e name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:26.425404 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:26.425364350Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a189a80a8dd46629637615857b83d8a03bf359e328678619ad96f9a4e15a3e25" id=6897fff5-e2a6-4ad1-93c5-8d37d03e8395 name=/runtime.v1.ImageService/PullImage Nov 16 14:58:26.428296 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:26.428258771Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a189a80a8dd46629637615857b83d8a03bf359e328678619ad96f9a4e15a3e25\"" Nov 16 14:58:27.664370 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:27.664314127Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a189a80a8dd46629637615857b83d8a03bf359e328678619ad96f9a4e15a3e25\"" Nov 16 14:58:30.094671 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.094558236Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a189a80a8dd46629637615857b83d8a03bf359e328678619ad96f9a4e15a3e25" id=6897fff5-e2a6-4ad1-93c5-8d37d03e8395 name=/runtime.v1.ImageService/PullImage Nov 16 14:58:30.095738 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.095700644Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a189a80a8dd46629637615857b83d8a03bf359e328678619ad96f9a4e15a3e25" id=87e403bd-91bb-4018-a8e5-3f720c1da7e3 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:30.097253 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.097163055Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:67bf4d26b6a4a63cc62193a84d642e8374d132012fc52ec48d8e02758617ca5d,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a189a80a8dd46629637615857b83d8a03bf359e328678619ad96f9a4e15a3e25],Size_:304154476,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=87e403bd-91bb-4018-a8e5-3f720c1da7e3 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:30.098224 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.098176462Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-vw2s6/routeoverride-cni" id=b39c048c-e5dd-4a0e-8e22-fa646014e68b name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:30.098322 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.098286663Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:58:30.164759 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-98c539549af9a8745528d63c6ea2c32a881cb1471818ef3464fb2225ba5999fe.scope. Nov 16 14:58:30.192635 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 98c539549af9a8745528d63c6ea2c32a881cb1471818ef3464fb2225ba5999fe. Nov 16 14:58:30.227596 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:58:30.295357 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.295304491Z" level=info msg="Created container 98c539549af9a8745528d63c6ea2c32a881cb1471818ef3464fb2225ba5999fe: openshift-multus/multus-additional-cni-plugins-vw2s6/routeoverride-cni" id=b39c048c-e5dd-4a0e-8e22-fa646014e68b name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:30.296007 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.295972296Z" level=info msg="Starting container: 98c539549af9a8745528d63c6ea2c32a881cb1471818ef3464fb2225ba5999fe" id=2dae3922-3799-4a98-8474-c6606a0dd2b1 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:58:30.307177 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.307092776Z" level=info msg="Started container" PID=193919 containerID=98c539549af9a8745528d63c6ea2c32a881cb1471818ef3464fb2225ba5999fe description=openshift-multus/multus-additional-cni-plugins-vw2s6/routeoverride-cni id=2dae3922-3799-4a98-8474-c6606a0dd2b1 name=/runtime.v1.RuntimeService/StartContainer sandboxID=99dbbd3c144110014f41261f1b3b4a57124d397e3ff49838d099ed63cff0cb80 Nov 16 14:58:30.317585 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.317532452Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/upgrade_c335d4e4-de3c-4969-9f13-b74cd7ef7bce\": CREATE" Nov 16 14:58:30.329493 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.329448838Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:30.329493 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.329486938Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:30.329671 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.329509639Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/route-override\": CREATE" Nov 16 14:58:30.340235 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.340193816Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:30.340235 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.340228216Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:30.340387 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.340257416Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/upgrade_c335d4e4-de3c-4969-9f13-b74cd7ef7bce\": REMOVE" Nov 16 14:58:30.345186 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-98c539549af9a8745528d63c6ea2c32a881cb1471818ef3464fb2225ba5999fe.scope: Succeeded. Nov 16 14:58:30.345920 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-98c539549af9a8745528d63c6ea2c32a881cb1471818ef3464fb2225ba5999fe.scope: Consumed 53ms CPU time Nov 16 14:58:30.359323 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-98c539549af9a8745528d63c6ea2c32a881cb1471818ef3464fb2225ba5999fe.scope: Succeeded. Nov 16 14:58:30.360037 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-98c539549af9a8745528d63c6ea2c32a881cb1471818ef3464fb2225ba5999fe.scope: Consumed 37ms CPU time Nov 16 14:58:30.439190 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:30.439153 1563 generic.go:296] "Generic (PLEG): container finished" podID=57197822-83e2-43fe-a3d3-1a63580e3155 containerID="98c539549af9a8745528d63c6ea2c32a881cb1471818ef3464fb2225ba5999fe" exitCode=0 Nov 16 14:58:30.439190 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:30.439208 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2s6" event=&{ID:57197822-83e2-43fe-a3d3-1a63580e3155 Type:ContainerDied Data:98c539549af9a8745528d63c6ea2c32a881cb1471818ef3464fb2225ba5999fe} Nov 16 14:58:30.440043 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.439998139Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d514369668723ab4ab137de35dacedc96bd30909f205ca0f302be367c8526c17" id=032e5cc4-2bbb-43e1-94b1-1c4a47ad8667 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:30.440364 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.440334742Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d514369668723ab4ab137de35dacedc96bd30909f205ca0f302be367c8526c17 not found" id=032e5cc4-2bbb-43e1-94b1-1c4a47ad8667 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:30.441256 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.441195648Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d514369668723ab4ab137de35dacedc96bd30909f205ca0f302be367c8526c17" id=93f2876d-6181-44c0-813f-ee2610d708b3 name=/runtime.v1.ImageService/PullImage Nov 16 14:58:30.444414 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:30.444378771Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d514369668723ab4ab137de35dacedc96bd30909f205ca0f302be367c8526c17\"" Nov 16 14:58:31.685071 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:31.684965457Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d514369668723ab4ab137de35dacedc96bd30909f205ca0f302be367c8526c17\"" Nov 16 14:58:36.514592 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.514430713Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d514369668723ab4ab137de35dacedc96bd30909f205ca0f302be367c8526c17" id=93f2876d-6181-44c0-813f-ee2610d708b3 name=/runtime.v1.ImageService/PullImage Nov 16 14:58:36.515685 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.515632023Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d514369668723ab4ab137de35dacedc96bd30909f205ca0f302be367c8526c17" id=ab9ee480-8305-4421-b751-9aa23301adfc name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:36.517283 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.517243335Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9f60962189a8c9fda72cc7de8b811389d7e9b8f0ad5fe8b6d5e809fd4d1f187f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d514369668723ab4ab137de35dacedc96bd30909f205ca0f302be367c8526c17],Size_:465671015,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=ab9ee480-8305-4421-b751-9aa23301adfc name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:36.518389 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.518349744Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-vw2s6/whereabouts-cni-bincopy" id=67a41ba5-d6fe-4f9d-a7dc-4da5169c3a09 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:36.518477 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.518450744Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:58:36.580808 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-cf8456008a5b07f8166343bb1f6ed4000ba51d2c38a7c7fef6228083e8a1b82b.scope. Nov 16 14:58:36.609545 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container cf8456008a5b07f8166343bb1f6ed4000ba51d2c38a7c7fef6228083e8a1b82b. Nov 16 14:58:36.644798 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:58:36.707911 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.707858504Z" level=info msg="Created container cf8456008a5b07f8166343bb1f6ed4000ba51d2c38a7c7fef6228083e8a1b82b: openshift-multus/multus-additional-cni-plugins-vw2s6/whereabouts-cni-bincopy" id=67a41ba5-d6fe-4f9d-a7dc-4da5169c3a09 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:36.708522 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.708490209Z" level=info msg="Starting container: cf8456008a5b07f8166343bb1f6ed4000ba51d2c38a7c7fef6228083e8a1b82b" id=aedd908e-8f5f-4ef3-82c8-65f92709a159 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:58:36.718854 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.718773788Z" level=info msg="Started container" PID=194071 containerID=cf8456008a5b07f8166343bb1f6ed4000ba51d2c38a7c7fef6228083e8a1b82b description=openshift-multus/multus-additional-cni-plugins-vw2s6/whereabouts-cni-bincopy id=aedd908e-8f5f-4ef3-82c8-65f92709a159 name=/runtime.v1.RuntimeService/StartContainer sandboxID=99dbbd3c144110014f41261f1b3b4a57124d397e3ff49838d099ed63cff0cb80 Nov 16 14:58:36.725511 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.725454440Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/upgrade_4441aa2f-e20c-47b5-9c90-24d67a968c28\": CREATE" Nov 16 14:58:36.738811 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.738759142Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:36.738811 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.738803142Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:36.760957 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.760899713Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/whereabouts\": CREATE" Nov 16 14:58:36.775176 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.775031722Z" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/kubernetes/cni/net.d/00-multus.conf" Nov 16 14:58:36.775176 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.775069822Z" level=info msg="Updated default CNI network name to multus-cni-network" Nov 16 14:58:36.775176 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:36.775103322Z" level=info msg="CNI monitoring event \"/var/lib/cni/bin/upgrade_4441aa2f-e20c-47b5-9c90-24d67a968c28\": REMOVE" Nov 16 14:58:36.783122 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-cf8456008a5b07f8166343bb1f6ed4000ba51d2c38a7c7fef6228083e8a1b82b.scope: Succeeded. Nov 16 14:58:36.783910 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-cf8456008a5b07f8166343bb1f6ed4000ba51d2c38a7c7fef6228083e8a1b82b.scope: Consumed 78ms CPU time Nov 16 14:58:36.792958 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-cf8456008a5b07f8166343bb1f6ed4000ba51d2c38a7c7fef6228083e8a1b82b.scope: Succeeded. Nov 16 14:58:36.793595 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-cf8456008a5b07f8166343bb1f6ed4000ba51d2c38a7c7fef6228083e8a1b82b.scope: Consumed 40ms CPU time Nov 16 14:58:37.464012 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:37.463961 1563 generic.go:296] "Generic (PLEG): container finished" podID=57197822-83e2-43fe-a3d3-1a63580e3155 containerID="cf8456008a5b07f8166343bb1f6ed4000ba51d2c38a7c7fef6228083e8a1b82b" exitCode=0 Nov 16 14:58:37.464012 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:37.464016 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2s6" event=&{ID:57197822-83e2-43fe-a3d3-1a63580e3155 Type:ContainerDied Data:cf8456008a5b07f8166343bb1f6ed4000ba51d2c38a7c7fef6228083e8a1b82b} Nov 16 14:58:37.465045 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:37.464991238Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d514369668723ab4ab137de35dacedc96bd30909f205ca0f302be367c8526c17" id=e512ee74-1337-4584-a87c-113921250ae5 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:37.466673 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:37.466601551Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9f60962189a8c9fda72cc7de8b811389d7e9b8f0ad5fe8b6d5e809fd4d1f187f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d514369668723ab4ab137de35dacedc96bd30909f205ca0f302be367c8526c17],Size_:465671015,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=e512ee74-1337-4584-a87c-113921250ae5 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:37.467742 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:37.467540158Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d514369668723ab4ab137de35dacedc96bd30909f205ca0f302be367c8526c17" id=20f06e35-887f-4a1c-8dd6-d4ddb8bf6676 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:37.469254 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:37.469207271Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9f60962189a8c9fda72cc7de8b811389d7e9b8f0ad5fe8b6d5e809fd4d1f187f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d514369668723ab4ab137de35dacedc96bd30909f205ca0f302be367c8526c17],Size_:465671015,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=20f06e35-887f-4a1c-8dd6-d4ddb8bf6676 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:37.475612 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:37.475471919Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-vw2s6/whereabouts-cni" id=583ff367-4187-4ce2-8869-2fef1b93b922 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:37.475612 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:37.475584920Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:58:37.535199 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-895a322d1d4ea9e01351e8d59ac7e6ce077bcc34238286db5ceae0bec8aa9bca.scope. Nov 16 14:58:37.564512 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 895a322d1d4ea9e01351e8d59ac7e6ce077bcc34238286db5ceae0bec8aa9bca. Nov 16 14:58:37.603059 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:58:37.678583 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:37.678432983Z" level=info msg="Created container 895a322d1d4ea9e01351e8d59ac7e6ce077bcc34238286db5ceae0bec8aa9bca: openshift-multus/multus-additional-cni-plugins-vw2s6/whereabouts-cni" id=583ff367-4187-4ce2-8869-2fef1b93b922 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:37.679183 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:37.679145689Z" level=info msg="Starting container: 895a322d1d4ea9e01351e8d59ac7e6ce077bcc34238286db5ceae0bec8aa9bca" id=d821d386-c6d9-42c1-80c2-38394ce21045 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:58:37.690352 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:37.690303775Z" level=info msg="Started container" PID=194151 containerID=895a322d1d4ea9e01351e8d59ac7e6ce077bcc34238286db5ceae0bec8aa9bca description=openshift-multus/multus-additional-cni-plugins-vw2s6/whereabouts-cni id=d821d386-c6d9-42c1-80c2-38394ce21045 name=/runtime.v1.RuntimeService/StartContainer sandboxID=99dbbd3c144110014f41261f1b3b4a57124d397e3ff49838d099ed63cff0cb80 Nov 16 14:58:37.723103 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-895a322d1d4ea9e01351e8d59ac7e6ce077bcc34238286db5ceae0bec8aa9bca.scope: Succeeded. Nov 16 14:58:37.723939 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-895a322d1d4ea9e01351e8d59ac7e6ce077bcc34238286db5ceae0bec8aa9bca.scope: Consumed 50ms CPU time Nov 16 14:58:37.732683 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-895a322d1d4ea9e01351e8d59ac7e6ce077bcc34238286db5ceae0bec8aa9bca.scope: Succeeded. Nov 16 14:58:37.733522 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-895a322d1d4ea9e01351e8d59ac7e6ce077bcc34238286db5ceae0bec8aa9bca.scope: Consumed 40ms CPU time Nov 16 14:58:38.471222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:38.471180 1563 generic.go:296] "Generic (PLEG): container finished" podID=57197822-83e2-43fe-a3d3-1a63580e3155 containerID="895a322d1d4ea9e01351e8d59ac7e6ce077bcc34238286db5ceae0bec8aa9bca" exitCode=0 Nov 16 14:58:38.471222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:38.471241 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2s6" event=&{ID:57197822-83e2-43fe-a3d3-1a63580e3155 Type:ContainerDied Data:895a322d1d4ea9e01351e8d59ac7e6ce077bcc34238286db5ceae0bec8aa9bca} Nov 16 14:58:38.472413 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:38.472365901Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6c99d61b704589581b000dc6dd224449caf9419879e49af19ccfae10831d9cea" id=eb397a9b-2399-48e1-8934-c320278b5858 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:38.472718 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:38.472648903Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5470cde07684980e3b22be2bc068067b5004d13080cf46547f7dc7f3ecab09b0,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6c99d61b704589581b000dc6dd224449caf9419879e49af19ccfae10831d9cea],Size_:471431625,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=eb397a9b-2399-48e1-8934-c320278b5858 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:38.473811 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:38.473721211Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6c99d61b704589581b000dc6dd224449caf9419879e49af19ccfae10831d9cea" id=0a0d10eb-40f0-4286-b535-503b6fe999d1 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:38.473953 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:38.473916713Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5470cde07684980e3b22be2bc068067b5004d13080cf46547f7dc7f3ecab09b0,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6c99d61b704589581b000dc6dd224449caf9419879e49af19ccfae10831d9cea],Size_:471431625,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=0a0d10eb-40f0-4286-b535-503b6fe999d1 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:58:38.474787 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:38.474746219Z" level=info msg="Creating container: openshift-multus/multus-additional-cni-plugins-vw2s6/kube-multus-additional-cni-plugins" id=b15eee89-2cfd-4c21-b45d-c2511e57e7d3 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:38.474875 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:38.474847520Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:58:38.534801 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-5b0b7785507112c5a5d0bdb6684db02aef01386a2c42d8d155408e11d124559b.scope. Nov 16 14:58:38.565273 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 5b0b7785507112c5a5d0bdb6684db02aef01386a2c42d8d155408e11d124559b. Nov 16 14:58:38.634334 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:38.634283049Z" level=info msg="Created container 5b0b7785507112c5a5d0bdb6684db02aef01386a2c42d8d155408e11d124559b: openshift-multus/multus-additional-cni-plugins-vw2s6/kube-multus-additional-cni-plugins" id=b15eee89-2cfd-4c21-b45d-c2511e57e7d3 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:58:38.634965 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:38.634848953Z" level=info msg="Starting container: 5b0b7785507112c5a5d0bdb6684db02aef01386a2c42d8d155408e11d124559b" id=a8e3e299-e37e-4682-85d7-bb83b369d719 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:58:38.646880 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:38.646837345Z" level=info msg="Started container" PID=194223 containerID=5b0b7785507112c5a5d0bdb6684db02aef01386a2c42d8d155408e11d124559b description=openshift-multus/multus-additional-cni-plugins-vw2s6/kube-multus-additional-cni-plugins id=a8e3e299-e37e-4682-85d7-bb83b369d719 name=/runtime.v1.RuntimeService/StartContainer sandboxID=99dbbd3c144110014f41261f1b3b4a57124d397e3ff49838d099ed63cff0cb80 Nov 16 14:58:39.477217 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:39.477181 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2s6" event=&{ID:57197822-83e2-43fe-a3d3-1a63580e3155 Type:ContainerStarted Data:5b0b7785507112c5a5d0bdb6684db02aef01386a2c42d8d155408e11d124559b} Nov 16 14:58:39.809873 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:39.809736 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-dns/dns-default-kg69s] Nov 16 14:58:39.810018 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:39.809990 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-dns/dns-default-kg69s" podUID=306e3397-e392-42af-b8d9-63e5f5dfae1b containerName="dns" containerID="cri-o://a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9" gracePeriod=30 Nov 16 14:58:39.810178 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:39.810116 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-dns/dns-default-kg69s" podUID=306e3397-e392-42af-b8d9-63e5f5dfae1b containerName="kube-rbac-proxy" containerID="cri-o://ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68" gracePeriod=30 Nov 16 14:58:39.810512 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:39.810423012Z" level=info msg="Stopping container: ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68 (timeout: 30s)" id=afb0618a-e3e1-499e-a78c-5b2d23e96467 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:58:39.810880 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:39.810472812Z" level=info msg="Stopping container: a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9 (timeout: 30s)" id=8bb2c1d8-af06-48f1-bf61-2925eb82807a name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:58:40.916026 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68.scope: Succeeded. Nov 16 14:58:40.916738 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68.scope: Consumed 2.913s CPU time Nov 16 14:58:40.920741 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68.scope: Succeeded. Nov 16 14:58:40.921506 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68.scope: Consumed 37ms CPU time Nov 16 14:58:40.979446 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-86567d4eec368dfd71119f0fcab5375c79cb45c70eab272710a368df95de2ab5-merged.mount: Succeeded. Nov 16 14:58:40.980365 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-86567d4eec368dfd71119f0fcab5375c79cb45c70eab272710a368df95de2ab5-merged.mount: Consumed 0 CPU time Nov 16 14:58:41.295390 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:41.295219 1563 patch_prober.go:29] interesting pod/dns-default-kg69s container/dns namespace/openshift-dns: Readiness probe status=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" start-of-body= Nov 16 14:58:41.295390 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:41.295315 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-dns/dns-default-kg69s" podUID=306e3397-e392-42af-b8d9-63e5f5dfae1b containerName="dns" probeResult=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" Nov 16 14:58:41.485643 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:41.485591 1563 generic.go:296] "Generic (PLEG): container finished" podID=306e3397-e392-42af-b8d9-63e5f5dfae1b containerID="ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68" exitCode=0 Nov 16 14:58:41.485643 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:41.485651 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kg69s" event=&{ID:306e3397-e392-42af-b8d9-63e5f5dfae1b Type:ContainerDied Data:ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68} Nov 16 14:58:44.295423 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:44.295329 1563 patch_prober.go:29] interesting pod/dns-default-kg69s container/dns namespace/openshift-dns: Readiness probe status=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" start-of-body= Nov 16 14:58:44.295423 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:44.295423 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-dns/dns-default-kg69s" podUID=306e3397-e392-42af-b8d9-63e5f5dfae1b containerName="dns" probeResult=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" Nov 16 14:58:47.294808 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:47.294758 1563 patch_prober.go:29] interesting pod/dns-default-kg69s container/dns namespace/openshift-dns: Readiness probe status=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" start-of-body= Nov 16 14:58:47.295454 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:47.294843 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-dns/dns-default-kg69s" podUID=306e3397-e392-42af-b8d9-63e5f5dfae1b containerName="dns" probeResult=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" Nov 16 14:58:47.295454 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:47.294961 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-dns/dns-default-kg69s" Nov 16 14:58:47.735637 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b-runc.YZu0sr.mount: Succeeded. Nov 16 14:58:47.742535 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:58:47.742447313Z" level=info msg="Stopped container ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68: openshift-dns/dns-default-kg69s/kube-rbac-proxy" id=afb0618a-e3e1-499e-a78c-5b2d23e96467 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:58:50.295774 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:50.295725 1563 patch_prober.go:29] interesting pod/dns-default-kg69s container/dns namespace/openshift-dns: Readiness probe status=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" start-of-body= Nov 16 14:58:50.296408 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:50.295849 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-dns/dns-default-kg69s" podUID=306e3397-e392-42af-b8d9-63e5f5dfae1b containerName="dns" probeResult=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" Nov 16 14:58:53.295781 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:53.295679 1563 patch_prober.go:29] interesting pod/dns-default-kg69s container/dns namespace/openshift-dns: Readiness probe status=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" start-of-body= Nov 16 14:58:53.295781 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:53.295791 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-dns/dns-default-kg69s" podUID=306e3397-e392-42af-b8d9-63e5f5dfae1b containerName="dns" probeResult=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" Nov 16 14:58:56.295467 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:56.295397 1563 patch_prober.go:29] interesting pod/dns-default-kg69s container/dns namespace/openshift-dns: Readiness probe status=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" start-of-body= Nov 16 14:58:56.296102 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:56.295531 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-dns/dns-default-kg69s" podUID=306e3397-e392-42af-b8d9-63e5f5dfae1b containerName="dns" probeResult=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" Nov 16 14:58:59.295271 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:59.295099 1563 patch_prober.go:29] interesting pod/dns-default-kg69s container/dns namespace/openshift-dns: Readiness probe status=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" start-of-body= Nov 16 14:58:59.295271 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:58:59.295245 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-dns/dns-default-kg69s" podUID=306e3397-e392-42af-b8d9-63e5f5dfae1b containerName="dns" probeResult=failure output="Get \"http://10.129.2.2:8181/ready\": dial tcp 10.129.2.2:8181: connect: connection refused" Nov 16 14:58:59.838906 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9.scope: Succeeded. Nov 16 14:58:59.839585 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9.scope: Consumed 34ms CPU time Nov 16 14:58:59.842945 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9.scope: Succeeded. Nov 16 14:58:59.843768 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9.scope: Consumed 27.987s CPU time Nov 16 14:58:59.989187 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6b84db603b43b99447b545ee2091b2ef0badaeaed2000a8d9a7242904d7efc97-merged.mount: Succeeded. Nov 16 14:58:59.990229 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6b84db603b43b99447b545ee2091b2ef0badaeaed2000a8d9a7242904d7efc97-merged.mount: Consumed 0 CPU time Nov 16 14:59:00.184437 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:00.184148376Z" level=info msg="Stopped container a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9: openshift-dns/dns-default-kg69s/dns" id=8bb2c1d8-af06-48f1-bf61-2925eb82807a name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:59:00.184922 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:00.184872082Z" level=info msg="Stopping pod sandbox: 9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f" id=b4959f63-493a-453e-87c1-9064a2d95b3c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:59:00.185265 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:00.185226384Z" level=info msg="Got pod network &{Name:dns-default-kg69s Namespace:openshift-dns ID:9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f UID:306e3397-e392-42af-b8d9-63e5f5dfae1b NetNS:/var/run/netns/beaa79cf-4611-4079-83a4-5e17cfd681cc Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:59:00.185460 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:00.185403086Z" level=info msg="Deleting pod openshift-dns_dns-default-kg69s from CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:59:00.261556 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00239|connmgr|INFO|br0<->unix#542: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:59:00.299431 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00240|connmgr|INFO|br0<->unix#545: 4 flow_mods in the last 0 s (4 deletes) Nov 16 14:59:00.330402 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00241|bridge|INFO|bridge br0: deleted interface vethb019706b on port 3 Nov 16 14:59:00.337194 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethb019706b left promiscuous mode Nov 16 14:59:00.345241 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:59:00Z [verbose] Del: openshift-dns:dns-default-kg69s:306e3397-e392-42af-b8d9-63e5f5dfae1b:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 14:59:00.355593 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-beaa79cf\x2d4611\x2d4079\x2d83a4\x2d5e17cfd681cc.mount: Succeeded. Nov 16 14:59:00.356449 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-beaa79cf\x2d4611\x2d4079\x2d83a4\x2d5e17cfd681cc.mount: Consumed 0 CPU time Nov 16 14:59:00.373887 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-beaa79cf\x2d4611\x2d4079\x2d83a4\x2d5e17cfd681cc.mount: Succeeded. Nov 16 14:59:00.375494 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-beaa79cf\x2d4611\x2d4079\x2d83a4\x2d5e17cfd681cc.mount: Consumed 0 CPU time Nov 16 14:59:00.403781 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-beaa79cf\x2d4611\x2d4079\x2d83a4\x2d5e17cfd681cc.mount: Succeeded. Nov 16 14:59:00.404612 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-beaa79cf\x2d4611\x2d4079\x2d83a4\x2d5e17cfd681cc.mount: Consumed 0 CPU time Nov 16 14:59:00.427093 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f-userdata-shm.mount: Succeeded. Nov 16 14:59:00.427988 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f-userdata-shm.mount: Consumed 0 CPU time Nov 16 14:59:00.555356 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:00.555224 1563 generic.go:296] "Generic (PLEG): container finished" podID=306e3397-e392-42af-b8d9-63e5f5dfae1b containerID="a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9" exitCode=0 Nov 16 14:59:00.555356 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:00.555280 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kg69s" event=&{ID:306e3397-e392-42af-b8d9-63e5f5dfae1b Type:ContainerDied Data:a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9} Nov 16 14:59:00.745369 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:00.745278947Z" level=info msg="Stopped pod sandbox: 9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f" id=b4959f63-493a-453e-87c1-9064a2d95b3c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:59:00.899505 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:00.899445 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306e3397-e392-42af-b8d9-63e5f5dfae1b-metrics-tls\") pod \"306e3397-e392-42af-b8d9-63e5f5dfae1b\" (UID: \"306e3397-e392-42af-b8d9-63e5f5dfae1b\") " Nov 16 14:59:00.899505 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:00.899520 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/306e3397-e392-42af-b8d9-63e5f5dfae1b-config-volume\") pod \"306e3397-e392-42af-b8d9-63e5f5dfae1b\" (UID: \"306e3397-e392-42af-b8d9-63e5f5dfae1b\") " Nov 16 14:59:00.899837 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:00.899608 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6wbv\" (UniqueName: \"kubernetes.io/projected/306e3397-e392-42af-b8d9-63e5f5dfae1b-kube-api-access-n6wbv\") pod \"306e3397-e392-42af-b8d9-63e5f5dfae1b\" (UID: \"306e3397-e392-42af-b8d9-63e5f5dfae1b\") " Nov 16 14:59:00.899880 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:59:00.899833 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/306e3397-e392-42af-b8d9-63e5f5dfae1b/volumes/kubernetes.io~configmap/config-volume: clearQuota called, but quotas disabled Nov 16 14:59:00.900113 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:00.900066 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/306e3397-e392-42af-b8d9-63e5f5dfae1b-config-volume" (OuterVolumeSpecName: "config-volume") pod "306e3397-e392-42af-b8d9-63e5f5dfae1b" (UID: "306e3397-e392-42af-b8d9-63e5f5dfae1b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 14:59:00.919478 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:00.919363 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/306e3397-e392-42af-b8d9-63e5f5dfae1b-kube-api-access-n6wbv" (OuterVolumeSpecName: "kube-api-access-n6wbv") pod "306e3397-e392-42af-b8d9-63e5f5dfae1b" (UID: "306e3397-e392-42af-b8d9-63e5f5dfae1b"). InnerVolumeSpecName "kube-api-access-n6wbv". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:59:00.927405 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:00.927359 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/306e3397-e392-42af-b8d9-63e5f5dfae1b-metrics-tls" (OuterVolumeSpecName: "metrics-tls") pod "306e3397-e392-42af-b8d9-63e5f5dfae1b" (UID: "306e3397-e392-42af-b8d9-63e5f5dfae1b"). InnerVolumeSpecName "metrics-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 14:59:00.988876 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-940d904969089787dcec5c841f619897ceff460d57a5ff499d2373ad9c865fc4-merged.mount: Succeeded. Nov 16 14:59:00.989985 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-940d904969089787dcec5c841f619897ceff460d57a5ff499d2373ad9c865fc4-merged.mount: Consumed 0 CPU time Nov 16 14:59:00.990025 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-306e3397\x2de392\x2d42af\x2db8d9\x2d63e5f5dfae1b-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dn6wbv.mount: Succeeded. Nov 16 14:59:00.990935 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-306e3397\x2de392\x2d42af\x2db8d9\x2d63e5f5dfae1b-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dn6wbv.mount: Consumed 0 CPU time Nov 16 14:59:00.990949 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-306e3397\x2de392\x2d42af\x2db8d9\x2d63e5f5dfae1b-volumes-kubernetes.io\x7esecret-metrics\x2dtls.mount: Succeeded. Nov 16 14:59:00.991706 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-306e3397\x2de392\x2d42af\x2db8d9\x2d63e5f5dfae1b-volumes-kubernetes.io\x7esecret-metrics\x2dtls.mount: Consumed 0 CPU time Nov 16 14:59:01.001077 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.001003 1563 reconciler.go:300] "Volume detached for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306e3397-e392-42af-b8d9-63e5f5dfae1b-metrics-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:59:01.001077 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.001048 1563 reconciler.go:300] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/306e3397-e392-42af-b8d9-63e5f5dfae1b-config-volume\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:59:01.001077 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.001068 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-n6wbv\" (UniqueName: \"kubernetes.io/projected/306e3397-e392-42af-b8d9-63e5f5dfae1b-kube-api-access-n6wbv\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:59:01.560354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.560314 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kg69s" event=&{ID:306e3397-e392-42af-b8d9-63e5f5dfae1b Type:ContainerDied Data:9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f} Nov 16 14:59:01.560354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.560390 1563 scope.go:110] "RemoveContainer" containerID="ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68" Nov 16 14:59:01.561626 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:01.561576259Z" level=info msg="Removing container: ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68" id=db9dc85f-acbf-4dd2-bd66-291172686781 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:59:01.568777 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod306e3397_e392_42af_b8d9_63e5f5dfae1b.slice. Nov 16 14:59:01.568918 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod306e3397_e392_42af_b8d9_63e5f5dfae1b.slice: Consumed 30.972s CPU time Nov 16 14:59:01.600266 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:01.600201053Z" level=info msg="Removed container ebdcfc7a0757e6100b3acdb2573d6e5efa9cef986a185c4a69db440a8bc63c68: openshift-dns/dns-default-kg69s/kube-rbac-proxy" id=db9dc85f-acbf-4dd2-bd66-291172686781 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:59:01.600551 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.600522 1563 scope.go:110] "RemoveContainer" containerID="a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9" Nov 16 14:59:01.601046 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.601017 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-dns/dns-default-kg69s] Nov 16 14:59:01.601593 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:01.601558463Z" level=info msg="Removing container: a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9" id=5815ca85-89dc-45af-8508-d3634f012c93 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:59:01.610325 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.610294 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-dns/dns-default-kg69s] Nov 16 14:59:01.636623 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:01.636579230Z" level=info msg="Removed container a4a6f8be98539aed0cabbb586f2a72d31a269af1dc3e9c201a6bd34f17bd51a9: openshift-dns/dns-default-kg69s/dns" id=5815ca85-89dc-45af-8508-d3634f012c93 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:59:01.653001 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.652943 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-dns/dns-default-r2xl8] Nov 16 14:59:01.653001 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.653008 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:59:01.653201 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:59:01.653108 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="306e3397-e392-42af-b8d9-63e5f5dfae1b" containerName="kube-rbac-proxy" Nov 16 14:59:01.653201 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.653122 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="306e3397-e392-42af-b8d9-63e5f5dfae1b" containerName="kube-rbac-proxy" Nov 16 14:59:01.653201 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:59:01.653155 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="306e3397-e392-42af-b8d9-63e5f5dfae1b" containerName="dns" Nov 16 14:59:01.653201 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.653164 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="306e3397-e392-42af-b8d9-63e5f5dfae1b" containerName="dns" Nov 16 14:59:01.653340 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.653255 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="306e3397-e392-42af-b8d9-63e5f5dfae1b" containerName="kube-rbac-proxy" Nov 16 14:59:01.653340 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.653272 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="306e3397-e392-42af-b8d9-63e5f5dfae1b" containerName="dns" Nov 16 14:59:01.664458 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod23402384_5d8d_47cb_9b23_e7128b9e0e74.slice. Nov 16 14:59:01.708434 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.708389 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sghcp\" (UniqueName: \"kubernetes.io/projected/23402384-5d8d-47cb-9b23-e7128b9e0e74-kube-api-access-sghcp\") pod \"dns-default-r2xl8\" (UID: \"23402384-5d8d-47cb-9b23-e7128b9e0e74\") " pod="openshift-dns/dns-default-r2xl8" Nov 16 14:59:01.708642 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.708472 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23402384-5d8d-47cb-9b23-e7128b9e0e74-metrics-tls\") pod \"dns-default-r2xl8\" (UID: \"23402384-5d8d-47cb-9b23-e7128b9e0e74\") " pod="openshift-dns/dns-default-r2xl8" Nov 16 14:59:01.708642 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.708575 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/23402384-5d8d-47cb-9b23-e7128b9e0e74-config-volume\") pod \"dns-default-r2xl8\" (UID: \"23402384-5d8d-47cb-9b23-e7128b9e0e74\") " pod="openshift-dns/dns-default-r2xl8" Nov 16 14:59:01.809518 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.809476 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/23402384-5d8d-47cb-9b23-e7128b9e0e74-config-volume\") pod \"dns-default-r2xl8\" (UID: \"23402384-5d8d-47cb-9b23-e7128b9e0e74\") " pod="openshift-dns/dns-default-r2xl8" Nov 16 14:59:01.809740 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.809555 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-sghcp\" (UniqueName: \"kubernetes.io/projected/23402384-5d8d-47cb-9b23-e7128b9e0e74-kube-api-access-sghcp\") pod \"dns-default-r2xl8\" (UID: \"23402384-5d8d-47cb-9b23-e7128b9e0e74\") " pod="openshift-dns/dns-default-r2xl8" Nov 16 14:59:01.809740 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.809595 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23402384-5d8d-47cb-9b23-e7128b9e0e74-metrics-tls\") pod \"dns-default-r2xl8\" (UID: \"23402384-5d8d-47cb-9b23-e7128b9e0e74\") " pod="openshift-dns/dns-default-r2xl8" Nov 16 14:59:01.810272 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.810244 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/23402384-5d8d-47cb-9b23-e7128b9e0e74-config-volume\") pod \"dns-default-r2xl8\" (UID: \"23402384-5d8d-47cb-9b23-e7128b9e0e74\") " pod="openshift-dns/dns-default-r2xl8" Nov 16 14:59:01.813086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.812950 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23402384-5d8d-47cb-9b23-e7128b9e0e74-metrics-tls\") pod \"dns-default-r2xl8\" (UID: \"23402384-5d8d-47cb-9b23-e7128b9e0e74\") " pod="openshift-dns/dns-default-r2xl8" Nov 16 14:59:01.833407 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.833326 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-sghcp\" (UniqueName: \"kubernetes.io/projected/23402384-5d8d-47cb-9b23-e7128b9e0e74-kube-api-access-sghcp\") pod \"dns-default-r2xl8\" (UID: \"23402384-5d8d-47cb-9b23-e7128b9e0e74\") " pod="openshift-dns/dns-default-r2xl8" Nov 16 14:59:01.988693 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:01.988638 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r2xl8" Nov 16 14:59:01.989488 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:01.989395615Z" level=info msg="Running pod sandbox: openshift-dns/dns-default-r2xl8/POD" id=9f16eb58-7b6f-4dc6-9e32-b5f17d55fad8 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:59:01.989591 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:01.989508416Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:59:02.064337 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:02.064195984Z" level=info msg="Got pod network &{Name:dns-default-r2xl8 Namespace:openshift-dns ID:be5627cc24a753ae3d8cee4423f7b8f43b59357470b4e170e0a36002384ff61e UID:23402384-5d8d-47cb-9b23-e7128b9e0e74 NetNS:/var/run/netns/9f330035-431d-46d8-87b3-e1295c9631a3 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:59:02.064337 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:02.064243884Z" level=info msg="Adding pod openshift-dns_dns-default-r2xl8 to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:59:02.111203 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:59:02.138023 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[194871]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:59:02.143457 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): veth6dd492db: link is not ready Nov 16 14:59:02.143560 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth6dd492db: link becomes ready Nov 16 14:59:02.143598 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:59:02.149308 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[194871]: Could not generate persistent MAC address for veth6dd492db: No such file or directory Nov 16 14:59:02.149588 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610742.1495] device (veth6dd492db): carrier: link connected Nov 16 14:59:02.149983 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610742.1499] manager: (veth6dd492db): new Veth device (/org/freedesktop/NetworkManager/Devices/84) Nov 16 14:59:02.172187 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610742.1721] manager: (veth6dd492db): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/85) Nov 16 14:59:02.178723 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00242|bridge|INFO|bridge br0: added interface veth6dd492db on port 40 Nov 16 14:59:02.179162 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth6dd492db entered promiscuous mode Nov 16 14:59:02.212892 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00243|connmgr|INFO|br0<->unix#548: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:59:02.258934 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00244|connmgr|INFO|br0<->unix#551: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:59:02.594260 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:02.594202718Z" level=info msg="Stopping pod sandbox: 9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f" id=eb91ba74-563c-41fc-9db3-7367e491cad8 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:59:02.594260 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:02.594261618Z" level=info msg="Stopped pod sandbox (already stopped): 9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f" id=eb91ba74-563c-41fc-9db3-7367e491cad8 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:59:02.596445 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:02.596410 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=306e3397-e392-42af-b8d9-63e5f5dfae1b path="/var/lib/kubelet/pods/306e3397-e392-42af-b8d9-63e5f5dfae1b/volumes" Nov 16 14:59:03.832482 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:59:03Z [verbose] Add: openshift-dns:dns-default-r2xl8:23402384-5d8d-47cb-9b23-e7128b9e0e74:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/9f330035-431d-46d8-87b3-e1295c9631a3"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.39/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:59:03.832482 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:59:03.804947 194848 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-dns", Name:"dns-default-r2xl8", UID:"23402384-5d8d-47cb-9b23-e7128b9e0e74", APIVersion:"v1", ResourceVersion:"105924", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.39/23] from openshift-sdn Nov 16 14:59:03.833258 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:03.833216769Z" level=info msg="Got pod network &{Name:dns-default-r2xl8 Namespace:openshift-dns ID:be5627cc24a753ae3d8cee4423f7b8f43b59357470b4e170e0a36002384ff61e UID:23402384-5d8d-47cb-9b23-e7128b9e0e74 NetNS:/var/run/netns/9f330035-431d-46d8-87b3-e1295c9631a3 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:59:03.833397 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:03.833371370Z" level=info msg="Checking pod openshift-dns_dns-default-r2xl8 for CNI network multus-cni-network (type=multus)" Nov 16 14:59:03.834683 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:03.834652 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-dns/dns-default-r2xl8] Nov 16 14:59:03.835954 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:59:03.835864 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23402384_5d8d_47cb_9b23_e7128b9e0e74.slice/crio-be5627cc24a753ae3d8cee4423f7b8f43b59357470b4e170e0a36002384ff61e.scope WatchSource:0}: Error finding container be5627cc24a753ae3d8cee4423f7b8f43b59357470b4e170e0a36002384ff61e: Status 404 returned error &{%!s(*http.body=&{0xc002964be8 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:59:03.845742 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:03.845670854Z" level=info msg="Ran pod sandbox be5627cc24a753ae3d8cee4423f7b8f43b59357470b4e170e0a36002384ff61e with infra container: openshift-dns/dns-default-r2xl8/POD" id=9f16eb58-7b6f-4dc6-9e32-b5f17d55fad8 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:59:03.846942 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:03.846847962Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16f82af39788414dcd7f889b5ee7e07bdc9760ee3d0320c65dcfe14d1f6786ce" id=2d55991c-1fe6-4f10-9ab7-91bf99f156b9 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:59:03.847217 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:03.847186564Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16f82af39788414dcd7f889b5ee7e07bdc9760ee3d0320c65dcfe14d1f6786ce not found" id=2d55991c-1fe6-4f10-9ab7-91bf99f156b9 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:59:03.847968 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:03.847919469Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16f82af39788414dcd7f889b5ee7e07bdc9760ee3d0320c65dcfe14d1f6786ce" id=c510e9c6-d097-4e74-9e11-bfa3b486506e name=/runtime.v1.ImageService/PullImage Nov 16 14:59:03.850652 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:03.850617887Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16f82af39788414dcd7f889b5ee7e07bdc9760ee3d0320c65dcfe14d1f6786ce\"" Nov 16 14:59:04.574878 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:04.574840 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r2xl8" event=&{ID:23402384-5d8d-47cb-9b23-e7128b9e0e74 Type:ContainerStarted Data:be5627cc24a753ae3d8cee4423f7b8f43b59357470b4e170e0a36002384ff61e} Nov 16 14:59:05.133847 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:05.133782710Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16f82af39788414dcd7f889b5ee7e07bdc9760ee3d0320c65dcfe14d1f6786ce\"" Nov 16 14:59:08.393407 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.393317767Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16f82af39788414dcd7f889b5ee7e07bdc9760ee3d0320c65dcfe14d1f6786ce" id=c510e9c6-d097-4e74-9e11-bfa3b486506e name=/runtime.v1.ImageService/PullImage Nov 16 14:59:08.394228 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.394183672Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16f82af39788414dcd7f889b5ee7e07bdc9760ee3d0320c65dcfe14d1f6786ce" id=e8bbe86a-78f6-434e-898b-ea8042f965d9 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:59:08.395792 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.395720183Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:fe01bd29238389b59bbd9097cb03f9420af7ebb13d027463c4b263784dcfae6f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:16f82af39788414dcd7f889b5ee7e07bdc9760ee3d0320c65dcfe14d1f6786ce],Size_:402331891,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=e8bbe86a-78f6-434e-898b-ea8042f965d9 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:59:08.396993 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.396950491Z" level=info msg="Creating container: openshift-dns/dns-default-r2xl8/dns" id=866c5781-9df5-431c-aaf2-55234b7be7c4 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:59:08.397094 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.397063692Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:59:08.462079 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-146b3eafe51ca1fd30cc8443b2f665234c4e41c6e9f66fcfc29ab2f28affe370.scope. Nov 16 14:59:08.491589 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 146b3eafe51ca1fd30cc8443b2f665234c4e41c6e9f66fcfc29ab2f28affe370. Nov 16 14:59:08.525030 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:59:08.586347 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.586283778Z" level=info msg="Created container 146b3eafe51ca1fd30cc8443b2f665234c4e41c6e9f66fcfc29ab2f28affe370: openshift-dns/dns-default-r2xl8/dns" id=866c5781-9df5-431c-aaf2-55234b7be7c4 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:59:08.588069 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.588020590Z" level=info msg="Starting container: 146b3eafe51ca1fd30cc8443b2f665234c4e41c6e9f66fcfc29ab2f28affe370" id=8be69379-7fae-4a5d-8446-2d7fa8ec5b7c name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:59:08.600084 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.599997971Z" level=info msg="Started container" PID=195092 containerID=146b3eafe51ca1fd30cc8443b2f665234c4e41c6e9f66fcfc29ab2f28affe370 description=openshift-dns/dns-default-r2xl8/dns id=8be69379-7fae-4a5d-8446-2d7fa8ec5b7c name=/runtime.v1.RuntimeService/StartContainer sandboxID=be5627cc24a753ae3d8cee4423f7b8f43b59357470b4e170e0a36002384ff61e Nov 16 14:59:08.627409 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.627346157Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=2fe636d2-6767-4aaa-b026-483733c8c694 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:59:08.627705 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.627662560Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=2fe636d2-6767-4aaa-b026-483733c8c694 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:59:08.633371 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.633329298Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=d59bee19-979f-4b43-8aca-9d660e9a3b52 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:59:08.633584 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.633551400Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=d59bee19-979f-4b43-8aca-9d660e9a3b52 name=/runtime.v1.ImageService/ImageStatus Nov 16 14:59:08.634562 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.634468206Z" level=info msg="Creating container: openshift-dns/dns-default-r2xl8/kube-rbac-proxy" id=7a810fd0-9054-44e4-9882-58b2b1c5246c name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:59:08.634667 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.634578507Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:59:08.695325 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-841f93d0a0993e82b483c2477540cf2f9f38494477f8a90d8d8ef2bbc5eb37fd.scope. Nov 16 14:59:08.724175 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 841f93d0a0993e82b483c2477540cf2f9f38494477f8a90d8d8ef2bbc5eb37fd. Nov 16 14:59:08.755081 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:59:08.828056 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.827945021Z" level=info msg="Created container 841f93d0a0993e82b483c2477540cf2f9f38494477f8a90d8d8ef2bbc5eb37fd: openshift-dns/dns-default-r2xl8/kube-rbac-proxy" id=7a810fd0-9054-44e4-9882-58b2b1c5246c name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:59:08.828622 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.828587425Z" level=info msg="Starting container: 841f93d0a0993e82b483c2477540cf2f9f38494477f8a90d8d8ef2bbc5eb37fd" id=2cb2c5b8-e2b0-4c18-b7c0-345418452dec name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:59:08.844772 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:08.844684835Z" level=info msg="Started container" PID=195183 containerID=841f93d0a0993e82b483c2477540cf2f9f38494477f8a90d8d8ef2bbc5eb37fd description=openshift-dns/dns-default-r2xl8/kube-rbac-proxy id=2cb2c5b8-e2b0-4c18-b7c0-345418452dec name=/runtime.v1.RuntimeService/StartContainer sandboxID=be5627cc24a753ae3d8cee4423f7b8f43b59357470b4e170e0a36002384ff61e Nov 16 14:59:09.600520 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:09.600436 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r2xl8" event=&{ID:23402384-5d8d-47cb-9b23-e7128b9e0e74 Type:ContainerStarted Data:146b3eafe51ca1fd30cc8443b2f665234c4e41c6e9f66fcfc29ab2f28affe370} Nov 16 14:59:09.600520 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:09.600522 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r2xl8" event=&{ID:23402384-5d8d-47cb-9b23-e7128b9e0e74 Type:ContainerStarted Data:841f93d0a0993e82b483c2477540cf2f9f38494477f8a90d8d8ef2bbc5eb37fd} Nov 16 14:59:09.601315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:09.600616 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-dns/dns-default-r2xl8" Nov 16 14:59:19.990363 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:19.990326 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r2xl8" Nov 16 14:59:23.831502 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:23.831423197Z" level=info msg="Stopping pod sandbox: 9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f" id=bbd7cd09-1092-4628-afce-10e9147d60b9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:59:23.831502 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:23.831499398Z" level=info msg="Stopped pod sandbox (already stopped): 9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f" id=bbd7cd09-1092-4628-afce-10e9147d60b9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:59:23.831981 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:23.831748600Z" level=info msg="Removing pod sandbox: 9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f" id=f78a1ef0-0b0b-42ed-ac5a-d0924c0a928f name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:59:23.849593 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:23.849520831Z" level=info msg="Removed pod sandbox: 9ee508ee69d9060de8b295cdd23c94bf5bff20f551d105649663d1c3f152a70f" id=f78a1ef0-0b0b-42ed-ac5a-d0924c0a928f name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 14:59:44.427943 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:44.427864 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-network-diagnostics/network-check-target-299n8] Nov 16 14:59:44.428559 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:44.428169 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-network-diagnostics/network-check-target-299n8" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 containerName="network-check-target-container" containerID="cri-o://5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c" gracePeriod=10 Nov 16 14:59:44.428958 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:44.428899279Z" level=info msg="Stopping container: 5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c (timeout: 10s)" id=50a70851-b9b1-4d5d-a79e-fe5144df7382 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:59:44.462658 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c.scope: Succeeded. Nov 16 14:59:44.463350 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c.scope: Consumed 965ms CPU time Nov 16 14:59:44.463657 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c.scope: Succeeded. Nov 16 14:59:44.464245 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c.scope: Consumed 29ms CPU time Nov 16 14:59:44.600583 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-81891a6053794123d548d8fe8f5cc658988b82c8d0b932b740b00aade51bb1f0-merged.mount: Succeeded. Nov 16 14:59:44.601417 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-81891a6053794123d548d8fe8f5cc658988b82c8d0b932b740b00aade51bb1f0-merged.mount: Consumed 0 CPU time Nov 16 14:59:44.713643 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:44.713512 1563 generic.go:296] "Generic (PLEG): container finished" podID=1bd11256-096f-4190-9ee6-f23c6eaabf65 containerID="5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c" exitCode=2 Nov 16 14:59:44.713643 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:44.713573 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-299n8" event=&{ID:1bd11256-096f-4190-9ee6-f23c6eaabf65 Type:ContainerDied Data:5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c} Nov 16 14:59:45.237387 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:45.237311120Z" level=info msg="Stopped container 5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c: openshift-network-diagnostics/network-check-target-299n8/network-check-target-container" id=50a70851-b9b1-4d5d-a79e-fe5144df7382 name=/runtime.v1.RuntimeService/StopContainer Nov 16 14:59:45.238183 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:45.238031725Z" level=info msg="Stopping pod sandbox: b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76" id=be9c0889-cc1a-4a31-8391-46d0b66e6c01 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:59:45.238474 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:45.238416428Z" level=info msg="Got pod network &{Name:network-check-target-299n8 Namespace:openshift-network-diagnostics ID:b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76 UID:1bd11256-096f-4190-9ee6-f23c6eaabf65 NetNS:/var/run/netns/dda9a386-e7c6-4a1c-a6c3-1e31c9168371 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:59:45.238675 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:45.238632030Z" level=info msg="Deleting pod openshift-network-diagnostics_network-check-target-299n8 from CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:59:45.308071 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00245|connmgr|INFO|br0<->unix#554: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:59:45.337120 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00246|connmgr|INFO|br0<->unix#557: 4 flow_mods in the last 0 s (4 deletes) Nov 16 14:59:45.367186 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00247|bridge|INFO|bridge br0: deleted interface veth886b239b on port 5 Nov 16 14:59:45.373162 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth886b239b left promiscuous mode Nov 16 14:59:45.382421 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:59:45Z [verbose] Del: openshift-network-diagnostics:network-check-target-299n8:1bd11256-096f-4190-9ee6-f23c6eaabf65:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 14:59:45.391990 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-dda9a386\x2de7c6\x2d4a1c\x2da6c3\x2d1e31c9168371.mount: Succeeded. Nov 16 14:59:45.392855 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-dda9a386\x2de7c6\x2d4a1c\x2da6c3\x2d1e31c9168371.mount: Consumed 0 CPU time Nov 16 14:59:45.410455 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-dda9a386\x2de7c6\x2d4a1c\x2da6c3\x2d1e31c9168371.mount: Succeeded. Nov 16 14:59:45.411258 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-dda9a386\x2de7c6\x2d4a1c\x2da6c3\x2d1e31c9168371.mount: Consumed 0 CPU time Nov 16 14:59:45.446188 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-dda9a386\x2de7c6\x2d4a1c\x2da6c3\x2d1e31c9168371.mount: Succeeded. Nov 16 14:59:45.446937 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-dda9a386\x2de7c6\x2d4a1c\x2da6c3\x2d1e31c9168371.mount: Consumed 0 CPU time Nov 16 14:59:45.470673 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76-userdata-shm.mount: Succeeded. Nov 16 14:59:45.471454 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76-userdata-shm.mount: Consumed 0 CPU time Nov 16 14:59:45.479598 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-36be7d7d4ff0c11d87ca7ad74bf3c5345e8551ccef7ed9db9eb1f7e95395b434-merged.mount: Succeeded. Nov 16 14:59:45.480585 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-36be7d7d4ff0c11d87ca7ad74bf3c5345e8551ccef7ed9db9eb1f7e95395b434-merged.mount: Consumed 0 CPU time Nov 16 14:59:45.808896 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:45.808798061Z" level=info msg="Stopped pod sandbox: b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76" id=be9c0889-cc1a-4a31-8391-46d0b66e6c01 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 14:59:46.005558 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:46.005455 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdfrr\" (UniqueName: \"kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr\") pod \"1bd11256-096f-4190-9ee6-f23c6eaabf65\" (UID: \"1bd11256-096f-4190-9ee6-f23c6eaabf65\") " Nov 16 14:59:46.016986 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-1bd11256\x2d096f\x2d4190\x2d9ee6\x2df23c6eaabf65-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dzdfrr.mount: Succeeded. Nov 16 14:59:46.017898 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-1bd11256\x2d096f\x2d4190\x2d9ee6\x2df23c6eaabf65-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dzdfrr.mount: Consumed 0 CPU time Nov 16 14:59:46.024494 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:46.024436 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr" (OuterVolumeSpecName: "kube-api-access-zdfrr") pod "1bd11256-096f-4190-9ee6-f23c6eaabf65" (UID: "1bd11256-096f-4190-9ee6-f23c6eaabf65"). InnerVolumeSpecName "kube-api-access-zdfrr". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 14:59:46.106392 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:46.106239 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-zdfrr\" (UniqueName: \"kubernetes.io/projected/1bd11256-096f-4190-9ee6-f23c6eaabf65-kube-api-access-zdfrr\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 14:59:46.602098 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod1bd11256_096f_4190_9ee6_f23c6eaabf65.slice. Nov 16 14:59:46.604338 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod1bd11256_096f_4190_9ee6_f23c6eaabf65.slice: Consumed 995ms CPU time Nov 16 14:59:46.721941 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:46.721871 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-299n8" event=&{ID:1bd11256-096f-4190-9ee6-f23c6eaabf65 Type:ContainerDied Data:b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76} Nov 16 14:59:46.722232 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:46.721962 1563 scope.go:110] "RemoveContainer" containerID="5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c" Nov 16 14:59:46.723390 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:46.723311209Z" level=info msg="Removing container: 5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c" id=7163e8c6-81d3-49bb-b829-0533bc197aad name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:59:46.757761 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:46.757686870Z" level=info msg="Removed container 5cbf95110e8403318c82f83b5f8d0842b343b9fd0617bf4638309b5f08b9ff9c: openshift-network-diagnostics/network-check-target-299n8/network-check-target-container" id=7163e8c6-81d3-49bb-b829-0533bc197aad name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 14:59:46.758402 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:46.758366 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-network-diagnostics/network-check-target-299n8] Nov 16 14:59:46.768172 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:46.768123 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-network-diagnostics/network-check-target-299n8] Nov 16 14:59:46.820061 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:46.820023 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-network-diagnostics/network-check-target-b5srf] Nov 16 14:59:46.820263 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:46.820089 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 14:59:46.820263 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 14:59:46.820235 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="1bd11256-096f-4190-9ee6-f23c6eaabf65" containerName="network-check-target-container" Nov 16 14:59:46.820263 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:46.820252 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bd11256-096f-4190-9ee6-f23c6eaabf65" containerName="network-check-target-container" Nov 16 14:59:46.820368 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:46.820328 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="1bd11256-096f-4190-9ee6-f23c6eaabf65" containerName="network-check-target-container" Nov 16 14:59:46.830374 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-poda77123b5_b068_4e08_bbb0_243a4ef22749.slice. Nov 16 14:59:47.015833 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:47.015684 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmr5x\" (UniqueName: \"kubernetes.io/projected/a77123b5-b068-4e08-bbb0-243a4ef22749-kube-api-access-fmr5x\") pod \"network-check-target-b5srf\" (UID: \"a77123b5-b068-4e08-bbb0-243a4ef22749\") " pod="openshift-network-diagnostics/network-check-target-b5srf" Nov 16 14:59:47.116287 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:47.116238 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-fmr5x\" (UniqueName: \"kubernetes.io/projected/a77123b5-b068-4e08-bbb0-243a4ef22749-kube-api-access-fmr5x\") pod \"network-check-target-b5srf\" (UID: \"a77123b5-b068-4e08-bbb0-243a4ef22749\") " pod="openshift-network-diagnostics/network-check-target-b5srf" Nov 16 14:59:47.141320 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:47.141270 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmr5x\" (UniqueName: \"kubernetes.io/projected/a77123b5-b068-4e08-bbb0-243a4ef22749-kube-api-access-fmr5x\") pod \"network-check-target-b5srf\" (UID: \"a77123b5-b068-4e08-bbb0-243a4ef22749\") " pod="openshift-network-diagnostics/network-check-target-b5srf" Nov 16 14:59:47.141496 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:47.141457 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b5srf" Nov 16 14:59:47.142061 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:47.141994289Z" level=info msg="Running pod sandbox: openshift-network-diagnostics/network-check-target-b5srf/POD" id=7626a082-21c0-4304-883d-187c2aad6b23 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:59:47.142428 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:47.142083090Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:59:47.583693 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:47.583607344Z" level=info msg="Got pod network &{Name:network-check-target-b5srf Namespace:openshift-network-diagnostics ID:ae1cfc93beb626b5dc81aa6ad7bb656180ea0cc21e44126bd3ff0f5f3cafbe56 UID:a77123b5-b068-4e08-bbb0-243a4ef22749 NetNS:/var/run/netns/c43103b9-21bd-48f3-afb5-b4cb7842edac Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:59:47.583693 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:47.583649944Z" level=info msg="Adding pod openshift-network-diagnostics_network-check-target-b5srf to CNI network \"multus-cni-network\" (type=multus)" Nov 16 14:59:47.629244 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 14:59:47.657396 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): vethedd6543d: link is not ready Nov 16 14:59:47.657487 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethedd6543d: link becomes ready Nov 16 14:59:47.657526 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 14:59:47.664704 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[196103]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 14:59:47.664891 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[196103]: Could not generate persistent MAC address for vethedd6543d: No such file or directory Nov 16 14:59:47.665225 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610787.6652] device (vethedd6543d): carrier: link connected Nov 16 14:59:47.665594 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610787.6655] manager: (vethedd6543d): new Veth device (/org/freedesktop/NetworkManager/Devices/86) Nov 16 14:59:47.685973 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668610787.6859] manager: (vethedd6543d): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/87) Nov 16 14:59:47.691940 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00248|bridge|INFO|bridge br0: added interface vethedd6543d on port 41 Nov 16 14:59:47.692386 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethedd6543d entered promiscuous mode Nov 16 14:59:47.724687 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00249|connmgr|INFO|br0<->unix#560: 5 flow_mods in the last 0 s (5 adds) Nov 16 14:59:47.771036 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00250|connmgr|INFO|br0<->unix#563: 2 flow_mods in the last 0 s (2 deletes) Nov 16 14:59:48.598515 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:48.598410 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=1bd11256-096f-4190-9ee6-f23c6eaabf65 path="/var/lib/kubelet/pods/1bd11256-096f-4190-9ee6-f23c6eaabf65/volumes" Nov 16 14:59:49.437821 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T14:59:49Z [verbose] Add: openshift-network-diagnostics:network-check-target-b5srf:a77123b5-b068-4e08-bbb0-243a4ef22749:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/c43103b9-21bd-48f3-afb5-b4cb7842edac"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.40/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 14:59:49.437821 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 14:59:49.412935 196086 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-network-diagnostics", Name:"network-check-target-b5srf", UID:"a77123b5-b068-4e08-bbb0-243a4ef22749", APIVersion:"v1", ResourceVersion:"106340", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.40/23] from openshift-sdn Nov 16 14:59:49.438390 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:49.438344434Z" level=info msg="Got pod network &{Name:network-check-target-b5srf Namespace:openshift-network-diagnostics ID:ae1cfc93beb626b5dc81aa6ad7bb656180ea0cc21e44126bd3ff0f5f3cafbe56 UID:a77123b5-b068-4e08-bbb0-243a4ef22749 NetNS:/var/run/netns/c43103b9-21bd-48f3-afb5-b4cb7842edac Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 14:59:49.438543 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:49.438519135Z" level=info msg="Checking pod openshift-network-diagnostics_network-check-target-b5srf for CNI network multus-cni-network (type=multus)" Nov 16 14:59:49.439360 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:49.439325 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-network-diagnostics/network-check-target-b5srf] Nov 16 14:59:49.448055 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 14:59:49.448021 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda77123b5_b068_4e08_bbb0_243a4ef22749.slice/crio-ae1cfc93beb626b5dc81aa6ad7bb656180ea0cc21e44126bd3ff0f5f3cafbe56.scope WatchSource:0}: Error finding container ae1cfc93beb626b5dc81aa6ad7bb656180ea0cc21e44126bd3ff0f5f3cafbe56: Status 404 returned error &{%!s(*http.body=&{0xc002e96108 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 14:59:49.450205 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:49.450160024Z" level=info msg="Ran pod sandbox ae1cfc93beb626b5dc81aa6ad7bb656180ea0cc21e44126bd3ff0f5f3cafbe56 with infra container: openshift-network-diagnostics/network-check-target-b5srf/POD" id=7626a082-21c0-4304-883d-187c2aad6b23 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 14:59:49.451192 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:49.451158531Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15c37c879ec4f2e38820ab273868c68406ab246c0c0b6888f2ec39a690734ac9" id=0747a154-bd0d-42b1-8f59-dbf8f5a88ced name=/runtime.v1.ImageService/ImageStatus Nov 16 14:59:49.451451 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:49.451421833Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15c37c879ec4f2e38820ab273868c68406ab246c0c0b6888f2ec39a690734ac9 not found" id=0747a154-bd0d-42b1-8f59-dbf8f5a88ced name=/runtime.v1.ImageService/ImageStatus Nov 16 14:59:49.452314 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:49.452278240Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15c37c879ec4f2e38820ab273868c68406ab246c0c0b6888f2ec39a690734ac9" id=481be5d5-c621-49c6-8fa5-c4bc526c41e9 name=/runtime.v1.ImageService/PullImage Nov 16 14:59:49.455147 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:49.455099061Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15c37c879ec4f2e38820ab273868c68406ab246c0c0b6888f2ec39a690734ac9\"" Nov 16 14:59:49.734656 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:49.734446 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b5srf" event=&{ID:a77123b5-b068-4e08-bbb0-243a4ef22749 Type:ContainerStarted Data:ae1cfc93beb626b5dc81aa6ad7bb656180ea0cc21e44126bd3ff0f5f3cafbe56} Nov 16 14:59:50.718708 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:50.718613460Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15c37c879ec4f2e38820ab273868c68406ab246c0c0b6888f2ec39a690734ac9\"" Nov 16 14:59:55.433581 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:55.433456266Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15c37c879ec4f2e38820ab273868c68406ab246c0c0b6888f2ec39a690734ac9" id=481be5d5-c621-49c6-8fa5-c4bc526c41e9 name=/runtime.v1.ImageService/PullImage Nov 16 14:59:55.434508 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:55.434463773Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15c37c879ec4f2e38820ab273868c68406ab246c0c0b6888f2ec39a690734ac9" id=0cc06c9c-d371-445d-917c-3721d2f6137b name=/runtime.v1.ImageService/ImageStatus Nov 16 14:59:55.436341 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:55.436201186Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:dfb235eaf4d1a98a4934fd70cfd4952b38227e0cc4226a8242ab2033f4d6bc18,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:15c37c879ec4f2e38820ab273868c68406ab246c0c0b6888f2ec39a690734ac9],Size_:496052558,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=0cc06c9c-d371-445d-917c-3721d2f6137b name=/runtime.v1.ImageService/ImageStatus Nov 16 14:59:55.437360 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:55.437316694Z" level=info msg="Creating container: openshift-network-diagnostics/network-check-target-b5srf/network-check-target-container" id=0b3eb234-ce22-4068-882d-4f3b69bd7087 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:59:55.437468 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:55.437424795Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 14:59:55.500490 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-d6fb83c7ecdf312a57aa1ee3a20950c56a6379522f6df431861362c140d15133.scope. Nov 16 14:59:55.548974 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container d6fb83c7ecdf312a57aa1ee3a20950c56a6379522f6df431861362c140d15133. Nov 16 14:59:55.593500 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 14:59:55.692591 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:55.692426346Z" level=info msg="Created container d6fb83c7ecdf312a57aa1ee3a20950c56a6379522f6df431861362c140d15133: openshift-network-diagnostics/network-check-target-b5srf/network-check-target-container" id=0b3eb234-ce22-4068-882d-4f3b69bd7087 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 14:59:55.693063 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:55.692968050Z" level=info msg="Starting container: d6fb83c7ecdf312a57aa1ee3a20950c56a6379522f6df431861362c140d15133" id=52af51ab-4920-4805-9c62-283c18c2a2f5 name=/runtime.v1.RuntimeService/StartContainer Nov 16 14:59:55.706891 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 14:59:55.706803950Z" level=info msg="Started container" PID=196317 containerID=d6fb83c7ecdf312a57aa1ee3a20950c56a6379522f6df431861362c140d15133 description=openshift-network-diagnostics/network-check-target-b5srf/network-check-target-container id=52af51ab-4920-4805-9c62-283c18c2a2f5 name=/runtime.v1.RuntimeService/StartContainer sandboxID=ae1cfc93beb626b5dc81aa6ad7bb656180ea0cc21e44126bd3ff0f5f3cafbe56 Nov 16 14:59:55.757368 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:55.757327 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b5srf" event=&{ID:a77123b5-b068-4e08-bbb0-243a4ef22749 Type:ContainerStarted Data:d6fb83c7ecdf312a57aa1ee3a20950c56a6379522f6df431861362c140d15133} Nov 16 14:59:55.757833 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 14:59:55.757467 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-network-diagnostics/network-check-target-b5srf" Nov 16 15:00:08.720406 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.dvHHWh.mount: Succeeded. Nov 16 15:00:23.853533 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:00:23.853469278Z" level=info msg="Stopping pod sandbox: b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76" id=b66bd25e-133c-4673-8559-1097c8061135 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:00:23.853533 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:00:23.853529979Z" level=info msg="Stopped pod sandbox (already stopped): b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76" id=b66bd25e-133c-4673-8559-1097c8061135 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:00:23.854086 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:00:23.853781181Z" level=info msg="Removing pod sandbox: b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76" id=373f3ec5-eca9-49da-914e-a69813b72f84 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:00:23.872578 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:00:23.872537026Z" level=info msg="Removed pod sandbox: b936c10b1f5f3b323981c6abfea349ba9a62e863efc491932827c0f311fbde76" id=373f3ec5-eca9-49da-914e-a69813b72f84 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:00:27.143434 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:00:27.143375 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b5srf" Nov 16 15:01:48.719991 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.50oMSW.mount: Succeeded. Nov 16 15:03:20.068064 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:20.068022 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-machine-config-operator/machine-config-daemon-kxqnl] Nov 16 15:03:20.068854 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:20.068675 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" podUID=2e56696b-4c5c-4c9f-8681-c75ba7546700 containerName="machine-config-daemon" containerID="cri-o://ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588" gracePeriod=600 Nov 16 15:03:20.068854 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:20.068753 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" podUID=2e56696b-4c5c-4c9f-8681-c75ba7546700 containerName="oauth-proxy" containerID="cri-o://f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403" gracePeriod=600 Nov 16 15:03:20.069187 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:20.069110314Z" level=info msg="Stopping container: ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588 (timeout: 600s)" id=666b0ce8-60e6-4b9b-9227-5ef26d328842 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:03:20.069548 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:20.069111014Z" level=info msg="Stopping container: f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403 (timeout: 600s)" id=93b53ebd-40ed-47c2-a2db-aacfd88e54de name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:03:20.092248 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403.scope: Succeeded. Nov 16 15:03:20.092935 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403.scope: Consumed 33ms CPU time Nov 16 15:03:20.095394 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403.scope: Succeeded. Nov 16 15:03:20.096002 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403.scope: Consumed 7.077s CPU time Nov 16 15:03:20.102650 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope: Succeeded. Nov 16 15:03:20.103249 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope: Consumed 54ms CPU time Nov 16 15:03:20.259341 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-b1ac0ecd713367622ce3666271f842eecdaea351cf039e8703d7ac1e8d5c44b6-merged.mount: Succeeded. Nov 16 15:03:20.260524 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-b1ac0ecd713367622ce3666271f842eecdaea351cf039e8703d7ac1e8d5c44b6-merged.mount: Consumed 0 CPU time Nov 16 15:03:20.261333 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope: Succeeded. Nov 16 15:03:20.262099 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope: Consumed 7.603s CPU time Nov 16 15:03:20.265476 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:03:20.265442 1563 remote_runtime.go:479] "StopContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to get container 'stopped' status ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588: failed to get container stopped status: `/usr/bin/runc --root /run/runc kill -a ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588 9` failed: time=\"2022-11-16T15:03:20Z\" level=error msg=\"lstat /sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e56696b_4c5c_4c9f_8681_c75ba7546700.slice/crio-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope: no such file or directory\"\n (exit status 1)" containerID="ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588" Nov 16 15:03:20.265611 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:03:20.265527 1563 kuberuntime_container.go:734] "Container termination failed with gracePeriod" err="rpc error: code = Unknown desc = failed to get container 'stopped' status ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588: failed to get container stopped status: `/usr/bin/runc --root /run/runc kill -a ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588 9` failed: time=\"2022-11-16T15:03:20Z\" level=error msg=\"lstat /sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e56696b_4c5c_4c9f_8681_c75ba7546700.slice/crio-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope: no such file or directory\"\n (exit status 1)" pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" podUID=2e56696b-4c5c-4c9f-8681-c75ba7546700 containerName="machine-config-daemon" containerID="cri-o://ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588" gracePeriod=600 Nov 16 15:03:20.265611 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:03:20.265571 1563 kuberuntime_container.go:762] "Kill container failed" err="rpc error: code = Unknown desc = failed to get container 'stopped' status ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588: failed to get container stopped status: `/usr/bin/runc --root /run/runc kill -a ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588 9` failed: time=\"2022-11-16T15:03:20Z\" level=error msg=\"lstat /sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e56696b_4c5c_4c9f_8681_c75ba7546700.slice/crio-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope: no such file or directory\"\n (exit status 1)" pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" podUID=2e56696b-4c5c-4c9f-8681-c75ba7546700 containerName="machine-config-daemon" containerID={Type:cri-o ID:ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588} Nov 16 15:03:20.456283 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:20.456173609Z" level=info msg="Stopped container f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403: openshift-machine-config-operator/machine-config-daemon-kxqnl/oauth-proxy" id=93b53ebd-40ed-47c2-a2db-aacfd88e54de name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:03:20.456734 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:20.456694813Z" level=info msg="Stopping pod sandbox: 1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c" id=c04b839c-024e-43e2-8504-221bf7cb653b name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:03:20.466485 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-25c0cbec\x2d0207\x2d44a3\x2d880b\x2d563923ad3a48.mount: Succeeded. Nov 16 15:03:20.467560 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-25c0cbec\x2d0207\x2d44a3\x2d880b\x2d563923ad3a48.mount: Consumed 0 CPU time Nov 16 15:03:20.483652 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:20.483624 1563 generic.go:296] "Generic (PLEG): container finished" podID=2e56696b-4c5c-4c9f-8681-c75ba7546700 containerID="f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403" exitCode=143 Nov 16 15:03:20.483652 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:20.483658 1563 generic.go:296] "Generic (PLEG): container finished" podID=2e56696b-4c5c-4c9f-8681-c75ba7546700 containerID="ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588" exitCode=0 Nov 16 15:03:20.483831 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:20.483693 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" event=&{ID:2e56696b-4c5c-4c9f-8681-c75ba7546700 Type:ContainerDied Data:f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403} Nov 16 15:03:20.483831 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:20.483721 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" event=&{ID:2e56696b-4c5c-4c9f-8681-c75ba7546700 Type:ContainerDied Data:ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588} Nov 16 15:03:20.487648 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-25c0cbec\x2d0207\x2d44a3\x2d880b\x2d563923ad3a48.mount: Succeeded. Nov 16 15:03:20.488400 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-25c0cbec\x2d0207\x2d44a3\x2d880b\x2d563923ad3a48.mount: Consumed 0 CPU time Nov 16 15:03:20.502672 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:20.502553056Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=581e97e0-364d-44a8-9d78-5aa51e45b14f name=/runtime.v1.ImageService/ImageStatus Nov 16 15:03:20.612599 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:20.502805658Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=581e97e0-364d-44a8-9d78-5aa51e45b14f name=/runtime.v1.ImageService/ImageStatus Nov 16 15:03:20.521459 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-25c0cbec\x2d0207\x2d44a3\x2d880b\x2d563923ad3a48.mount: Succeeded. Nov 16 15:03:20.522222 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-25c0cbec\x2d0207\x2d44a3\x2d880b\x2d563923ad3a48.mount: Consumed 0 CPU time Nov 16 15:03:20.541307 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c-userdata-shm.mount: Succeeded. Nov 16 15:03:20.542249 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:03:20.625999 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:20.625914979Z" level=info msg="Stopped pod sandbox: 1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c" id=c04b839c-024e-43e2-8504-221bf7cb653b name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:03:20.626308 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:03:20.626277 1563 kubelet.go:1859] failed to "KillContainer" for "machine-config-daemon" with KillContainerError: "rpc error: code = Unknown desc = failed to get container 'stopped' status ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588: failed to get container stopped status: `/usr/bin/runc --root /run/runc kill -a ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588 9` failed: time=\"2022-11-16T15:03:20Z\" level=error msg=\"lstat /sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e56696b_4c5c_4c9f_8681_c75ba7546700.slice/crio-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope: no such file or directory\"\n (exit status 1)" Nov 16 15:03:20.626453 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:03:20.626336 1563 pod_workers.go:951] "Error syncing pod, skipping" err="failed to \"KillContainer\" for \"machine-config-daemon\" with KillContainerError: \"rpc error: code = Unknown desc = failed to get container 'stopped' status ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588: failed to get container stopped status: `/usr/bin/runc --root /run/runc kill -a ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588 9` failed: time=\\\"2022-11-16T15:03:20Z\\\" level=error msg=\\\"lstat /sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e56696b_4c5c_4c9f_8681_c75ba7546700.slice/crio-ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588.scope: no such file or directory\\\"\\n (exit status 1)\"" pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" podUID=2e56696b-4c5c-4c9f-8681-c75ba7546700 Nov 16 15:03:21.260004 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-41ff769a0822341dc13944e6aaca046e1e119b14c9e16135bb486a679a02538a-merged.mount: Succeeded. Nov 16 15:03:21.261077 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-41ff769a0822341dc13944e6aaca046e1e119b14c9e16135bb486a679a02538a-merged.mount: Consumed 0 CPU time Nov 16 15:03:21.488696 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.488656 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-kxqnl" event=&{ID:2e56696b-4c5c-4c9f-8681-c75ba7546700 Type:ContainerDied Data:1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c} Nov 16 15:03:21.488696 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.488696 1563 pod_container_deletor.go:79] "Container not found in pod's containers" containerID="1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c" Nov 16 15:03:21.489240 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:21.488863433Z" level=info msg="Stopping pod sandbox: 1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c" id=5bfee8e5-c8de-4f66-9277-3c48ea4da5a9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:03:21.489240 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:21.488905433Z" level=info msg="Stopped pod sandbox (already stopped): 1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c" id=5bfee8e5-c8de-4f66-9277-3c48ea4da5a9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:03:21.548330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.548210 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/2e56696b-4c5c-4c9f-8681-c75ba7546700-rootfs\") pod \"2e56696b-4c5c-4c9f-8681-c75ba7546700\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " Nov 16 15:03:21.548330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.548287 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e56696b-4c5c-4c9f-8681-c75ba7546700-proxy-tls\") pod \"2e56696b-4c5c-4c9f-8681-c75ba7546700\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " Nov 16 15:03:21.548330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.548324 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"cookie-secret\" (UniqueName: \"kubernetes.io/secret/2e56696b-4c5c-4c9f-8681-c75ba7546700-cookie-secret\") pod \"2e56696b-4c5c-4c9f-8681-c75ba7546700\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " Nov 16 15:03:21.548557 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.548325 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/2e56696b-4c5c-4c9f-8681-c75ba7546700-rootfs" (OuterVolumeSpecName: "rootfs") pod "2e56696b-4c5c-4c9f-8681-c75ba7546700" (UID: "2e56696b-4c5c-4c9f-8681-c75ba7546700"). InnerVolumeSpecName "rootfs". PluginName "kubernetes.io/host-path", VolumeGidValue "" Nov 16 15:03:21.548557 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.548389 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57s25\" (UniqueName: \"kubernetes.io/projected/2e56696b-4c5c-4c9f-8681-c75ba7546700-kube-api-access-57s25\") pod \"2e56696b-4c5c-4c9f-8681-c75ba7546700\" (UID: \"2e56696b-4c5c-4c9f-8681-c75ba7546700\") " Nov 16 15:03:21.548557 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.548541 1563 reconciler.go:300] "Volume detached for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/2e56696b-4c5c-4c9f-8681-c75ba7546700-rootfs\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:03:21.560320 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-2e56696b\x2d4c5c\x2d4c9f\x2d8681\x2dc75ba7546700-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d57s25.mount: Succeeded. Nov 16 15:03:21.561261 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-2e56696b\x2d4c5c\x2d4c9f\x2d8681\x2dc75ba7546700-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d57s25.mount: Consumed 0 CPU time Nov 16 15:03:21.561282 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-2e56696b\x2d4c5c\x2d4c9f\x2d8681\x2dc75ba7546700-volumes-kubernetes.io\x7esecret-cookie\x2dsecret.mount: Succeeded. Nov 16 15:03:21.561956 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-2e56696b\x2d4c5c\x2d4c9f\x2d8681\x2dc75ba7546700-volumes-kubernetes.io\x7esecret-cookie\x2dsecret.mount: Consumed 0 CPU time Nov 16 15:03:21.561971 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-2e56696b\x2d4c5c\x2d4c9f\x2d8681\x2dc75ba7546700-volumes-kubernetes.io\x7esecret-proxy\x2dtls.mount: Succeeded. Nov 16 15:03:21.562756 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-2e56696b\x2d4c5c\x2d4c9f\x2d8681\x2dc75ba7546700-volumes-kubernetes.io\x7esecret-proxy\x2dtls.mount: Consumed 0 CPU time Nov 16 15:03:21.565457 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.565414 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e56696b-4c5c-4c9f-8681-c75ba7546700-kube-api-access-57s25" (OuterVolumeSpecName: "kube-api-access-57s25") pod "2e56696b-4c5c-4c9f-8681-c75ba7546700" (UID: "2e56696b-4c5c-4c9f-8681-c75ba7546700"). InnerVolumeSpecName "kube-api-access-57s25". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:03:21.566410 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.566363 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e56696b-4c5c-4c9f-8681-c75ba7546700-cookie-secret" (OuterVolumeSpecName: "cookie-secret") pod "2e56696b-4c5c-4c9f-8681-c75ba7546700" (UID: "2e56696b-4c5c-4c9f-8681-c75ba7546700"). InnerVolumeSpecName "cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:03:21.575437 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.575386 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e56696b-4c5c-4c9f-8681-c75ba7546700-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2e56696b-4c5c-4c9f-8681-c75ba7546700" (UID: "2e56696b-4c5c-4c9f-8681-c75ba7546700"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:03:21.649673 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.649593 1563 reconciler.go:300] "Volume detached for volume \"cookie-secret\" (UniqueName: \"kubernetes.io/secret/2e56696b-4c5c-4c9f-8681-c75ba7546700-cookie-secret\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:03:21.649673 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.649639 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-57s25\" (UniqueName: \"kubernetes.io/projected/2e56696b-4c5c-4c9f-8681-c75ba7546700-kube-api-access-57s25\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:03:21.649673 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:21.649660 1563 reconciler.go:300] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e56696b-4c5c-4c9f-8681-c75ba7546700-proxy-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:03:22.501650 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod2e56696b_4c5c_4c9f_8681_c75ba7546700.slice. Nov 16 15:03:22.501729 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod2e56696b_4c5c_4c9f_8681_c75ba7546700.slice: Consumed 14.769s CPU time Nov 16 15:03:22.526185 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.526086 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-machine-config-operator/machine-config-daemon-kxqnl] Nov 16 15:03:22.534341 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.534306 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-machine-config-operator/machine-config-daemon-kxqnl] Nov 16 15:03:22.579858 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.579779 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-machine-config-operator/machine-config-daemon-q7lkj] Nov 16 15:03:22.579858 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.579856 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 15:03:22.580067 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:03:22.579958 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="2e56696b-4c5c-4c9f-8681-c75ba7546700" containerName="machine-config-daemon" Nov 16 15:03:22.580067 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.579973 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e56696b-4c5c-4c9f-8681-c75ba7546700" containerName="machine-config-daemon" Nov 16 15:03:22.580067 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:03:22.579988 1563 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="2e56696b-4c5c-4c9f-8681-c75ba7546700" containerName="oauth-proxy" Nov 16 15:03:22.580067 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.579997 1563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e56696b-4c5c-4c9f-8681-c75ba7546700" containerName="oauth-proxy" Nov 16 15:03:22.580216 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.580082 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="2e56696b-4c5c-4c9f-8681-c75ba7546700" containerName="machine-config-daemon" Nov 16 15:03:22.580216 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.580099 1563 memory_manager.go:345] "RemoveStaleState removing state" podUID="2e56696b-4c5c-4c9f-8681-c75ba7546700" containerName="oauth-proxy" Nov 16 15:03:22.590838 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod29d0f8e3_e33b_4c9f_a290_193626492310.slice. Nov 16 15:03:22.593640 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:22.593528395Z" level=info msg="Stopping pod sandbox: 1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c" id=87332139-6469-41cf-aba9-69190852e6cf name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:03:22.593640 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:22.593600296Z" level=info msg="Stopped pod sandbox (already stopped): 1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c" id=87332139-6469-41cf-aba9-69190852e6cf name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:03:22.602453 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.595918 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=2e56696b-4c5c-4c9f-8681-c75ba7546700 path="/var/lib/kubelet/pods/2e56696b-4c5c-4c9f-8681-c75ba7546700/volumes" Nov 16 15:03:22.659577 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.659491 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cookie-secret\" (UniqueName: \"kubernetes.io/secret/29d0f8e3-e33b-4c9f-a290-193626492310-cookie-secret\") pod \"machine-config-daemon-q7lkj\" (UID: \"29d0f8e3-e33b-4c9f-a290-193626492310\") " pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" Nov 16 15:03:22.659577 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.659552 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/29d0f8e3-e33b-4c9f-a290-193626492310-rootfs\") pod \"machine-config-daemon-q7lkj\" (UID: \"29d0f8e3-e33b-4c9f-a290-193626492310\") " pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" Nov 16 15:03:22.659789 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.659647 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfrl8\" (UniqueName: \"kubernetes.io/projected/29d0f8e3-e33b-4c9f-a290-193626492310-kube-api-access-bfrl8\") pod \"machine-config-daemon-q7lkj\" (UID: \"29d0f8e3-e33b-4c9f-a290-193626492310\") " pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" Nov 16 15:03:22.659789 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.659694 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/29d0f8e3-e33b-4c9f-a290-193626492310-proxy-tls\") pod \"machine-config-daemon-q7lkj\" (UID: \"29d0f8e3-e33b-4c9f-a290-193626492310\") " pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" Nov 16 15:03:22.760442 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.760323 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-bfrl8\" (UniqueName: \"kubernetes.io/projected/29d0f8e3-e33b-4c9f-a290-193626492310-kube-api-access-bfrl8\") pod \"machine-config-daemon-q7lkj\" (UID: \"29d0f8e3-e33b-4c9f-a290-193626492310\") " pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" Nov 16 15:03:22.760442 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.760390 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/29d0f8e3-e33b-4c9f-a290-193626492310-proxy-tls\") pod \"machine-config-daemon-q7lkj\" (UID: \"29d0f8e3-e33b-4c9f-a290-193626492310\") " pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" Nov 16 15:03:22.760634 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.760455 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"cookie-secret\" (UniqueName: \"kubernetes.io/secret/29d0f8e3-e33b-4c9f-a290-193626492310-cookie-secret\") pod \"machine-config-daemon-q7lkj\" (UID: \"29d0f8e3-e33b-4c9f-a290-193626492310\") " pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" Nov 16 15:03:22.760634 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.760490 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/29d0f8e3-e33b-4c9f-a290-193626492310-rootfs\") pod \"machine-config-daemon-q7lkj\" (UID: \"29d0f8e3-e33b-4c9f-a290-193626492310\") " pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" Nov 16 15:03:22.760634 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.760560 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/29d0f8e3-e33b-4c9f-a290-193626492310-rootfs\") pod \"machine-config-daemon-q7lkj\" (UID: \"29d0f8e3-e33b-4c9f-a290-193626492310\") " pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" Nov 16 15:03:22.763562 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.763484 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"cookie-secret\" (UniqueName: \"kubernetes.io/secret/29d0f8e3-e33b-4c9f-a290-193626492310-cookie-secret\") pod \"machine-config-daemon-q7lkj\" (UID: \"29d0f8e3-e33b-4c9f-a290-193626492310\") " pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" Nov 16 15:03:22.763562 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.763492 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/29d0f8e3-e33b-4c9f-a290-193626492310-proxy-tls\") pod \"machine-config-daemon-q7lkj\" (UID: \"29d0f8e3-e33b-4c9f-a290-193626492310\") " pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" Nov 16 15:03:22.786141 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.786088 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfrl8\" (UniqueName: \"kubernetes.io/projected/29d0f8e3-e33b-4c9f-a290-193626492310-kube-api-access-bfrl8\") pod \"machine-config-daemon-q7lkj\" (UID: \"29d0f8e3-e33b-4c9f-a290-193626492310\") " pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" Nov 16 15:03:22.905952 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.905894 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" Nov 16 15:03:22.906677 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:22.906603037Z" level=info msg="Running pod sandbox: openshift-machine-config-operator/machine-config-daemon-q7lkj/POD" id=556f09f8-7b31-4532-ba54-62c739af785f name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 15:03:22.906785 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:22.906712438Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 15:03:22.966676 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:22.966586085Z" level=warning msg="Skipping invalid sysctl specified by config {net.ipv4.ping_group_range 0 2147483647}: \"net.ipv4.ping_group_range\" not allowed with host net enabled" id=556f09f8-7b31-4532-ba54-62c739af785f name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 15:03:22.970407 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:03:22.970361 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29d0f8e3_e33b_4c9f_a290_193626492310.slice/crio-da4c8f2ef641fd17bd2111d7f7aae94e93f6e8d6feba61c95e2c0adfdfded470.scope WatchSource:0}: Error finding container da4c8f2ef641fd17bd2111d7f7aae94e93f6e8d6feba61c95e2c0adfdfded470: Status 404 returned error &{%!s(*http.body=&{0xc0029ba030 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 15:03:22.983631 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:22.983562512Z" level=info msg="Ran pod sandbox da4c8f2ef641fd17bd2111d7f7aae94e93f6e8d6feba61c95e2c0adfdfded470 with infra container: openshift-machine-config-operator/machine-config-daemon-q7lkj/POD" id=556f09f8-7b31-4532-ba54-62c739af785f name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 15:03:22.984866 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:22.984801522Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:00dc61290f10ec21a547731460f067f957e845eb7a6cc9e29044c73a62a41e04" id=2764741e-1ec9-45c0-bb99-64b9ee794566 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:03:22.985144 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:22.985098624Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:00dc61290f10ec21a547731460f067f957e845eb7a6cc9e29044c73a62a41e04 not found" id=2764741e-1ec9-45c0-bb99-64b9ee794566 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:03:22.985638 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:22.985592 1563 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 16 15:03:22.986239 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:22.986179232Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:00dc61290f10ec21a547731460f067f957e845eb7a6cc9e29044c73a62a41e04" id=124edf6b-56c9-4527-a62d-555ad8a482bd name=/runtime.v1.ImageService/PullImage Nov 16 15:03:22.989583 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:22.989543557Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:00dc61290f10ec21a547731460f067f957e845eb7a6cc9e29044c73a62a41e04\"" Nov 16 15:03:23.496843 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:23.496801 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" event=&{ID:29d0f8e3-e33b-4c9f-a290-193626492310 Type:ContainerStarted Data:da4c8f2ef641fd17bd2111d7f7aae94e93f6e8d6feba61c95e2c0adfdfded470} Nov 16 15:03:23.886359 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:23.886300 1563 scope.go:110] "RemoveContainer" containerID="f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403" Nov 16 15:03:23.887697 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:23.887613774Z" level=info msg="Removing container: f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403" id=46d512af-99c9-4a5a-a772-0ffc00273cc3 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:03:23.921285 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:23.921237326Z" level=info msg="Removed container f982dd17e1f51c1c96ed4eab6f4ba889c4f586337768750253e14cf5341c8403: openshift-machine-config-operator/machine-config-daemon-kxqnl/oauth-proxy" id=46d512af-99c9-4a5a-a772-0ffc00273cc3 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:03:23.921601 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:23.921566 1563 scope.go:110] "RemoveContainer" containerID="ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588" Nov 16 15:03:23.922765 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:23.922694936Z" level=info msg="Removing container: ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588" id=880cef79-9dbf-4325-b418-f69cd3e2b71a name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:03:23.951462 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-804bed90b5cd62f98e2e4b3ec3fa0596e4e0f7f71999b1943ef0bd50166e5b4d-merged.mount: Succeeded. Nov 16 15:03:23.952374 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-804bed90b5cd62f98e2e4b3ec3fa0596e4e0f7f71999b1943ef0bd50166e5b4d-merged.mount: Consumed 0 CPU time Nov 16 15:03:24.323061 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:24.322850029Z" level=info msg="Removed container ffbd529968c7fa5ee4854d38844d763b915a481849c4ef6b64b8a85f3dc84588: openshift-machine-config-operator/machine-config-daemon-kxqnl/machine-config-daemon" id=880cef79-9dbf-4325-b418-f69cd3e2b71a name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:03:24.325470 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:24.325429549Z" level=info msg="Stopping pod sandbox: 1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c" id=044f8bb2-64db-463f-93be-f75e0ad1f178 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:03:24.325585 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:24.325472949Z" level=info msg="Stopped pod sandbox (already stopped): 1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c" id=044f8bb2-64db-463f-93be-f75e0ad1f178 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:03:24.325809 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:24.325770751Z" level=info msg="Removing pod sandbox: 1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c" id=bed8a1c3-9576-487e-aa11-3de4f0be21e1 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:03:24.326735 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:24.326669358Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:00dc61290f10ec21a547731460f067f957e845eb7a6cc9e29044c73a62a41e04\"" Nov 16 15:03:24.365272 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:24.365089345Z" level=info msg="Removed pod sandbox: 1c83c6120dd174d8dbdc31ded880c5fb92f5de5a61eb06b4206a78a0f603e17c" id=bed8a1c3-9576-487e-aa11-3de4f0be21e1 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:03:29.433803 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.433700787Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:00dc61290f10ec21a547731460f067f957e845eb7a6cc9e29044c73a62a41e04" id=124edf6b-56c9-4527-a62d-555ad8a482bd name=/runtime.v1.ImageService/PullImage Nov 16 15:03:29.434881 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.434803395Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:00dc61290f10ec21a547731460f067f957e845eb7a6cc9e29044c73a62a41e04" id=dc57328f-0271-4993-be0a-be8b3930d289 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:03:29.436601 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.436501707Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:d8bd350d823e9770c541b125704b6b3ce0bcb6b6393d7b31898b9c58f8f7e88f,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:00dc61290f10ec21a547731460f067f957e845eb7a6cc9e29044c73a62a41e04],Size_:506886444,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=dc57328f-0271-4993-be0a-be8b3930d289 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:03:29.437559 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.437521815Z" level=info msg="Creating container: openshift-machine-config-operator/machine-config-daemon-q7lkj/machine-config-daemon" id=48a11b82-3647-4947-a26e-b0e1c4ea9fb6 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:03:29.437651 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.437627416Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 15:03:29.505210 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope. Nov 16 15:03:29.527719 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a. Nov 16 15:03:29.603273 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.603192526Z" level=info msg="Created container cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a: openshift-machine-config-operator/machine-config-daemon-q7lkj/machine-config-daemon" id=48a11b82-3647-4947-a26e-b0e1c4ea9fb6 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:03:29.603973 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.603900731Z" level=info msg="Starting container: cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a" id=7a51c480-abbe-4952-b6f1-57e1b88eeb75 name=/runtime.v1.RuntimeService/StartContainer Nov 16 15:03:29.616923 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.616866726Z" level=info msg="Started container" PID=200509 containerID=cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a description=openshift-machine-config-operator/machine-config-daemon-q7lkj/machine-config-daemon id=7a51c480-abbe-4952-b6f1-57e1b88eeb75 name=/runtime.v1.RuntimeService/StartContainer sandboxID=da4c8f2ef641fd17bd2111d7f7aae94e93f6e8d6feba61c95e2c0adfdfded470 Nov 16 15:03:29.638678 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.638624185Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634" id=585c8e25-01e7-4ffd-9c4a-7772231ae4c4 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:03:29.638912 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.638876887Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:2f3a1ac7bb93faa2312d7fca21ca26dc55cbcd57502b470ad0bc40866a6ce73d,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634],Size_:340927490,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=585c8e25-01e7-4ffd-9c4a-7772231ae4c4 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:03:29.646309 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.646268841Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634" id=760e5f3c-52ef-478e-b564-840536e1ece5 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:03:29.646501 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.646468143Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:2f3a1ac7bb93faa2312d7fca21ca26dc55cbcd57502b470ad0bc40866a6ce73d,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634],Size_:340927490,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=760e5f3c-52ef-478e-b564-840536e1ece5 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:03:29.647514 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.647474650Z" level=info msg="Creating container: openshift-machine-config-operator/machine-config-daemon-q7lkj/oauth-proxy" id=08b8e141-656b-4878-b3ff-037b2709d5ea name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:03:29.647623 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.647598851Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 15:03:29.655352 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting rpm-ostree System Management Daemon... Nov 16 15:03:29.689349 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: Reading config file '/etc/rpm-ostreed.conf' Nov 16 15:03:29.700681 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: In idle state; will auto-exit in 60 seconds Nov 16 15:03:29.813655 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started rpm-ostree System Management Daemon. Nov 16 15:03:29.816028 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-8a23ac6b93369f7a2b71c3e4f5224b60b4769cc455757bc2e46bd93b07451bf9.scope. Nov 16 15:03:29.845538 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 8a23ac6b93369f7a2b71c3e4f5224b60b4769cc455757bc2e46bd93b07451bf9. Nov 16 15:03:29.855670 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: client(id:machine-config-operator dbus:1.653 unit:crio-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope uid:0) added; new total=1 Nov 16 15:03:29.871669 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: client(id:machine-config-operator dbus:1.653 unit:crio-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope uid:0) vanished; remaining=0 Nov 16 15:03:29.871691 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: In idle state; will auto-exit in 60 seconds Nov 16 15:03:29.879685 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:29.882392 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Reloading. Nov 16 15:03:29.950222 ropatil1611az-gs6zj-worker-westus-j4ngg coreos-platform-chrony: /run/coreos-platform-chrony.conf already exists; skipping Nov 16 15:03:29.957819 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.957749519Z" level=info msg="Created container 8a23ac6b93369f7a2b71c3e4f5224b60b4769cc455757bc2e46bd93b07451bf9: openshift-machine-config-operator/machine-config-daemon-q7lkj/oauth-proxy" id=08b8e141-656b-4878-b3ff-037b2709d5ea name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:03:29.958403 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.958357423Z" level=info msg="Starting container: 8a23ac6b93369f7a2b71c3e4f5224b60b4769cc455757bc2e46bd93b07451bf9" id=03cc3c5c-b0e5-4c37-b560-d548c8ba0981 name=/runtime.v1.RuntimeService/StartContainer Nov 16 15:03:29.970716 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:03:29.970673913Z" level=info msg="Started container" PID=200566 containerID=8a23ac6b93369f7a2b71c3e4f5224b60b4769cc455757bc2e46bd93b07451bf9 description=openshift-machine-config-operator/machine-config-daemon-q7lkj/oauth-proxy id=03cc3c5c-b0e5-4c37-b560-d548c8ba0981 name=/runtime.v1.RuntimeService/StartContainer sandboxID=da4c8f2ef641fd17bd2111d7f7aae94e93f6e8d6feba61c95e2c0adfdfded470 Nov 16 15:03:29.987985 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: /usr/lib/systemd/system/bootupd.service:22: Unknown lvalue 'ProtectHostname' in section 'Service' Nov 16 15:03:30.220961 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.238299 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.240362 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.242465 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.249564 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.251845 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.253359 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.257371 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.259516 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.260979 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.271415 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.281849 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.285023 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.291660 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.298396 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.304325 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.308659 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.313314 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.315276 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.318691 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.326660 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.330415 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.331924 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.337477 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.339443 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.342946 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.345695 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.349062 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.352023 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.362472 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.364365 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.365973 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.370236 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.372309 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.380892 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.382939 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.385251 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.386969 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.393650 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.400458 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.401864 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.403588 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.409640 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.417158 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.420587 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.422322 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:03:30.467463 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: client(id:machine-config-operator dbus:1.654 unit:crio-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope uid:0) added; new total=1 Nov 16 15:03:30.482765 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: client(id:machine-config-operator dbus:1.654 unit:crio-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope uid:0) vanished; remaining=0 Nov 16 15:03:30.482787 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: In idle state; will auto-exit in 60 seconds Nov 16 15:03:30.521956 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:30.521915 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" event=&{ID:29d0f8e3-e33b-4c9f-a290-193626492310 Type:ContainerStarted Data:8a23ac6b93369f7a2b71c3e4f5224b60b4769cc455757bc2e46bd93b07451bf9} Nov 16 15:03:30.521956 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:03:30.521968 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-q7lkj" event=&{ID:29d0f8e3-e33b-4c9f-a290-193626492310 Type:ContainerStarted Data:cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a} Nov 16 15:03:30.529689 ropatil1611az-gs6zj-worker-westus-j4ngg root[200656]: machine-config-daemon[200509]: Starting to manage node: ropatil1611az-gs6zj-worker-westus-j4ngg Nov 16 15:03:30.564065 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: client(id:machine-config-operator dbus:1.655 unit:crio-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope uid:0) added; new total=1 Nov 16 15:03:30.578345 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: client(id:machine-config-operator dbus:1.655 unit:crio-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope uid:0) vanished; remaining=0 Nov 16 15:03:30.578365 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: In idle state; will auto-exit in 63 seconds Nov 16 15:03:31.591749 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: client(id:machine-config-operator dbus:1.656 unit:crio-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope uid:0) added; new total=1 Nov 16 15:03:33.819932 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: Locked sysroot Nov 16 15:03:33.820530 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: EXT4-fs (sda3): re-mounted. Opts: Nov 16 15:03:33.820086 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: Initiated txn Cleanup for client(id:machine-config-operator dbus:1.656 unit:crio-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope uid:0): /org/projectatomic/rpmostree1/rhcos Nov 16 15:03:34.531161 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: Bootloader updated; bootconfig swap: yes; deployment count change: -1 Nov 16 15:03:38.735573 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.anaylU.mount: Succeeded. Nov 16 15:03:38.735600 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.74yoQQ.mount: Succeeded. Nov 16 15:03:40.577896 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: Txn Cleanup on /org/projectatomic/rpmostree1/rhcos successful Nov 16 15:03:40.579023 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: Unlocked sysroot Nov 16 15:03:40.581704 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: client(id:machine-config-operator dbus:1.656 unit:crio-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope uid:0) vanished; remaining=0 Nov 16 15:03:40.581786 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: In idle state; will auto-exit in 60 seconds Nov 16 15:03:40.619940 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: client(id:machine-config-operator dbus:1.657 unit:crio-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope uid:0) added; new total=1 Nov 16 15:03:40.631850 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: client(id:machine-config-operator dbus:1.657 unit:crio-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope uid:0) vanished; remaining=0 Nov 16 15:03:40.631899 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: In idle state; will auto-exit in 62 seconds Nov 16 15:04:42.745975 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[200544]: In idle state; will auto-exit in 64 seconds Nov 16 15:04:42.765589 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: rpm-ostreed.service: Succeeded. Nov 16 15:04:42.766391 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: rpm-ostreed.service: Consumed 2.449s CPU time Nov 16 15:05:08.721043 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454-runc.NwJKbl.mount: Succeeded. Nov 16 15:06:36.014753 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.014709 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-ingress/router-default-cff655768-nrbzb] Nov 16 15:06:36.015324 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.014783 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 15:06:36.031202 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod03363f87_776c_47c3_9bd8_8c0de24a4de5.slice. Nov 16 15:06:36.188011 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.187965 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-default-certificate\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.188011 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.188034 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03363f87-776c-47c3-9bd8-8c0de24a4de5-service-ca-bundle\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.188327 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.188082 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqcdh\" (UniqueName: \"kubernetes.io/projected/03363f87-776c-47c3-9bd8-8c0de24a4de5-kube-api-access-mqcdh\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.188327 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.188183 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-metrics-certs\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.188464 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.188364 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-stats-auth\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.290162 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.289895 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-stats-auth\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.290162 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.289987 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-default-certificate\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.290162 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.290052 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03363f87-776c-47c3-9bd8-8c0de24a4de5-service-ca-bundle\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.290162 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.290167 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-mqcdh\" (UniqueName: \"kubernetes.io/projected/03363f87-776c-47c3-9bd8-8c0de24a4de5-kube-api-access-mqcdh\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.290501 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.290222 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-metrics-certs\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.290973 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.290929 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03363f87-776c-47c3-9bd8-8c0de24a4de5-service-ca-bundle\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.293176 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.293120 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-stats-auth\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.293583 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.293556 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-default-certificate\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.294383 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.294355 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-metrics-certs\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.320620 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.320542 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqcdh\" (UniqueName: \"kubernetes.io/projected/03363f87-776c-47c3-9bd8-8c0de24a4de5-kube-api-access-mqcdh\") pod \"router-default-cff655768-nrbzb\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.345600 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:36.345548 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:36.346421 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:36.346344355Z" level=info msg="Running pod sandbox: openshift-ingress/router-default-cff655768-nrbzb/POD" id=c27425de-a270-484c-a841-2c8dbcbbd4ab name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 15:06:36.346822 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:36.346442756Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 15:06:36.425662 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:36.425612029Z" level=info msg="Got pod network &{Name:router-default-cff655768-nrbzb Namespace:openshift-ingress ID:f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2 UID:03363f87-776c-47c3-9bd8-8c0de24a4de5 NetNS:/var/run/netns/05707f1f-11b9-44c3-9087-b9367f58b73c Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:06:36.425662 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:36.425655429Z" level=info msg="Adding pod openshift-ingress_router-default-cff655768-nrbzb to CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:06:36.474203 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 15:06:36.481168 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668611196.4811] manager: (vethade753ea): new Veth device (/org/freedesktop/NetworkManager/Devices/88) Nov 16 15:06:36.482044 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668611196.4820] device (vethade753ea): carrier: link connected Nov 16 15:06:36.488364 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[203941]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 15:06:36.495030 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): vethade753ea: link is not ready Nov 16 15:06:36.495104 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethade753ea: link becomes ready Nov 16 15:06:36.495156 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 15:06:36.501359 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[203941]: Could not generate persistent MAC address for vethade753ea: No such file or directory Nov 16 15:06:36.509653 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668611196.5096] manager: (vethade753ea): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/89) Nov 16 15:06:36.515561 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00251|bridge|INFO|bridge br0: added interface vethade753ea on port 42 Nov 16 15:06:36.516171 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethade753ea entered promiscuous mode Nov 16 15:06:36.547467 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00252|connmgr|INFO|br0<->unix#570: 5 flow_mods in the last 0 s (5 adds) Nov 16 15:06:36.597941 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00253|connmgr|INFO|br0<->unix#574: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:06:36.600823 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00254|connmgr|INFO|br0<->unix#576: 6 flow_mods in the last 0 s (2 adds, 4 deletes) Nov 16 15:06:38.268158 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:06:38Z [verbose] Add: openshift-ingress:router-default-cff655768-nrbzb:03363f87-776c-47c3-9bd8-8c0de24a4de5:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/05707f1f-11b9-44c3-9087-b9367f58b73c"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.41/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 15:06:38.268158 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 15:06:38.237466 203924 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-ingress", Name:"router-default-cff655768-nrbzb", UID:"03363f87-776c-47c3-9bd8-8c0de24a4de5", APIVersion:"v1", ResourceVersion:"109709", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.41/23] from openshift-sdn Nov 16 15:06:38.268897 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:38.268837263Z" level=info msg="Got pod network &{Name:router-default-cff655768-nrbzb Namespace:openshift-ingress ID:f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2 UID:03363f87-776c-47c3-9bd8-8c0de24a4de5 NetNS:/var/run/netns/05707f1f-11b9-44c3-9087-b9367f58b73c Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:06:38.269020 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:38.268990064Z" level=info msg="Checking pod openshift-ingress_router-default-cff655768-nrbzb for CNI network multus-cni-network (type=multus)" Nov 16 15:06:38.270682 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:38.270645 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-ingress/router-default-cff655768-nrbzb] Nov 16 15:06:38.276480 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:06:38.271533 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03363f87_776c_47c3_9bd8_8c0de24a4de5.slice/crio-f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2.scope WatchSource:0}: Error finding container f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2: Status 404 returned error &{%!s(*http.body=&{0xc001423110 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 15:06:38.279041 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:38.278955336Z" level=info msg="Ran pod sandbox f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2 with infra container: openshift-ingress/router-default-cff655768-nrbzb/POD" id=c27425de-a270-484c-a841-2c8dbcbbd4ab name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 15:06:38.280332 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:38.280120245Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:663df5d0e74d92129e92f1387e4a81d3faa95a3e34e254cdc82cb7d6a4306219" id=2ecf40a1-2cca-4436-b589-167842188f46 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:38.280437 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:38.280416947Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:663df5d0e74d92129e92f1387e4a81d3faa95a3e34e254cdc82cb7d6a4306219 not found" id=2ecf40a1-2cca-4436-b589-167842188f46 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:38.281149 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:38.281102252Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:663df5d0e74d92129e92f1387e4a81d3faa95a3e34e254cdc82cb7d6a4306219" id=b2f488b4-b5dd-47b6-8fdc-ecfeb8ebfba9 name=/runtime.v1.ImageService/PullImage Nov 16 15:06:38.284329 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:38.284292475Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:663df5d0e74d92129e92f1387e4a81d3faa95a3e34e254cdc82cb7d6a4306219\"" Nov 16 15:06:39.146279 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:39.146199 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-cff655768-nrbzb" event=&{ID:03363f87-776c-47c3-9bd8-8c0de24a4de5 Type:ContainerStarted Data:f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2} Nov 16 15:06:39.557699 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:39.557487176Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:663df5d0e74d92129e92f1387e4a81d3faa95a3e34e254cdc82cb7d6a4306219\"" Nov 16 15:06:40.374777 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.374721 1563 kubelet.go:2137] "SyncLoop ADD" source="api" pods=[openshift-monitoring/prometheus-k8s-1] Nov 16 15:06:40.374777 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.374801 1563 topology_manager.go:200] "Topology Admit Handler" Nov 16 15:06:40.392472 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Created slice libcontainer container kubepods-burstable-pod3670081b_e385_4685_8dda_b3a3ee01f233.slice. Nov 16 15:06:40.536602 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.536549 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3670081b-e385-4685-8dda-b3a3ee01f233-config-out\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.536602 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.536628 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-k8s-db\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.536857 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.536662 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-config\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.536857 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.536699 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.536857 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.536726 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-metrics-client-ca\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.536857 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.536769 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-etcd-client-certs\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-kube-etcd-client-certs\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.536857 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.536846 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3670081b-e385-4685-8dda-b3a3ee01f233-tls-assets\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.537026 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.536890 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.537026 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.536927 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-kube-rbac-proxy\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.537026 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.536957 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnqkh\" (UniqueName: \"kubernetes.io/projected/3670081b-e385-4685-8dda-b3a3ee01f233-kube-api-access-vnqkh\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.537026 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.537009 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-metrics-client-ca\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.537184 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.537045 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.537184 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.537074 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-web-config\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.537184 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.537101 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.537184 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.537147 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.537317 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.537225 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-proxy\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-proxy\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.537317 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.537271 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-metrics-client-certs\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.537399 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.537338 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.537437 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.537405 1563 reconciler.go:221] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-grpc-tls\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.638864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.638595 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.638864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.638700 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-web-config\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.638864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.638754 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.638864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.638812 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.638864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.638849 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-proxy\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-proxy\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639171 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.638881 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-metrics-client-certs\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639171 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.638921 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639171 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.638956 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-grpc-tls\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639171 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.638996 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3670081b-e385-4685-8dda-b3a3ee01f233-config-out\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639171 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.639030 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-k8s-db\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639171 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.639061 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-config\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639171 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.639097 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-kube-etcd-client-certs\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-kube-etcd-client-certs\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639171 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.639145 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.639178 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-metrics-client-ca\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.639210 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3670081b-e385-4685-8dda-b3a3ee01f233-tls-assets\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.639240 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.639306 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-kube-rbac-proxy\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.639341 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-metrics-client-ca\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.639374 1563 reconciler.go:258] "operationExecutor.MountVolume started for volume \"kube-api-access-vnqkh\" (UniqueName: \"kubernetes.io/projected/3670081b-e385-4685-8dda-b3a3ee01f233-kube-api-access-vnqkh\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.639489 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.639510 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-k8s-db\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.639740 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.639725 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.640290 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.640220 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3670081b-e385-4685-8dda-b3a3ee01f233-config-out\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.640290 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.640245 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.640660 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.640634 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-metrics-client-ca\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.640754 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.640702 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-metrics-client-ca\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.642230 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.642199 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-web-config\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.643789 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.643140 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-kube-rbac-proxy\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.643789 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.643251 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-proxy\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-proxy\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.643789 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.643360 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-grpc-tls\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.643789 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.643428 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-metrics-client-certs\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.644315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.644249 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.644315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.644251 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-config\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.644685 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.644655 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.644739 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.644686 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-kube-etcd-client-certs\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-kube-etcd-client-certs\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.646196 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.646168 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3670081b-e385-4685-8dda-b3a3ee01f233-tls-assets\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.646455 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.646427 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.664474 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.664431 1563 operation_generator.go:756] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnqkh\" (UniqueName: \"kubernetes.io/projected/3670081b-e385-4685-8dda-b3a3ee01f233-kube-api-access-vnqkh\") pod \"prometheus-k8s-1\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.710244 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:40.710187 1563 kuberuntime_manager.go:487] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:40.711194 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:40.711018231Z" level=info msg="Running pod sandbox: openshift-monitoring/prometheus-k8s-1/POD" id=159638dc-2948-4a77-92ee-d85507a0e6b1 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 15:06:40.711194 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:40.711179332Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 15:06:40.784769 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:40.784720752Z" level=info msg="Got pod network &{Name:prometheus-k8s-1 Namespace:openshift-monitoring ID:0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a UID:3670081b-e385-4685-8dda-b3a3ee01f233 NetNS:/var/run/netns/8defb3e9-54f6-49a9-a23b-266a54369420 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:06:40.784769 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:40.784761352Z" level=info msg="Adding pod openshift-monitoring_prometheus-k8s-1 to CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:06:40.843187 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready Nov 16 15:06:40.853464 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668611200.8534] manager: (vethb55d967f): new Veth device (/org/freedesktop/NetworkManager/Devices/90) Nov 16 15:06:40.854409 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668611200.8543] device (vethb55d967f): carrier: link connected Nov 16 15:06:40.861182 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[204225]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Nov 16 15:06:40.867172 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_UP): vethb55d967f: link is not ready Nov 16 15:06:40.867247 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethb55d967f: link becomes ready Nov 16 15:06:40.867283 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Nov 16 15:06:40.872933 ropatil1611az-gs6zj-worker-westus-j4ngg systemd-udevd[204225]: Could not generate persistent MAC address for vethb55d967f: No such file or directory Nov 16 15:06:40.900925 ropatil1611az-gs6zj-worker-westus-j4ngg NetworkManager[1379]: [1668611200.9008] manager: (vethb55d967f): new Open vSwitch Port device (/org/freedesktop/NetworkManager/Devices/91) Nov 16 15:06:40.907187 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethb55d967f entered promiscuous mode Nov 16 15:06:40.907112 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00255|bridge|INFO|bridge br0: added interface vethb55d967f on port 43 Nov 16 15:06:40.939858 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00256|connmgr|INFO|br0<->unix#579: 5 flow_mods in the last 0 s (5 adds) Nov 16 15:06:40.986797 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00257|connmgr|INFO|br0<->unix#582: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:06:42.114271 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:06:42Z [verbose] Add: openshift-monitoring:prometheus-k8s-1:3670081b-e385-4685-8dda-b3a3ee01f233:openshift-sdn(openshift-sdn):eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/var/run/netns/8defb3e9-54f6-49a9-a23b-266a54369420"}],"ips":[{"version":"4","interface":0,"address":"10.129.2.42/23","gateway":"10.129.2.1"}],"dns":{}} Nov 16 15:06:42.114271 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: I1116 15:06:42.075953 204210 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-monitoring", Name:"prometheus-k8s-1", UID:"3670081b-e385-4685-8dda-b3a3ee01f233", APIVersion:"v1", ResourceVersion:"110012", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [10.129.2.42/23] from openshift-sdn Nov 16 15:06:42.114271 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.113880048Z" level=info msg="Got pod network &{Name:prometheus-k8s-1 Namespace:openshift-monitoring ID:0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a UID:3670081b-e385-4685-8dda-b3a3ee01f233 NetNS:/var/run/netns/8defb3e9-54f6-49a9-a23b-266a54369420 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:06:42.114271 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.114034649Z" level=info msg="Checking pod openshift-monitoring_prometheus-k8s-1 for CNI network multus-cni-network (type=multus)" Nov 16 15:06:42.115422 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:42.115387 1563 kubelet.go:2144] "SyncLoop UPDATE" source="api" pods=[openshift-monitoring/prometheus-k8s-1] Nov 16 15:06:42.401086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:06:42.400939 1563 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3670081b_e385_4685_8dda_b3a3ee01f233.slice/crio-0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a.scope WatchSource:0}: Error finding container 0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a: Status 404 returned error &{%!s(*http.body=&{0xc00408bf08 false false {0 0} false false false }) {%!s(int32=0) %!s(uint32=0)} %!s(bool=false) %!s(func(error) error=0x85b280) %!s(func() error=0x85b380)} Nov 16 15:06:42.411002 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.410956848Z" level=info msg="Ran pod sandbox 0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a with infra container: openshift-monitoring/prometheus-k8s-1/POD" id=159638dc-2948-4a77-92ee-d85507a0e6b1 name=/runtime.v1.RuntimeService/RunPodSandbox Nov 16 15:06:42.412249 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.412207057Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3" id=49e5731d-1c88-4634-b6b4-119be85e40e0 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:42.412472 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.412438859Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:f4a6bddcf29f1036d73e15b666897a7d42426b1a799e533d71d356d6f22b6826,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3],Size_:344385412,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=49e5731d-1c88-4634-b6b4-119be85e40e0 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:42.413374 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.413317265Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3" id=fd8b4ce1-759c-4d96-8b62-add90b4a3843 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:42.413572 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.413528066Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:f4a6bddcf29f1036d73e15b666897a7d42426b1a799e533d71d356d6f22b6826,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3],Size_:344385412,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=fd8b4ce1-759c-4d96-8b62-add90b4a3843 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:42.414735 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.414692775Z" level=info msg="Creating container: openshift-monitoring/prometheus-k8s-1/init-config-reloader" id=acc5c4b2-310b-4bc7-8c5a-515c8b2ec4bd name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:42.414813 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.414793775Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 15:06:42.836886 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4.scope. Nov 16 15:06:42.878279 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4. Nov 16 15:06:42.919941 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:06:42.969972 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.969885300Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:663df5d0e74d92129e92f1387e4a81d3faa95a3e34e254cdc82cb7d6a4306219" id=b2f488b4-b5dd-47b6-8fdc-ecfeb8ebfba9 name=/runtime.v1.ImageService/PullImage Nov 16 15:06:42.971197 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.971151109Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:663df5d0e74d92129e92f1387e4a81d3faa95a3e34e254cdc82cb7d6a4306219" id=6af72340-0a2b-4512-ad92-10737442e97e name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:42.973212 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.973147623Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:4bacd40c07cc3519f1384ba96cd56468751c65799e3bd22beac15a3b88be65d2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:663df5d0e74d92129e92f1387e4a81d3faa95a3e34e254cdc82cb7d6a4306219],Size_:413546764,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=6af72340-0a2b-4512-ad92-10737442e97e name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:42.974455 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.974399931Z" level=info msg="Creating container: openshift-ingress/router-default-cff655768-nrbzb/router" id=da7ab319-e2ff-4cc6-9361-f21b0d50393a name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:42.974555 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.974523632Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 15:06:42.989187 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.989103035Z" level=info msg="Created container 55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4: openshift-monitoring/prometheus-k8s-1/init-config-reloader" id=acc5c4b2-310b-4bc7-8c5a-515c8b2ec4bd name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:42.989746 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:42.989601439Z" level=info msg="Starting container: 55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4" id=555d2443-248e-4cc3-bc35-911933d9f28e name=/runtime.v1.RuntimeService/StartContainer Nov 16 15:06:43.001894 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:43.001725825Z" level=info msg="Started container" PID=204350 containerID=55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4 description=openshift-monitoring/prometheus-k8s-1/init-config-reloader id=555d2443-248e-4cc3-bc35-911933d9f28e name=/runtime.v1.RuntimeService/StartContainer sandboxID=0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a Nov 16 15:06:43.038774 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4.scope: Succeeded. Nov 16 15:06:43.039420 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4.scope: Consumed 44ms CPU time Nov 16 15:06:43.042362 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4.scope: Succeeded. Nov 16 15:06:43.042992 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4.scope: Consumed 43ms CPU time Nov 16 15:06:43.044461 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3.scope. Nov 16 15:06:43.065630 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3. Nov 16 15:06:43.108280 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:06:43.160478 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:43.160442 1563 generic.go:296] "Generic (PLEG): container finished" podID=3670081b-e385-4685-8dda-b3a3ee01f233 containerID="55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4" exitCode=0 Nov 16 15:06:43.160959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:43.160501 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerDied Data:55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4} Nov 16 15:06:43.160959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:43.160533 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerStarted Data:0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a} Nov 16 15:06:43.161572 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:43.161523254Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:beef7f46abbc40bb953bbf2351d6fc5f95b043bb66924401f9b09d811659110e" id=d0489b91-3a6d-446f-b358-c3650bc6e694 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:43.161923 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:43.161814056Z" level=info msg="Image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:beef7f46abbc40bb953bbf2351d6fc5f95b043bb66924401f9b09d811659110e not found" id=d0489b91-3a6d-446f-b358-c3650bc6e694 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:43.162760 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:43.162722863Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:beef7f46abbc40bb953bbf2351d6fc5f95b043bb66924401f9b09d811659110e" id=5caf1fb8-6f6e-4303-9c1b-b6602adda2cb name=/runtime.v1.ImageService/PullImage Nov 16 15:06:43.166002 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:43.165969586Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:beef7f46abbc40bb953bbf2351d6fc5f95b043bb66924401f9b09d811659110e\"" Nov 16 15:06:43.179226 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:43.179072678Z" level=info msg="Created container 5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3: openshift-ingress/router-default-cff655768-nrbzb/router" id=da7ab319-e2ff-4cc6-9361-f21b0d50393a name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:43.179526 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:43.179492081Z" level=info msg="Starting container: 5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3" id=22f52f5b-615b-4a33-b27e-f4918e894f55 name=/runtime.v1.RuntimeService/StartContainer Nov 16 15:06:43.192775 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:43.192712975Z" level=info msg="Started container" PID=204411 containerID=5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3 description=openshift-ingress/router-default-cff655768-nrbzb/router id=22f52f5b-615b-4a33-b27e-f4918e894f55 name=/runtime.v1.RuntimeService/StartContainer sandboxID=f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2 Nov 16 15:06:44.164314 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:44.164270 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-cff655768-nrbzb" event=&{ID:03363f87-776c-47c3-9bd8-8c0de24a4de5 Type:ContainerStarted Data:5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3} Nov 16 15:06:44.346705 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:44.346646 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:44.348934 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:44.348905 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:44.423235 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:44.423021072Z" level=info msg="Trying to access \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:beef7f46abbc40bb953bbf2351d6fc5f95b043bb66924401f9b09d811659110e\"" Nov 16 15:06:45.167855 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:45.167772 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:45.169389 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:45.169363 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:06:49.104270 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.104055783Z" level=info msg="Pulled image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:beef7f46abbc40bb953bbf2351d6fc5f95b043bb66924401f9b09d811659110e" id=5caf1fb8-6f6e-4303-9c1b-b6602adda2cb name=/runtime.v1.ImageService/PullImage Nov 16 15:06:49.121718 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.121670512Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:beef7f46abbc40bb953bbf2351d6fc5f95b043bb66924401f9b09d811659110e" id=d6fff802-a8e9-40e5-aa52-a7516bb60566 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.123573 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.123528526Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:3c02153c39657a525f5cf12b48db9d02a325278e1281d3ab28f11f997c53e2d2,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:beef7f46abbc40bb953bbf2351d6fc5f95b043bb66924401f9b09d811659110e],Size_:447816241,Uid:nil,Username:nobody,Spec:nil,},Info:map[string]string{},}" id=d6fff802-a8e9-40e5-aa52-a7516bb60566 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.125018 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.124952336Z" level=info msg="Creating container: openshift-monitoring/prometheus-k8s-1/prometheus" id=f2aec45b-04cc-4593-ac82-ee03de2acd69 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:49.125113 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.125082137Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 15:06:49.196585 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae.scope. Nov 16 15:06:49.218358 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae. Nov 16 15:06:49.266375 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:06:49.341482 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.341410723Z" level=info msg="Created container 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae: openshift-monitoring/prometheus-k8s-1/prometheus" id=f2aec45b-04cc-4593-ac82-ee03de2acd69 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:49.342073 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.342003627Z" level=info msg="Starting container: 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae" id=4e1cd5e5-5aa4-4b70-afa5-7b121111603f name=/runtime.v1.RuntimeService/StartContainer Nov 16 15:06:49.357639 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.357498541Z" level=info msg="Started container" PID=204723 containerID=6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae description=openshift-monitoring/prometheus-k8s-1/prometheus id=4e1cd5e5-5aa4-4b70-afa5-7b121111603f name=/runtime.v1.RuntimeService/StartContainer sandboxID=0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a Nov 16 15:06:49.379002 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.378952898Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3" id=ab583644-91df-468e-a539-5a71555b1e0e name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.379324 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.379258001Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:f4a6bddcf29f1036d73e15b666897a7d42426b1a799e533d71d356d6f22b6826,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3],Size_:344385412,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=ab583644-91df-468e-a539-5a71555b1e0e name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.380499 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.380460509Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3" id=6e1fa949-eb3a-4edd-bbac-6b717e7d0c72 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.380728 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.380693311Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:f4a6bddcf29f1036d73e15b666897a7d42426b1a799e533d71d356d6f22b6826,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:edb0a2678e55062e555b3a2bfc5a99d87bc3e0eedf93eb0b368d7a864261f5f3],Size_:344385412,Uid:&Int64Value{Value:1001,},Username:,Spec:nil,},Info:map[string]string{},}" id=6e1fa949-eb3a-4edd-bbac-6b717e7d0c72 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.381892 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.381853320Z" level=info msg="Creating container: openshift-monitoring/prometheus-k8s-1/config-reloader" id=735a2403-5647-4728-b416-a973a9213d4e name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:49.381991 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.381959420Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 15:06:49.462451 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a.scope. Nov 16 15:06:49.486961 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a. Nov 16 15:06:49.526302 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:06:49.608845 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.608687483Z" level=info msg="Created container e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a: openshift-monitoring/prometheus-k8s-1/config-reloader" id=735a2403-5647-4728-b416-a973a9213d4e name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:49.609326 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.609288087Z" level=info msg="Starting container: e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a" id=d7adc57f-ec81-4b6d-86d6-346b1f3189b1 name=/runtime.v1.RuntimeService/StartContainer Nov 16 15:06:49.626146 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.626063210Z" level=info msg="Started container" PID=204791 containerID=e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a description=openshift-monitoring/prometheus-k8s-1/config-reloader id=d7adc57f-ec81-4b6d-86d6-346b1f3189b1 name=/runtime.v1.RuntimeService/StartContainer sandboxID=0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a Nov 16 15:06:49.662212 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.662155375Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c7327d965050fa0e8fbc48300a8c6ce1bb04fad7e4e6349290dde68b53e9cffd" id=4247a64b-e98f-4e51-bcf5-419e2132c331 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.662463 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.662431077Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:c38c6316f11e9dad3a0b7f9c03fb8c7b1d5bd8dbb625d13bc6774e49cb5fd72c,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c7327d965050fa0e8fbc48300a8c6ce1bb04fad7e4e6349290dde68b53e9cffd],Size_:403086886,Uid:nil,Username:nobody,Spec:nil,},Info:map[string]string{},}" id=4247a64b-e98f-4e51-bcf5-419e2132c331 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.663398 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.663359184Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c7327d965050fa0e8fbc48300a8c6ce1bb04fad7e4e6349290dde68b53e9cffd" id=6aa870c3-8af8-43e0-bd54-2270000e4703 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.663597 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.663547985Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:c38c6316f11e9dad3a0b7f9c03fb8c7b1d5bd8dbb625d13bc6774e49cb5fd72c,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c7327d965050fa0e8fbc48300a8c6ce1bb04fad7e4e6349290dde68b53e9cffd],Size_:403086886,Uid:nil,Username:nobody,Spec:nil,},Info:map[string]string{},}" id=6aa870c3-8af8-43e0-bd54-2270000e4703 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.664540 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.664505892Z" level=info msg="Creating container: openshift-monitoring/prometheus-k8s-1/thanos-sidecar" id=4b9b9072-6068-444c-9d77-e3985d90b714 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:49.664649 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.664623893Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 15:06:49.756489 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09.scope. Nov 16 15:06:49.788740 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09. Nov 16 15:06:49.825038 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:06:49.892262 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.892040760Z" level=info msg="Created container f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09: openshift-monitoring/prometheus-k8s-1/thanos-sidecar" id=4b9b9072-6068-444c-9d77-e3985d90b714 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:49.892795 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.892701065Z" level=info msg="Starting container: f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09" id=73b6f335-5908-4e1a-b9c6-a242b151ffe9 name=/runtime.v1.RuntimeService/StartContainer Nov 16 15:06:49.903702 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.903640645Z" level=info msg="Started container" PID=204837 containerID=f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09 description=openshift-monitoring/prometheus-k8s-1/thanos-sidecar id=73b6f335-5908-4e1a-b9c6-a242b151ffe9 name=/runtime.v1.RuntimeService/StartContainer sandboxID=0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a Nov 16 15:06:49.929015 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.928960131Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634" id=0c68d5e1-74a2-4314-b2c2-ab1924da0286 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.929267 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.929233833Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:2f3a1ac7bb93faa2312d7fca21ca26dc55cbcd57502b470ad0bc40866a6ce73d,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634],Size_:340927490,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=0c68d5e1-74a2-4314-b2c2-ab1924da0286 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.930307 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.930267340Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634" id=66be0922-dede-4e7d-b92f-1a25bfc10391 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.930535 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.930492842Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:2f3a1ac7bb93faa2312d7fca21ca26dc55cbcd57502b470ad0bc40866a6ce73d,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:528e768b683ad82bf41a80e738c19fd43abb6d244c5b60fa24be966d4c9cd634],Size_:340927490,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=66be0922-dede-4e7d-b92f-1a25bfc10391 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:49.931678 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.931641750Z" level=info msg="Creating container: openshift-monitoring/prometheus-k8s-1/prometheus-proxy" id=5588c1b8-3f52-47f3-8e36-eef5ff6da463 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:49.931793 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:49.931762251Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 15:06:50.007959 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4.scope. Nov 16 15:06:50.037449 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4. Nov 16 15:06:50.078823 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:06:50.149452 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.149307446Z" level=info msg="Created container 526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4: openshift-monitoring/prometheus-k8s-1/prometheus-proxy" id=5588c1b8-3f52-47f3-8e36-eef5ff6da463 name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:50.149970 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.149929251Z" level=info msg="Starting container: 526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4" id=cd2cf2e6-485c-4eb1-8259-852b9b26ddf7 name=/runtime.v1.RuntimeService/StartContainer Nov 16 15:06:50.161872 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.161771438Z" level=info msg="Started container" PID=204886 containerID=526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4 description=openshift-monitoring/prometheus-k8s-1/prometheus-proxy id=cd2cf2e6-485c-4eb1-8259-852b9b26ddf7 name=/runtime.v1.RuntimeService/StartContainer sandboxID=0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a Nov 16 15:06:50.182425 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.182378789Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=c66fc150-0794-45c7-a7e8-4f5b0063f150 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:50.182667 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.182630291Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=c66fc150-0794-45c7-a7e8-4f5b0063f150 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:50.183612 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.183574698Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=c891ae74-7475-4472-b075-55dcce097a0f name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:50.183804 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.183771299Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=c891ae74-7475-4472-b075-55dcce097a0f name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:50.184660 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.184625205Z" level=info msg="Creating container: openshift-monitoring/prometheus-k8s-1/kube-rbac-proxy" id=fc6f9a61-deb5-46e8-9291-a9e67e226d2b name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:50.184756 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.184727006Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 15:06:50.185403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:50.185367 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerStarted Data:526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4} Nov 16 15:06:50.185744 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:50.185413 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerStarted Data:f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09} Nov 16 15:06:50.185744 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:50.185431 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerStarted Data:e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a} Nov 16 15:06:50.185744 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:50.185447 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerStarted Data:6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae} Nov 16 15:06:50.238254 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2.scope. Nov 16 15:06:50.260359 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container 9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2. Nov 16 15:06:50.301113 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:06:50.385687 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.385613679Z" level=info msg="Created container 9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2: openshift-monitoring/prometheus-k8s-1/kube-rbac-proxy" id=fc6f9a61-deb5-46e8-9291-a9e67e226d2b name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:50.386357 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.386313684Z" level=info msg="Starting container: 9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2" id=762bd9bf-448f-4d7c-ae71-7989915eeb1f name=/runtime.v1.RuntimeService/StartContainer Nov 16 15:06:50.398265 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.398215371Z" level=info msg="Started container" PID=204942 containerID=9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2 description=openshift-monitoring/prometheus-k8s-1/kube-rbac-proxy id=762bd9bf-448f-4d7c-ae71-7989915eeb1f name=/runtime.v1.RuntimeService/StartContainer sandboxID=0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a Nov 16 15:06:50.419117 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.418980423Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=47d79bb1-832e-438a-bc0d-3c0cd6856317 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:50.419306 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.419268026Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=47d79bb1-832e-438a-bc0d-3c0cd6856317 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:50.420315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.420277233Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7" id=5e350d79-74c0-4b5a-b7e1-10138d95c525 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:50.420519 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.420481634Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:5fedf22b30a570ef8b284e9e90e17922bfbd54b224c054338f4f9295f89f5ad4,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d9c95c485abf0c842ec6f7b0c82e05176b3f8351fe840ae5ffa4d91f0100fb7],Size_:375773669,Uid:&Int64Value{Value:65534,},Username:,Spec:nil,},Info:map[string]string{},}" id=5e350d79-74c0-4b5a-b7e1-10138d95c525 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:06:50.421722 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.421678643Z" level=info msg="Creating container: openshift-monitoring/prometheus-k8s-1/kube-rbac-proxy-thanos" id=acbb9587-fa5d-4b2c-a6c5-f4d47af2afbd name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:50.421817 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.421803444Z" level=warning msg="Allowed annotations are specified for workload [io.containers.trace-syscall]" Nov 16 15:06:50.474560 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started crio-conmon-db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95.scope. Nov 16 15:06:50.497714 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started libcontainer container db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95. Nov 16 15:06:50.532415 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Couldn't stat device /dev/char/10:200: No such file or directory Nov 16 15:06:50.605212 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.605116088Z" level=info msg="Created container db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95: openshift-monitoring/prometheus-k8s-1/kube-rbac-proxy-thanos" id=acbb9587-fa5d-4b2c-a6c5-f4d47af2afbd name=/runtime.v1.RuntimeService/CreateContainer Nov 16 15:06:50.605682 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.605648492Z" level=info msg="Starting container: db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95" id=c47a7bb3-e1a2-4707-9657-4056be62b0d7 name=/runtime.v1.RuntimeService/StartContainer Nov 16 15:06:50.617072 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:06:50.617007675Z" level=info msg="Started container" PID=204990 containerID=db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95 description=openshift-monitoring/prometheus-k8s-1/kube-rbac-proxy-thanos id=c47a7bb3-e1a2-4707-9657-4056be62b0d7 name=/runtime.v1.RuntimeService/StartContainer sandboxID=0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a Nov 16 15:06:51.193092 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:51.193049 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerStarted Data:db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95} Nov 16 15:06:51.193092 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:51.193103 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerStarted Data:9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2} Nov 16 15:06:55.710575 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:55.710528 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:55.710575 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:55.710589 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:55.769602 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:55.769565 1563 kubelet.go:2247] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:06:56.286609 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:06:56.286535 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-1" Nov 16 15:07:25.905820 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae-runc.MJAUPX.mount: Succeeded. Nov 16 15:07:25.905851 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae-runc.yCGctX.mount: Succeeded. Nov 16 15:08:20.509111 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:08:20.508926179Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=0172327f-7c6c-402e-a311-4ef9e5043584 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:08:20.525836 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:08:20.509292182Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=0172327f-7c6c-402e-a311-4ef9e5043584 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:11:05.455335 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1176]: [system] Activating via systemd: service name='org.projectatomic.rpmostree1' unit='rpm-ostreed.service' requested by ':1.680' (uid=0 pid=212176 comm="rpm-ostree kargs " label="system_u:system_r:spc_t:s0") Nov 16 15:11:05.459175 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Starting rpm-ostree System Management Daemon... Nov 16 15:11:05.490898 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[212179]: Reading config file '/etc/rpm-ostreed.conf' Nov 16 15:11:05.498844 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[212179]: In idle state; will auto-exit in 64 seconds Nov 16 15:11:05.499223 ropatil1611az-gs6zj-worker-westus-j4ngg dbus-daemon[1176]: [system] Successfully activated service 'org.projectatomic.rpmostree1' Nov 16 15:11:05.499387 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started rpm-ostree System Management Daemon. Nov 16 15:11:05.503588 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[212179]: client(id:machine-config-operator dbus:1.680 unit:crio-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope uid:0) added; new total=1 Nov 16 15:11:05.514777 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[212179]: client(id:machine-config-operator dbus:1.680 unit:crio-cfc64e3fc19348063ef2da4fba1a93ccb9487454cb23ce177c3e085dc26aef5a.scope uid:0) vanished; remaining=0 Nov 16 15:11:05.514801 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[212179]: In idle state; will auto-exit in 61 seconds Nov 16 15:11:05.612928 ropatil1611az-gs6zj-worker-westus-j4ngg root[212184]: machine-config-daemon[200509]: Starting update from rendered-worker-f6b2243eda256f645217bce72b92c7db to rendered-worker-f49365d62e1aca3c8af88f3d276c79a2: &{osUpdate:true kargs:false fips:false passwd:false files:true units:true kernelType:false extensions:false} Nov 16 15:11:05.616438 ropatil1611az-gs6zj-worker-westus-j4ngg root[212185]: machine-config-daemon[200509]: Update prepared; requesting cordon and drain via annotation to controller Nov 16 15:11:10.354748 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.354529 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4] Nov 16 15:11:10.355336 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.354866 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" podUID=f5df478d-fc7b-4fda-b15e-1a9a05fdf611 containerName="mydep-49678" containerID="cri-o://7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d" gracePeriod=30 Nov 16 15:11:10.355611 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.355551274Z" level=info msg="Stopping container: 7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d (timeout: 30s)" id=26cc8829-80ca-41e1-8b82-c080c72ea63f name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.357880 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.357849 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-9hthz] Nov 16 15:11:10.358062 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.358037 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-marketplace/qe-app-registry-9hthz" podUID=5eb5be5f-46bd-425a-a486-e2641bb0ad88 containerName="registry-server" containerID="cri-o://ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454" gracePeriod=30 Nov 16 15:11:10.358287 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.358242697Z" level=info msg="Stopping container: ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454 (timeout: 30s)" id=8f46451d-0151-4197-addd-45b672872d5b name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.360775 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.360719 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[migration-upgrade-48669/mydep-48669-79647ff996-g97kh] Nov 16 15:11:10.361081 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.360962 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" podUID=5df14924-dd8b-4deb-a83b-53ff611d5e47 containerName="mydep-48669" containerID="cri-o://2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a" gracePeriod=30 Nov 16 15:11:10.361439 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.361398123Z" level=info msg="Stopping container: 2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a (timeout: 30s)" id=e892d376-3943-4f66-87a9-ddc8ef466bae name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.364458 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.364431 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv] Nov 16 15:11:10.364864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.364831 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" podUID=9502a499-9f8b-4a15-99d8-714a0cc37bea containerName="manager" containerID="cri-o://8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4" gracePeriod=10 Nov 16 15:11:10.364967 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.364865 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" podUID=9502a499-9f8b-4a15-99d8-714a0cc37bea containerName="kube-rbac-proxy" containerID="cri-o://2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb" gracePeriod=10 Nov 16 15:11:10.368267 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.368237 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx] Nov 16 15:11:10.368464 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.368418582Z" level=info msg="Stopping container: 8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4 (timeout: 10s)" id=2e33a9b8-5dce-4b7d-83eb-5a4af9a73a3a name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.368572 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.368452 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" podUID=e3d0df21-083b-4e97-a997-e9a7e992eda4 containerName="mydep-49496" containerID="cri-o://e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8" gracePeriod=30 Nov 16 15:11:10.368702 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.368434682Z" level=info msg="Stopping container: 2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb (timeout: 10s)" id=01b98aba-5a81-4687-b304-ec9c8238d15c name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.369677 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.369634993Z" level=info msg="Stopping container: e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8 (timeout: 30s)" id=8e1c0e95-7eb5-4e7a-9d42-0fd12fc5fc5c name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.373183 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.371635 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/kube-state-metrics-68667ff848-f92nw] Nov 16 15:11:10.373183 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.372297 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" podUID=454b3a37-199c-4abc-a79a-8de93eab2423 containerName="kube-state-metrics" containerID="cri-o://d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610" gracePeriod=30 Nov 16 15:11:10.373183 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.372383 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" podUID=454b3a37-199c-4abc-a79a-8de93eab2423 containerName="kube-rbac-proxy-self" containerID="cri-o://6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b" gracePeriod=30 Nov 16 15:11:10.373183 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.372452 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" podUID=454b3a37-199c-4abc-a79a-8de93eab2423 containerName="kube-rbac-proxy-main" containerID="cri-o://4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4" gracePeriod=30 Nov 16 15:11:10.374996 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.374468433Z" level=info msg="Stopping container: d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610 (timeout: 30s)" id=01c05030-f52a-4b04-ae43-7eb8fb9a5487 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.374996 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.374546934Z" level=info msg="Stopping container: 6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b (timeout: 30s)" id=373b7569-7efc-4051-847e-07a964711226 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.374996 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.374516034Z" level=info msg="Stopping container: 4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4 (timeout: 30s)" id=eb0f9944-bf25-4bf3-abb7-c88797c494b8 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.382213 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.379176 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-image-registry/image-registry-944cf86d8-kb2fb] Nov 16 15:11:10.382213 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.381088 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg] Nov 16 15:11:10.382213 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.381277 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" podUID=4e58fe69-e225-425e-a6ca-0a7f136dbdca containerName="mydep-50425" containerID="cri-o://7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858" gracePeriod=30 Nov 16 15:11:10.382439 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.381469392Z" level=info msg="Stopping container: 7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858 (timeout: 30s)" id=773bd3f3-f401-4265-8f0c-560d4b35909b name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.383804 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.383764 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm] Nov 16 15:11:10.384257 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.384041 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" podUID=a776019a-1ce1-4328-aed5-a5962a4f775d containerName="thanos-query" containerID="cri-o://94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d" gracePeriod=120 Nov 16 15:11:10.384257 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.384072 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" podUID=a776019a-1ce1-4328-aed5-a5962a4f775d containerName="kube-rbac-proxy-rules" containerID="cri-o://f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665" gracePeriod=120 Nov 16 15:11:10.384257 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.384111 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" podUID=a776019a-1ce1-4328-aed5-a5962a4f775d containerName="prom-label-proxy" containerID="cri-o://8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619" gracePeriod=120 Nov 16 15:11:10.384257 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.384080 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" podUID=a776019a-1ce1-4328-aed5-a5962a4f775d containerName="kube-rbac-proxy" containerID="cri-o://fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf" gracePeriod=120 Nov 16 15:11:10.384257 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.384231 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" podUID=a776019a-1ce1-4328-aed5-a5962a4f775d containerName="kube-rbac-proxy-metrics" containerID="cri-o://0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0" gracePeriod=120 Nov 16 15:11:10.384257 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.384154 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" podUID=a776019a-1ce1-4328-aed5-a5962a4f775d containerName="oauth-proxy" containerID="cri-o://a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222" gracePeriod=120 Nov 16 15:11:10.384532 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.384479017Z" level=info msg="Stopping container: a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222 (timeout: 120s)" id=9599626e-a3de-47e6-99ce-170bc272eb7d name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.385443 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.384664719Z" level=info msg="Stopping container: 94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d (timeout: 120s)" id=6097dcae-59c7-4742-bc58-b26564928284 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.385443 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.384799620Z" level=info msg="Stopping container: f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665 (timeout: 120s)" id=4c6ba989-ebd1-40f1-b475-25249f571490 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.385443 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.385152523Z" level=info msg="Stopping container: fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf (timeout: 120s)" id=3767722d-fe8f-4c5a-b14b-d41e7980928b name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.385443 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.385222024Z" level=info msg="Stopping container: 0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0 (timeout: 120s)" id=450decc4-623a-4ece-b6e1-6240f917c44e name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.385443 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.385289624Z" level=info msg="Stopping container: 8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619 (timeout: 120s)" id=9ee7548c-d821-4bda-89bc-961a8b2eb778 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.386213 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d.scope: Succeeded. Nov 16 15:11:10.386881 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d.scope: Consumed 38ms CPU time Nov 16 15:11:10.389845 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d.scope: Succeeded. Nov 16 15:11:10.390445 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d.scope: Consumed 55ms CPU time Nov 16 15:11:10.392332 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.392119 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/prometheus-k8s-1] Nov 16 15:11:10.392459 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.392414 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-1" podUID=3670081b-e385-4685-8dda-b3a3ee01f233 containerName="prometheus" containerID="cri-o://6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae" gracePeriod=600 Nov 16 15:11:10.393145 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.392575 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-1" podUID=3670081b-e385-4685-8dda-b3a3ee01f233 containerName="kube-rbac-proxy-thanos" containerID="cri-o://db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95" gracePeriod=600 Nov 16 15:11:10.393145 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.392690 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-1" podUID=3670081b-e385-4685-8dda-b3a3ee01f233 containerName="kube-rbac-proxy" containerID="cri-o://9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2" gracePeriod=600 Nov 16 15:11:10.393145 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.392760 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-1" podUID=3670081b-e385-4685-8dda-b3a3ee01f233 containerName="prometheus-proxy" containerID="cri-o://526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4" gracePeriod=600 Nov 16 15:11:10.393145 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.392811 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-1" podUID=3670081b-e385-4685-8dda-b3a3ee01f233 containerName="thanos-sidecar" containerID="cri-o://f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09" gracePeriod=600 Nov 16 15:11:10.393145 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.392858 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-1" podUID=3670081b-e385-4685-8dda-b3a3ee01f233 containerName="config-reloader" containerID="cri-o://e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a" gracePeriod=600 Nov 16 15:11:10.394177 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.393573094Z" level=info msg="Stopping container: e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a (timeout: 600s)" id=896640ed-852c-4d58-878c-0f3dab152fa5 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.394177 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.393760995Z" level=info msg="Stopping container: 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae (timeout: 600s)" id=7557d587-deb2-49b5-8742-1bf5b6102464 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.394177 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.393882496Z" level=info msg="Stopping container: db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95 (timeout: 600s)" id=24e88f0a-354e-44f0-b118-c495b37c7816 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.394177 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.394037598Z" level=info msg="Stopping container: 9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2 (timeout: 600s)" id=856688bc-6cee-4e17-9d8a-a0b536059296 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.395186 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.394420701Z" level=info msg="Stopping container: 526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4 (timeout: 600s)" id=e6df18d0-2296-4912-baf8-cdea244e869c name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.395186 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.394570602Z" level=info msg="Stopping container: f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09 (timeout: 600s)" id=77d9c072-1221-4f11-971d-6d4ef13804a9 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.395617 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a.scope: Succeeded. Nov 16 15:11:10.396287 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a.scope: Consumed 53ms CPU time Nov 16 15:11:10.403203 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454.scope: Succeeded. Nov 16 15:11:10.403846 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454.scope: Consumed 6.590s CPU time Nov 16 15:11:10.404190 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a.scope: Succeeded. Nov 16 15:11:10.404762 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a.scope: Consumed 39ms CPU time Nov 16 15:11:10.404937 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454.scope: Succeeded. Nov 16 15:11:10.405629 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454.scope: Consumed 40ms CPU time Nov 16 15:11:10.414580 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8.scope: Succeeded. Nov 16 15:11:10.415434 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8.scope: Consumed 108ms CPU time Nov 16 15:11:10.415901 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8.scope: Succeeded. Nov 16 15:11:10.416521 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8.scope: Consumed 43ms CPU time Nov 16 15:11:10.427355 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4.scope: Succeeded. Nov 16 15:11:10.428057 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4.scope: Consumed 32.792s CPU time Nov 16 15:11:10.440046 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4.scope: Succeeded. Nov 16 15:11:10.441261 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4.scope: Consumed 39ms CPU time Nov 16 15:11:10.443044 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222.scope: Succeeded. Nov 16 15:11:10.443843 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222.scope: Consumed 38ms CPU time Nov 16 15:11:10.449070 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665.scope: Succeeded. Nov 16 15:11:10.449685 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665.scope: Consumed 205ms CPU time Nov 16 15:11:10.449940 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610.scope: Succeeded. Nov 16 15:11:10.454222 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610.scope: Consumed 4.997s CPU time Nov 16 15:11:10.465048 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222.scope: Succeeded. Nov 16 15:11:10.469427 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222.scope: Consumed 4.032s CPU time Nov 16 15:11:10.469902 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610.scope: Succeeded. Nov 16 15:11:10.470592 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610.scope: Consumed 37ms CPU time Nov 16 15:11:10.470772 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858.scope: Succeeded. Nov 16 15:11:10.471473 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858.scope: Consumed 75ms CPU time Nov 16 15:11:10.471856 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858.scope: Succeeded. Nov 16 15:11:10.472638 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858.scope: Consumed 39ms CPU time Nov 16 15:11:10.472804 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619.scope: Succeeded. Nov 16 15:11:10.473430 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619.scope: Consumed 73ms CPU time Nov 16 15:11:10.473939 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf.scope: Succeeded. Nov 16 15:11:10.474703 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf.scope: Consumed 198ms CPU time Nov 16 15:11:10.475014 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665.scope: Succeeded. Nov 16 15:11:10.496590 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665.scope: Consumed 41ms CPU time Nov 16 15:11:10.497233 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a.scope: Succeeded. Nov 16 15:11:10.497793 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a.scope: Consumed 130ms CPU time Nov 16 15:11:10.498042 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a.scope: Succeeded. Nov 16 15:11:10.498686 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a.scope: Consumed 37ms CPU time Nov 16 15:11:10.498937 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4.scope: Succeeded. Nov 16 15:11:10.499501 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4.scope: Consumed 997ms CPU time Nov 16 15:11:10.499716 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09.scope: Succeeded. Nov 16 15:11:10.500296 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09.scope: Consumed 651ms CPU time Nov 16 15:11:10.500520 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d.scope: Succeeded. Nov 16 15:11:10.501055 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d.scope: Consumed 1.437s CPU time Nov 16 15:11:10.501292 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4.scope: Succeeded. Nov 16 15:11:10.501837 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4.scope: Consumed 39ms CPU time Nov 16 15:11:10.501991 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d.scope: Succeeded. Nov 16 15:11:10.502545 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d.scope: Consumed 37ms CPU time Nov 16 15:11:10.502704 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf.scope: Succeeded. Nov 16 15:11:10.503239 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf.scope: Consumed 35ms CPU time Nov 16 15:11:10.503405 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619.scope: Succeeded. Nov 16 15:11:10.503930 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619.scope: Consumed 43ms CPU time Nov 16 15:11:10.514935 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09.scope: Succeeded. Nov 16 15:11:10.516249 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09.scope: Consumed 40ms CPU time Nov 16 15:11:10.543087 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-328cb1edef88343547ccfbce4d71b8858b1aa7faa25498603d677055e1336a22-merged.mount: Succeeded. Nov 16 15:11:10.544207 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-328cb1edef88343547ccfbce4d71b8858b1aa7faa25498603d677055e1336a22-merged.mount: Consumed 0 CPU time Nov 16 15:11:10.562078 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae.scope: Succeeded. Nov 16 15:11:10.562816 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae.scope: Consumed 41.066s CPU time Nov 16 15:11:10.573329 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae.scope: Succeeded. Nov 16 15:11:10.573954 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae.scope: Consumed 52ms CPU time Nov 16 15:11:10.600706 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.599853 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 15:11:10.600706 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.600160 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=e0a5ccca-d055-4b4c-820e-515fbf3f92d5 containerName="alertmanager" containerID="cri-o://96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972" gracePeriod=120 Nov 16 15:11:10.600706 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.600398 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=e0a5ccca-d055-4b4c-820e-515fbf3f92d5 containerName="prom-label-proxy" containerID="cri-o://eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5" gracePeriod=120 Nov 16 15:11:10.600706 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.600463 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=e0a5ccca-d055-4b4c-820e-515fbf3f92d5 containerName="kube-rbac-proxy-metric" containerID="cri-o://d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3" gracePeriod=120 Nov 16 15:11:10.600706 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.600526 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=e0a5ccca-d055-4b4c-820e-515fbf3f92d5 containerName="kube-rbac-proxy" containerID="cri-o://aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1" gracePeriod=120 Nov 16 15:11:10.600706 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.600577 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=e0a5ccca-d055-4b4c-820e-515fbf3f92d5 containerName="alertmanager-proxy" containerID="cri-o://c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651" gracePeriod=120 Nov 16 15:11:10.600706 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.600621 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID=e0a5ccca-d055-4b4c-820e-515fbf3f92d5 containerName="config-reloader" containerID="cri-o://27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518" gracePeriod=120 Nov 16 15:11:10.602599 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.602333949Z" level=info msg="Stopping container: 27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518 (timeout: 120s)" id=a5cbe5a1-c52d-47ff-9496-af7d1dba7288 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.602712 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.602688552Z" level=info msg="Stopping container: 96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972 (timeout: 120s)" id=0f8c39cc-15fc-4d76-8995-51000bf70d73 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.604239 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.602838553Z" level=info msg="Stopping container: eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5 (timeout: 120s)" id=e6d52273-dc20-46cc-ae9d-cef02805e30f name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.604239 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.602975755Z" level=info msg="Stopping container: d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3 (timeout: 120s)" id=91af6143-6266-4a92-8c13-8ac5acecff01 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.604239 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.603095256Z" level=info msg="Stopping container: aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1 (timeout: 120s)" id=0ddb3a1b-d6ce-49eb-b829-4aa611b8a936 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.604239 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.603244557Z" level=info msg="Stopping container: c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651 (timeout: 120s)" id=1818c301-c5fa-474f-8dbb-e8db85a675d7 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.650577 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518.scope: Succeeded. Nov 16 15:11:10.651502 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518.scope: Consumed 39ms CPU time Nov 16 15:11:10.653848 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518.scope: Succeeded. Nov 16 15:11:10.655028 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518.scope: Consumed 108ms CPU time Nov 16 15:11:10.655458 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5.scope: Succeeded. Nov 16 15:11:10.656179 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5.scope: Consumed 38ms CPU time Nov 16 15:11:10.656635 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651.scope: Succeeded. Nov 16 15:11:10.657202 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651.scope: Consumed 43ms CPU time Nov 16 15:11:10.662504 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651.scope: Succeeded. Nov 16 15:11:10.663829 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651.scope: Consumed 2.626s CPU time Nov 16 15:11:10.664091 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1.scope: Succeeded. Nov 16 15:11:10.665072 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1.scope: Consumed 171ms CPU time Nov 16 15:11:10.711265 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:10.711227 1563 remote_runtime.go:704] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae is running failed: container process not found" containerID="6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae" cmd=[sh -c if [ -x "$(command -v curl)" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x "$(command -v wget)" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi] Nov 16 15:11:10.711623 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:10.711600 1563 remote_runtime.go:704] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae is running failed: container process not found" containerID="6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae" cmd=[sh -c if [ -x "$(command -v curl)" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x "$(command -v wget)" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi] Nov 16 15:11:10.711895 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:10.711873 1563 remote_runtime.go:704] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae is running failed: container process not found" containerID="6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae" cmd=[sh -c if [ -x "$(command -v curl)" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x "$(command -v wget)" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi] Nov 16 15:11:10.711945 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:10.711918 1563 prober.go:118] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae is running failed: container process not found" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-1" podUID=3670081b-e385-4685-8dda-b3a3ee01f233 containerName="prometheus" Nov 16 15:11:10.749797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.749000 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/telemeter-client-55c5556665-7vs77] Nov 16 15:11:10.749797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.749325 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" podUID=add06140-663d-48c0-ac00-643bd58a3f7d containerName="telemeter-client" containerID="cri-o://3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d" gracePeriod=30 Nov 16 15:11:10.749797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.749651 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" podUID=add06140-663d-48c0-ac00-643bd58a3f7d containerName="kube-rbac-proxy" containerID="cri-o://c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e" gracePeriod=30 Nov 16 15:11:10.749797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.749719 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" podUID=add06140-663d-48c0-ac00-643bd58a3f7d containerName="reload" containerID="cri-o://d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487" gracePeriod=30 Nov 16 15:11:10.750471 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.750419894Z" level=info msg="Stopping container: d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487 (timeout: 30s)" id=81d4fbd8-0f68-41dc-8ed2-3a00d4d452e1 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.750856 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.750714997Z" level=info msg="Stopping container: 3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d (timeout: 30s)" id=9cfd00b5-109a-4dc7-9901-991b7dc5da95 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.750856 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.750760997Z" level=info msg="Stopping container: c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e (timeout: 30s)" id=5dc6d760-b384-4192-9ad4-d08265fd0c51 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:10.782052 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487.scope: Succeeded. Nov 16 15:11:10.782708 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487.scope: Consumed 44ms CPU time Nov 16 15:11:10.783070 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d.scope: Succeeded. Nov 16 15:11:10.783739 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d.scope: Consumed 41ms CPU time Nov 16 15:11:10.785161 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487.scope: Succeeded. Nov 16 15:11:10.785710 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487.scope: Consumed 86ms CPU time Nov 16 15:11:10.785928 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d.scope: Succeeded. Nov 16 15:11:10.786491 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d.scope: Consumed 562ms CPU time Nov 16 15:11:10.981911 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.981756 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj] Nov 16 15:11:10.982189 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:10.982018 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj" podUID=35f6c830-e480-446f-a5cc-a6aaf007ea9a containerName="prometheus-operator-admission-webhook" containerID="cri-o://cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778" gracePeriod=30 Nov 16 15:11:10.982678 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:10.982446045Z" level=info msg="Stopping container: cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778 (timeout: 30s)" id=f8f4987a-8c69-4ebb-9d80-d7112f7a2304 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:11.014504 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778.scope: Succeeded. Nov 16 15:11:11.015186 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778.scope: Consumed 2.207s CPU time Nov 16 15:11:11.019319 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1.scope: Succeeded. Nov 16 15:11:11.020008 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1.scope: Consumed 38ms CPU time Nov 16 15:11:11.020283 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5.scope: Succeeded. Nov 16 15:11:11.020852 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5.scope: Consumed 38ms CPU time Nov 16 15:11:11.093414 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.093370 1563 generic.go:296] "Generic (PLEG): container finished" podID=5eb5be5f-46bd-425a-a486-e2641bb0ad88 containerID="ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454" exitCode=0 Nov 16 15:11:11.093609 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.093460 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-9hthz" event=&{ID:5eb5be5f-46bd-425a-a486-e2641bb0ad88 Type:ContainerDied Data:ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454} Nov 16 15:11:11.095323 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.095298 1563 generic.go:296] "Generic (PLEG): container finished" podID=9502a499-9f8b-4a15-99d8-714a0cc37bea containerID="8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4" exitCode=0 Nov 16 15:11:11.095416 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.095368 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" event=&{ID:9502a499-9f8b-4a15-99d8-714a0cc37bea Type:ContainerDied Data:8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4} Nov 16 15:11:11.096621 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.096597 1563 generic.go:296] "Generic (PLEG): container finished" podID=4e58fe69-e225-425e-a6ca-0a7f136dbdca containerID="7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858" exitCode=2 Nov 16 15:11:11.096718 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.096670 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" event=&{ID:4e58fe69-e225-425e-a6ca-0a7f136dbdca Type:ContainerDied Data:7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858} Nov 16 15:11:11.098553 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.098524 1563 generic.go:296] "Generic (PLEG): container finished" podID=454b3a37-199c-4abc-a79a-8de93eab2423 containerID="d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610" exitCode=2 Nov 16 15:11:11.098663 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.098570 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" event=&{ID:454b3a37-199c-4abc-a79a-8de93eab2423 Type:ContainerDied Data:d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610} Nov 16 15:11:11.100438 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.100414 1563 generic.go:296] "Generic (PLEG): container finished" podID=add06140-663d-48c0-ac00-643bd58a3f7d containerID="d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487" exitCode=2 Nov 16 15:11:11.100438 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.100443 1563 generic.go:296] "Generic (PLEG): container finished" podID=add06140-663d-48c0-ac00-643bd58a3f7d containerID="3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d" exitCode=2 Nov 16 15:11:11.100554 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.100467 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" event=&{ID:add06140-663d-48c0-ac00-643bd58a3f7d Type:ContainerDied Data:d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487} Nov 16 15:11:11.100554 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.100503 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" event=&{ID:add06140-663d-48c0-ac00-643bd58a3f7d Type:ContainerDied Data:3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d} Nov 16 15:11:11.104773 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.104743 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-1_3670081b-e385-4685-8dda-b3a3ee01f233/prometheus-proxy/0.log" Nov 16 15:11:11.105536 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.105512 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-1_3670081b-e385-4685-8dda-b3a3ee01f233/config-reloader/0.log" Nov 16 15:11:11.106231 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.106209 1563 generic.go:296] "Generic (PLEG): container finished" podID=3670081b-e385-4685-8dda-b3a3ee01f233 containerID="526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4" exitCode=2 Nov 16 15:11:11.106231 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.106231 1563 generic.go:296] "Generic (PLEG): container finished" podID=3670081b-e385-4685-8dda-b3a3ee01f233 containerID="f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09" exitCode=0 Nov 16 15:11:11.106361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.106244 1563 generic.go:296] "Generic (PLEG): container finished" podID=3670081b-e385-4685-8dda-b3a3ee01f233 containerID="e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a" exitCode=2 Nov 16 15:11:11.106361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.106258 1563 generic.go:296] "Generic (PLEG): container finished" podID=3670081b-e385-4685-8dda-b3a3ee01f233 containerID="6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae" exitCode=0 Nov 16 15:11:11.106361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.106276 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerDied Data:526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4} Nov 16 15:11:11.106361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.106309 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerDied Data:f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09} Nov 16 15:11:11.106361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.106328 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerDied Data:e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a} Nov 16 15:11:11.106361 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.106343 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerDied Data:6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae} Nov 16 15:11:11.107791 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.107764 1563 generic.go:296] "Generic (PLEG): container finished" podID=e3d0df21-083b-4e97-a997-e9a7e992eda4 containerID="e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8" exitCode=2 Nov 16 15:11:11.107881 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.107835 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" event=&{ID:e3d0df21-083b-4e97-a997-e9a7e992eda4 Type:ContainerDied Data:e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8} Nov 16 15:11:11.118196 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.118163 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b6b7d7fd9-d95wm_a776019a-1ce1-4328-aed5-a5962a4f775d/oauth-proxy/0.log" Nov 16 15:11:11.118572 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.118547 1563 generic.go:296] "Generic (PLEG): container finished" podID=a776019a-1ce1-4328-aed5-a5962a4f775d containerID="f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665" exitCode=0 Nov 16 15:11:11.118572 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.118578 1563 generic.go:296] "Generic (PLEG): container finished" podID=a776019a-1ce1-4328-aed5-a5962a4f775d containerID="8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619" exitCode=0 Nov 16 15:11:11.118674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.118592 1563 generic.go:296] "Generic (PLEG): container finished" podID=a776019a-1ce1-4328-aed5-a5962a4f775d containerID="fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf" exitCode=0 Nov 16 15:11:11.118674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.118605 1563 generic.go:296] "Generic (PLEG): container finished" podID=a776019a-1ce1-4328-aed5-a5962a4f775d containerID="a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222" exitCode=2 Nov 16 15:11:11.118674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.118617 1563 generic.go:296] "Generic (PLEG): container finished" podID=a776019a-1ce1-4328-aed5-a5962a4f775d containerID="94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d" exitCode=0 Nov 16 15:11:11.118674 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.118650 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerDied Data:f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665} Nov 16 15:11:11.118802 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.118686 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerDied Data:8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619} Nov 16 15:11:11.118802 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.118705 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerDied Data:fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf} Nov 16 15:11:11.118802 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.118735 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerDied Data:a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222} Nov 16 15:11:11.118802 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.118754 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerDied Data:94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d} Nov 16 15:11:11.119890 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.119863 1563 generic.go:296] "Generic (PLEG): container finished" podID=5df14924-dd8b-4deb-a83b-53ff611d5e47 containerID="2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a" exitCode=2 Nov 16 15:11:11.119981 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.119953 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" event=&{ID:5df14924-dd8b-4deb-a83b-53ff611d5e47 Type:ContainerDied Data:2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a} Nov 16 15:11:11.122172 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.122147 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a5ccca-d055-4b4c-820e-515fbf3f92d5/alertmanager-proxy/0.log" Nov 16 15:11:11.122589 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.122566 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a5ccca-d055-4b4c-820e-515fbf3f92d5/config-reloader/0.log" Nov 16 15:11:11.122890 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.122869 1563 generic.go:296] "Generic (PLEG): container finished" podID=e0a5ccca-d055-4b4c-820e-515fbf3f92d5 containerID="eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5" exitCode=0 Nov 16 15:11:11.122890 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.122893 1563 generic.go:296] "Generic (PLEG): container finished" podID=e0a5ccca-d055-4b4c-820e-515fbf3f92d5 containerID="aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1" exitCode=0 Nov 16 15:11:11.122992 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.122902 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerDied Data:eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5} Nov 16 15:11:11.122992 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.122931 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerDied Data:aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1} Nov 16 15:11:11.122992 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.122949 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerDied Data:c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651} Nov 16 15:11:11.122992 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.122907 1563 generic.go:296] "Generic (PLEG): container finished" podID=e0a5ccca-d055-4b4c-820e-515fbf3f92d5 containerID="c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651" exitCode=2 Nov 16 15:11:11.122992 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.122974 1563 generic.go:296] "Generic (PLEG): container finished" podID=e0a5ccca-d055-4b4c-820e-515fbf3f92d5 containerID="27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518" exitCode=2 Nov 16 15:11:11.123178 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.123023 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerDied Data:27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518} Nov 16 15:11:11.124450 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.124425 1563 generic.go:296] "Generic (PLEG): container finished" podID=f5df478d-fc7b-4fda-b15e-1a9a05fdf611 containerID="7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d" exitCode=2 Nov 16 15:11:11.124538 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.124466 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" event=&{ID:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Type:ContainerDied Data:7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d} Nov 16 15:11:11.169152 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.169086 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-ingress/router-default-cff655768-nrbzb] Nov 16 15:11:11.169368 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.169342 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-ingress/router-default-cff655768-nrbzb" podUID=03363f87-776c-47c3-9bd8-8c0de24a4de5 containerName="router" containerID="cri-o://5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3" gracePeriod=3600 Nov 16 15:11:11.169994 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:11.169948922Z" level=info msg="Stopping container: 5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3 (timeout: 3600s)" id=522df1d8-436a-4ad1-9093-8a918a9b9229 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:11.351789 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.351687 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv] Nov 16 15:11:11.352061 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.352011 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" podUID=c8013b4f-15b6-4bac-bfc9-a45301362f0c containerName="kube-rbac-proxy-main" containerID="cri-o://89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25" gracePeriod=30 Nov 16 15:11:11.352196 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.352122 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" podUID=c8013b4f-15b6-4bac-bfc9-a45301362f0c containerName="openshift-state-metrics" containerID="cri-o://ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778" gracePeriod=30 Nov 16 15:11:11.352249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.352215 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" podUID=c8013b4f-15b6-4bac-bfc9-a45301362f0c containerName="kube-rbac-proxy-self" containerID="cri-o://83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a" gracePeriod=30 Nov 16 15:11:11.352510 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:11.352455756Z" level=info msg="Stopping container: 89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25 (timeout: 30s)" id=17470c70-5853-4b7f-8990-ebe08eaa13d5 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:11.352510 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:11.352456056Z" level=info msg="Stopping container: 83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a (timeout: 30s)" id=fc4d9ec5-cc20-4bca-999c-ee8181f4ca3b name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:11.352697 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:11.352455756Z" level=info msg="Stopping container: ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778 (timeout: 30s)" id=ce96b784-0075-40c0-9f9a-cd3c1a17a266 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:11.378050 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778.scope: Succeeded. Nov 16 15:11:11.378759 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778.scope: Consumed 33ms CPU time Nov 16 15:11:11.381338 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778.scope: Succeeded. Nov 16 15:11:11.381976 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778.scope: Consumed 384ms CPU time Nov 16 15:11:11.484301 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb.scope: Succeeded. Nov 16 15:11:11.484978 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb.scope: Consumed 4.153s CPU time Nov 16 15:11:11.500206 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b.scope: Succeeded. Nov 16 15:11:11.500866 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b.scope: Consumed 823ms CPU time Nov 16 15:11:11.508359 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4.scope: Succeeded. Nov 16 15:11:11.508985 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4.scope: Consumed 1.834s CPU time Nov 16 15:11:11.550362 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.550316 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt] Nov 16 15:11:11.550845 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.550628 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" podUID=220ce491-f766-41ee-ab02-aa1546eb7ed3 containerName="kube-rbac-proxy" containerID="cri-o://881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667" gracePeriod=30 Nov 16 15:11:11.550845 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.550714 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" podUID=220ce491-f766-41ee-ab02-aa1546eb7ed3 containerName="elasticsearch-operator" containerID="cri-o://dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf" gracePeriod=30 Nov 16 15:11:11.550976 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:11.550930725Z" level=info msg="Stopping container: 881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667 (timeout: 30s)" id=055def66-9486-45dd-b8ea-05fb121c2577 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:11.551270 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:11.550930725Z" level=info msg="Stopping container: dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf (timeout: 30s)" id=90941735-5f36-4363-9de4-13b860e2e32b name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:11.551539 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0.scope: Succeeded. Nov 16 15:11:11.552227 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0.scope: Consumed 875ms CPU time Nov 16 15:11:11.579220 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972.scope: Succeeded. Nov 16 15:11:11.579880 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972.scope: Consumed 3.330s CPU time Nov 16 15:11:11.580726 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf.scope: Succeeded. Nov 16 15:11:11.581367 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf.scope: Consumed 30.460s CPU time Nov 16 15:11:11.581608 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2.scope: Succeeded. Nov 16 15:11:11.582211 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2.scope: Consumed 266ms CPU time Nov 16 15:11:11.592068 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95.scope: Succeeded. Nov 16 15:11:11.592753 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95.scope: Consumed 230ms CPU time Nov 16 15:11:11.622968 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.622831 1563 patch_prober.go:29] interesting pod/prometheus-operator-admission-webhook-6c69dfd44-hxqbj container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.129.2.26:8443/healthz\": dial tcp 10.129.2.26:8443: connect: connection refused" start-of-body= Nov 16 15:11:11.622968 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.622909 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj" podUID=35f6c830-e480-446f-a5cc-a6aaf007ea9a containerName="prometheus-operator-admission-webhook" probeResult=failure output="Get \"https://10.129.2.26:8443/healthz\": dial tcp 10.129.2.26:8443: connect: connection refused" Nov 16 15:11:11.672673 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778.scope: Succeeded. Nov 16 15:11:11.673337 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778.scope: Consumed 42ms CPU time Nov 16 15:11:11.673562 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972.scope: Succeeded. Nov 16 15:11:11.674192 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972.scope: Consumed 38ms CPU time Nov 16 15:11:11.734173 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3.scope: Succeeded. Nov 16 15:11:11.734823 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3.scope: Consumed 802ms CPU time Nov 16 15:11:11.753901 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.753824 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf] Nov 16 15:11:11.754119 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.754091 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf" podUID=b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd containerName="cluster-logging-operator" containerID="cri-o://d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c" gracePeriod=30 Nov 16 15:11:11.754472 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:11.754411436Z" level=info msg="Stopping container: d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c (timeout: 30s)" id=4b406790-cf1f-4934-add8-34eb3a2d136f name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:11.783358 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c.scope: Succeeded. Nov 16 15:11:11.784064 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c.scope: Consumed 36ms CPU time Nov 16 15:11:11.786874 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c.scope: Succeeded. Nov 16 15:11:11.787480 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c.scope: Consumed 10.969s CPU time Nov 16 15:11:11.831658 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-67f7e5f1c87f7d52077e7b6877b577a05923513938508ef850746a7cd8113782-merged.mount: Succeeded. Nov 16 15:11:11.832596 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-67f7e5f1c87f7d52077e7b6877b577a05923513938508ef850746a7cd8113782-merged.mount: Consumed 0 CPU time Nov 16 15:11:11.833518 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0.scope: Succeeded. Nov 16 15:11:11.834108 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0.scope: Consumed 43ms CPU time Nov 16 15:11:11.834410 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4.scope: Succeeded. Nov 16 15:11:11.837465 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:11.837266832Z" level=info msg="Stopped container 2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a: migration-upgrade-48669/mydep-48669-79647ff996-g97kh/mydep-48669" id=e892d376-3943-4f66-87a9-ddc8ef466bae name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:11.837378 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4.scope: Consumed 39ms CPU time Nov 16 15:11:11.837607 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b.scope: Succeeded. Nov 16 15:11:11.838189 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b.scope: Consumed 41ms CPU time Nov 16 15:11:11.842667 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:11.838232041Z" level=info msg="Stopping pod sandbox: 11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30" id=6eff3a9c-6ebd-40de-9fbb-232b88e39c10 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:11.842667 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:11.838578443Z" level=info msg="Got pod network &{Name:mydep-48669-79647ff996-g97kh Namespace:migration-upgrade-48669 ID:11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30 UID:5df14924-dd8b-4deb-a83b-53ff611d5e47 NetNS:/var/run/netns/5138656a-9b5c-411e-8cef-d25e11462c24 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:11.842667 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:11.838746645Z" level=info msg="Deleting pod migration-upgrade-48669_mydep-48669-79647ff996-g97kh from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:11.838377 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf.scope: Succeeded. Nov 16 15:11:11.838937 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf.scope: Consumed 43ms CPU time Nov 16 15:11:11.839106 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95.scope: Succeeded. Nov 16 15:11:11.839756 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95.scope: Consumed 40ms CPU time Nov 16 15:11:11.839941 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb.scope: Succeeded. Nov 16 15:11:11.840526 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb.scope: Consumed 38ms CPU time Nov 16 15:11:11.840694 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3.scope: Succeeded. Nov 16 15:11:11.841377 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3.scope: Consumed 40ms CPU time Nov 16 15:11:11.841548 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2.scope: Succeeded. Nov 16 15:11:11.842078 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2.scope: Consumed 41ms CPU time Nov 16 15:11:11.860053 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e.scope: Succeeded. Nov 16 15:11:11.860746 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e.scope: Consumed 841ms CPU time Nov 16 15:11:11.914347 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00258|connmgr|INFO|br0<->unix#587: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:11.945649 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00259|connmgr|INFO|br0<->unix#590: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:11.970552 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:11.970508 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx] Nov 16 15:11:11.974424 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00260|bridge|INFO|bridge br0: deleted interface vetha1373bc6 on port 21 Nov 16 15:11:11.981151 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vetha1373bc6 left promiscuous mode Nov 16 15:11:11.989276 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:11Z [verbose] Del: migration-upgrade-48669:mydep-48669-79647ff996-g97kh:5df14924-dd8b-4deb-a83b-53ff611d5e47:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:12.001023 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-5138656a\x2d9b5c\x2d411e\x2d8cef\x2dd25e11462c24.mount: Succeeded. Nov 16 15:11:12.002036 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-5138656a\x2d9b5c\x2d411e\x2d8cef\x2dd25e11462c24.mount: Consumed 0 CPU time Nov 16 15:11:12.023332 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-5138656a\x2d9b5c\x2d411e\x2d8cef\x2dd25e11462c24.mount: Succeeded. Nov 16 15:11:12.024306 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-5138656a\x2d9b5c\x2d411e\x2d8cef\x2dd25e11462c24.mount: Consumed 0 CPU time Nov 16 15:11:12.056732 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-5138656a\x2d9b5c\x2d411e\x2d8cef\x2dd25e11462c24.mount: Succeeded. Nov 16 15:11:12.057646 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-5138656a\x2d9b5c\x2d411e\x2d8cef\x2dd25e11462c24.mount: Consumed 0 CPU time Nov 16 15:11:12.074087 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30-userdata-shm.mount: Succeeded. Nov 16 15:11:12.075528 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:12.131004 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.130970 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b6b7d7fd9-d95wm_a776019a-1ce1-4328-aed5-a5962a4f775d/oauth-proxy/0.log" Nov 16 15:11:12.131374 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.131351 1563 generic.go:296] "Generic (PLEG): container finished" podID=a776019a-1ce1-4328-aed5-a5962a4f775d containerID="0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0" exitCode=0 Nov 16 15:11:12.131435 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.131418 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerDied Data:0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0} Nov 16 15:11:12.133988 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.133957 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a5ccca-d055-4b4c-820e-515fbf3f92d5/alertmanager-proxy/0.log" Nov 16 15:11:12.134432 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.134407 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a5ccca-d055-4b4c-820e-515fbf3f92d5/config-reloader/0.log" Nov 16 15:11:12.134857 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.134802 1563 generic.go:296] "Generic (PLEG): container finished" podID=e0a5ccca-d055-4b4c-820e-515fbf3f92d5 containerID="d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3" exitCode=0 Nov 16 15:11:12.134857 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.134824 1563 generic.go:296] "Generic (PLEG): container finished" podID=e0a5ccca-d055-4b4c-820e-515fbf3f92d5 containerID="96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972" exitCode=0 Nov 16 15:11:12.134974 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.134862 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerDied Data:d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3} Nov 16 15:11:12.134974 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.134893 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerDied Data:96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972} Nov 16 15:11:12.136055 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.135999 1563 generic.go:296] "Generic (PLEG): container finished" podID=35f6c830-e480-446f-a5cc-a6aaf007ea9a containerID="cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778" exitCode=0 Nov 16 15:11:12.136186 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.136064 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj" event=&{ID:35f6c830-e480-446f-a5cc-a6aaf007ea9a Type:ContainerDied Data:cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778} Nov 16 15:11:12.137511 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.137450 1563 generic.go:296] "Generic (PLEG): container finished" podID=220ce491-f766-41ee-ab02-aa1546eb7ed3 containerID="dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf" exitCode=0 Nov 16 15:11:12.137626 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.137524 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" event=&{ID:220ce491-f766-41ee-ab02-aa1546eb7ed3 Type:ContainerDied Data:dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf} Nov 16 15:11:12.138946 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.138925 1563 generic.go:296] "Generic (PLEG): container finished" podID=9502a499-9f8b-4a15-99d8-714a0cc37bea containerID="2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb" exitCode=0 Nov 16 15:11:12.139034 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.138986 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" event=&{ID:9502a499-9f8b-4a15-99d8-714a0cc37bea Type:ContainerDied Data:2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb} Nov 16 15:11:12.140078 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.140020 1563 generic.go:296] "Generic (PLEG): container finished" podID=b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd containerID="d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c" exitCode=0 Nov 16 15:11:12.140078 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.140081 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf" event=&{ID:b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd Type:ContainerDied Data:d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c} Nov 16 15:11:12.141753 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.141696 1563 generic.go:296] "Generic (PLEG): container finished" podID=454b3a37-199c-4abc-a79a-8de93eab2423 containerID="6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b" exitCode=0 Nov 16 15:11:12.141753 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.141721 1563 generic.go:296] "Generic (PLEG): container finished" podID=454b3a37-199c-4abc-a79a-8de93eab2423 containerID="4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4" exitCode=0 Nov 16 15:11:12.141875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.141767 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" event=&{ID:454b3a37-199c-4abc-a79a-8de93eab2423 Type:ContainerDied Data:6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b} Nov 16 15:11:12.141875 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.141798 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" event=&{ID:454b3a37-199c-4abc-a79a-8de93eab2423 Type:ContainerDied Data:4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4} Nov 16 15:11:12.143584 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.143510 1563 generic.go:296] "Generic (PLEG): container finished" podID=c8013b4f-15b6-4bac-bfc9-a45301362f0c containerID="ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778" exitCode=2 Nov 16 15:11:12.143584 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.143570 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" event=&{ID:c8013b4f-15b6-4bac-bfc9-a45301362f0c Type:ContainerDied Data:ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778} Nov 16 15:11:12.145666 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.145600 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-1_3670081b-e385-4685-8dda-b3a3ee01f233/prometheus-proxy/0.log" Nov 16 15:11:12.146418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.146393 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-1_3670081b-e385-4685-8dda-b3a3ee01f233/config-reloader/0.log" Nov 16 15:11:12.147086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.147064 1563 generic.go:296] "Generic (PLEG): container finished" podID=3670081b-e385-4685-8dda-b3a3ee01f233 containerID="db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95" exitCode=0 Nov 16 15:11:12.147086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.147088 1563 generic.go:296] "Generic (PLEG): container finished" podID=3670081b-e385-4685-8dda-b3a3ee01f233 containerID="9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2" exitCode=0 Nov 16 15:11:12.147258 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.147143 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerDied Data:db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95} Nov 16 15:11:12.147258 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.147172 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerDied Data:9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2} Nov 16 15:11:12.147343 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.147329 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" podUID=17691c72-c4d7-4e7f-b345-a0447ae7d4c6 containerName="prometheus-adapter" containerID="cri-o://9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30" gracePeriod=30 Nov 16 15:11:12.147535 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:12.147488664Z" level=info msg="Stopping container: 9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30 (timeout: 30s)" id=34ede7bb-9b27-4d59-bcb7-66df895fc143 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:12.413252 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e.scope: Succeeded. Nov 16 15:11:12.413866 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e.scope: Consumed 39ms CPU time Nov 16 15:11:12.460443 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25.scope: Succeeded. Nov 16 15:11:12.461346 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25.scope: Consumed 326ms CPU time Nov 16 15:11:12.461773 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a.scope: Succeeded. Nov 16 15:11:12.462512 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a.scope: Consumed 324ms CPU time Nov 16 15:11:12.657438 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667.scope: Succeeded. Nov 16 15:11:12.658402 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667.scope: Consumed 2.848s CPU time Nov 16 15:11:12.726675 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.726511 1563 patch_prober.go:29] interesting pod/prometheus-adapter-6fb766b4f6-jcwvx container/prometheus-adapter namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.129.2.32:6443/readyz\": dial tcp 10.129.2.32:6443: connect: connection refused" start-of-body= Nov 16 15:11:12.726675 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:12.726601 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" podUID=17691c72-c4d7-4e7f-b345-a0447ae7d4c6 containerName="prometheus-adapter" probeResult=failure output="Get \"https://10.129.2.32:6443/readyz\": dial tcp 10.129.2.32:6443: connect: connection refused" Nov 16 15:11:12.831938 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-67cd1cf31143626b3eafe3058249d6277c5c5d4cc6643aefe17f3db2216ab9f6-merged.mount: Succeeded. Nov 16 15:11:12.832907 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-67cd1cf31143626b3eafe3058249d6277c5c5d4cc6643aefe17f3db2216ab9f6-merged.mount: Consumed 0 CPU time Nov 16 15:11:12.869840 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:12.869779703Z" level=info msg="Stopped container e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8: migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx/mydep-49496" id=8e1c0e95-7eb5-4e7a-9d42-0fd12fc5fc5c name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:12.870442 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:12.870407808Z" level=info msg="Stopping pod sandbox: 18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4" id=86c38137-de95-4196-9d99-fab879cf7ea5 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:12.870741 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:12.870700011Z" level=info msg="Got pod network &{Name:mydep-49496-65f7bb956d-stlkx Namespace:migration-upgrade-49496 ID:18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4 UID:e3d0df21-083b-4e97-a997-e9a7e992eda4 NetNS:/var/run/netns/d48ac26c-599c-4021-a314-85262e441414 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:12.870917 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:12.870871312Z" level=info msg="Deleting pod migration-upgrade-49496_mydep-49496-65f7bb956d-stlkx from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:12.871237 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a.scope: Succeeded. Nov 16 15:11:12.871855 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a.scope: Consumed 40ms CPU time Nov 16 15:11:12.887110 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-eff3f763c63ecc9f8ff952599ef1db47b59d15e7f5662b181293c417655b5b2b-merged.mount: Succeeded. Nov 16 15:11:12.887321 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:12.887264452Z" level=info msg="Stopped container 7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d: migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4/mydep-49678" id=26cc8829-80ca-41e1-8b82-c080c72ea63f name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:12.889196 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:12.887839257Z" level=info msg="Stopping pod sandbox: 94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b" id=c099f0b9-70db-42bd-99fa-e648797f0250 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:12.889196 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:12.888146759Z" level=info msg="Got pod network &{Name:mydep-49678-6c64845f74-kvbp4 Namespace:migration-upgrade-49678 ID:94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b UID:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 NetNS:/var/run/netns/e11d37ec-82ef-4841-85ed-8c2da018d6b5 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:12.889196 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:12.888293560Z" level=info msg="Deleting pod migration-upgrade-49678_mydep-49678-6c64845f74-kvbp4 from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:12.888028 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-eff3f763c63ecc9f8ff952599ef1db47b59d15e7f5662b181293c417655b5b2b-merged.mount: Consumed 0 CPU time Nov 16 15:11:12.888775 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667.scope: Succeeded. Nov 16 15:11:12.889398 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667.scope: Consumed 42ms CPU time Nov 16 15:11:12.889585 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25.scope: Succeeded. Nov 16 15:11:12.890236 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25.scope: Consumed 38ms CPU time Nov 16 15:11:12.947952 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00261|connmgr|INFO|br0<->unix#593: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:12.980759 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00262|connmgr|INFO|br0<->unix#596: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:13.011588 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00263|bridge|INFO|bridge br0: deleted interface veth694c84dd on port 24 Nov 16 15:11:13.018180 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth694c84dd left promiscuous mode Nov 16 15:11:13.028026 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:12Z [verbose] Del: migration-upgrade-49496:mydep-49496-65f7bb956d-stlkx:e3d0df21-083b-4e97-a997-e9a7e992eda4:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:13.038460 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-d48ac26c\x2d599c\x2d4021\x2da314\x2d85262e441414.mount: Succeeded. Nov 16 15:11:13.039569 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-d48ac26c\x2d599c\x2d4021\x2da314\x2d85262e441414.mount: Consumed 0 CPU time Nov 16 15:11:13.049054 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00264|connmgr|INFO|br0<->unix#599: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:13.058194 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-d48ac26c\x2d599c\x2d4021\x2da314\x2d85262e441414.mount: Succeeded. Nov 16 15:11:13.059140 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-d48ac26c\x2d599c\x2d4021\x2da314\x2d85262e441414.mount: Consumed 0 CPU time Nov 16 15:11:13.084962 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00265|connmgr|INFO|br0<->unix#602: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:13.101576 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-d48ac26c\x2d599c\x2d4021\x2da314\x2d85262e441414.mount: Succeeded. Nov 16 15:11:13.102656 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-d48ac26c\x2d599c\x2d4021\x2da314\x2d85262e441414.mount: Consumed 0 CPU time Nov 16 15:11:13.116789 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00266|bridge|INFO|bridge br0: deleted interface vethbe42f6a4 on port 23 Nov 16 15:11:13.123201 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethbe42f6a4 left promiscuous mode Nov 16 15:11:13.132422 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:12Z [verbose] Del: migration-upgrade-49678:mydep-49678-6c64845f74-kvbp4:f5df478d-fc7b-4fda-b15e-1a9a05fdf611:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:13.147985 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:13.147903467Z" level=info msg="Stopped container 8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/prom-label-proxy" id=9ee7548c-d821-4bda-89bc-961a8b2eb778 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:13.152192 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:13.152162 1563 generic.go:296] "Generic (PLEG): container finished" podID=220ce491-f766-41ee-ab02-aa1546eb7ed3 containerID="881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667" exitCode=0 Nov 16 15:11:13.152326 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:13.152244 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" event=&{ID:220ce491-f766-41ee-ab02-aa1546eb7ed3 Type:ContainerDied Data:881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667} Nov 16 15:11:13.154231 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:13.154192 1563 generic.go:296] "Generic (PLEG): container finished" podID=c8013b4f-15b6-4bac-bfc9-a45301362f0c containerID="83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a" exitCode=0 Nov 16 15:11:13.154231 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:13.154235 1563 generic.go:296] "Generic (PLEG): container finished" podID=c8013b4f-15b6-4bac-bfc9-a45301362f0c containerID="89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25" exitCode=0 Nov 16 15:11:13.154354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:13.154279 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" event=&{ID:c8013b4f-15b6-4bac-bfc9-a45301362f0c Type:ContainerDied Data:83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a} Nov 16 15:11:13.154354 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:13.154310 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" event=&{ID:c8013b4f-15b6-4bac-bfc9-a45301362f0c Type:ContainerDied Data:89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25} Nov 16 15:11:13.156262 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:13.156239 1563 generic.go:296] "Generic (PLEG): container finished" podID=add06140-663d-48c0-ac00-643bd58a3f7d containerID="c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e" exitCode=0 Nov 16 15:11:13.156408 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:13.156278 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" event=&{ID:add06140-663d-48c0-ac00-643bd58a3f7d Type:ContainerDied Data:c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e} Nov 16 15:11:13.259379 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30.scope: Succeeded. Nov 16 15:11:13.260070 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30.scope: Consumed 40ms CPU time Nov 16 15:11:13.262795 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30.scope: Succeeded. Nov 16 15:11:13.263423 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30.scope: Consumed 10.694s CPU time Nov 16 15:11:13.414712 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:13.414586033Z" level=info msg="Stopped container ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454: openshift-marketplace/qe-app-registry-9hthz/registry-server" id=8f46451d-0151-4197-addd-45b672872d5b name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:13.415379 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:13.415347040Z" level=info msg="Stopping pod sandbox: 9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9" id=b2f6af93-0a75-475b-a081-cabc28f85153 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:13.415659 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:13.415627242Z" level=info msg="Got pod network &{Name:qe-app-registry-9hthz Namespace:openshift-marketplace ID:9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9 UID:5eb5be5f-46bd-425a-a486-e2641bb0ad88 NetNS:/var/run/netns/29c01d2e-8012-44ee-9594-9dcbd2a39acd Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:13.415853 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:13.415828944Z" level=info msg="Deleting pod openshift-marketplace_qe-app-registry-9hthz from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:13.488888 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00267|connmgr|INFO|br0<->unix#605: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:13.539061 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:13.538884090Z" level=info msg="Stopped container 526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4: openshift-monitoring/prometheus-k8s-1/prometheus-proxy" id=e6df18d0-2296-4912-baf8-cdea244e869c name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:13.562222 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00268|connmgr|INFO|br0<->unix#608: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:13.596488 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00269|bridge|INFO|bridge br0: deleted interface veth09f52757 on port 37 Nov 16 15:11:13.603201 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth09f52757 left promiscuous mode Nov 16 15:11:13.612716 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:13Z [verbose] Del: openshift-marketplace:qe-app-registry-9hthz:5eb5be5f-46bd-425a-a486-e2641bb0ad88:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:13.653864 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:13.653746566Z" level=info msg="Stopped container 8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4: openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv/manager" id=2e33a9b8-5dce-4b7d-83eb-5a4af9a73a3a name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:13.787345 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:13.787279901Z" level=info msg="Stopped container f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy-rules" id=4c6ba989-ebd1-40f1-b475-25249f571490 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:13.832213 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:13.832076082Z" level=info msg="Stopped container 7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858: migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg/mydep-50425" id=773bd3f3-f401-4265-8f0c-560d4b35909b name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:13.832695 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-63a77d9faa41a804da850b370693f899d88d589e9ddedcb1c78b48f09ce8ade1-merged.mount: Succeeded. Nov 16 15:11:13.833058 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:13.832685487Z" level=info msg="Stopping pod sandbox: 5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78" id=5c0fff1b-ece6-424b-8539-ae56c1708535 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:13.833058 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:13.832956089Z" level=info msg="Got pod network &{Name:mydep-50425-85f87ccbc6-wgxqg Namespace:migration-upgrade-50425 ID:5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78 UID:4e58fe69-e225-425e-a6ca-0a7f136dbdca NetNS:/var/run/netns/f236704e-396e-4bc2-b833-20144735ffde Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:13.833236 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:13.833144991Z" level=info msg="Deleting pod migration-upgrade-50425_mydep-50425-85f87ccbc6-wgxqg from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:13.833701 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-63a77d9faa41a804da850b370693f899d88d589e9ddedcb1c78b48f09ce8ade1-merged.mount: Consumed 0 CPU time Nov 16 15:11:13.833771 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0c193514b093b9c45f14a4524860e5df667f835b959afbf61e3f7e95ea5ba675-merged.mount: Succeeded. Nov 16 15:11:13.834509 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0c193514b093b9c45f14a4524860e5df667f835b959afbf61e3f7e95ea5ba675-merged.mount: Consumed 0 CPU time Nov 16 15:11:13.834528 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-4c5e761e6d08f07c905511ff94a286ed5dca35b552c048994f86ecda2ddf1bd1-merged.mount: Succeeded. Nov 16 15:11:13.835290 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-4c5e761e6d08f07c905511ff94a286ed5dca35b552c048994f86ecda2ddf1bd1-merged.mount: Consumed 0 CPU time Nov 16 15:11:13.835317 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-29c01d2e\x2d8012\x2d44ee\x2d9594\x2d9dcbd2a39acd.mount: Succeeded. Nov 16 15:11:13.836029 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-29c01d2e\x2d8012\x2d44ee\x2d9594\x2d9dcbd2a39acd.mount: Consumed 0 CPU time Nov 16 15:11:13.836045 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-29c01d2e\x2d8012\x2d44ee\x2d9594\x2d9dcbd2a39acd.mount: Succeeded. Nov 16 15:11:13.836737 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-29c01d2e\x2d8012\x2d44ee\x2d9594\x2d9dcbd2a39acd.mount: Consumed 0 CPU time Nov 16 15:11:13.836751 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-29c01d2e\x2d8012\x2d44ee\x2d9594\x2d9dcbd2a39acd.mount: Succeeded. Nov 16 15:11:13.837467 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-29c01d2e\x2d8012\x2d44ee\x2d9594\x2d9dcbd2a39acd.mount: Consumed 0 CPU time Nov 16 15:11:13.837482 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9-userdata-shm.mount: Succeeded. Nov 16 15:11:13.838155 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:13.838198 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4-userdata-shm.mount: Succeeded. Nov 16 15:11:13.838864 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:13.838880 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-e11d37ec\x2d82ef\x2d4841\x2d85ed\x2d8c2da018d6b5.mount: Succeeded. Nov 16 15:11:13.839560 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-e11d37ec\x2d82ef\x2d4841\x2d85ed\x2d8c2da018d6b5.mount: Consumed 0 CPU time Nov 16 15:11:13.839575 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-e11d37ec\x2d82ef\x2d4841\x2d85ed\x2d8c2da018d6b5.mount: Succeeded. Nov 16 15:11:13.840379 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-e11d37ec\x2d82ef\x2d4841\x2d85ed\x2d8c2da018d6b5.mount: Consumed 0 CPU time Nov 16 15:11:13.840397 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-e11d37ec\x2d82ef\x2d4841\x2d85ed\x2d8c2da018d6b5.mount: Succeeded. Nov 16 15:11:13.841082 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-e11d37ec\x2d82ef\x2d4841\x2d85ed\x2d8c2da018d6b5.mount: Consumed 0 CPU time Nov 16 15:11:13.841096 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b-userdata-shm.mount: Succeeded. Nov 16 15:11:13.841811 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:13.841828 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0ba1d92f410c195a1877dd6ac3ca5a88abc870cc79b03398a60761c4069f260c-merged.mount: Succeeded. Nov 16 15:11:13.842508 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0ba1d92f410c195a1877dd6ac3ca5a88abc870cc79b03398a60761c4069f260c-merged.mount: Consumed 0 CPU time Nov 16 15:11:13.842535 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-29093b497093ab51ad1fc4cf6620c6e214fa817f2f8aeb7ee3d20906b61ab63e-merged.mount: Succeeded. Nov 16 15:11:13.843245 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-29093b497093ab51ad1fc4cf6620c6e214fa817f2f8aeb7ee3d20906b61ab63e-merged.mount: Consumed 0 CPU time Nov 16 15:11:13.854553 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-36de8d21b6129135613f9f913c526269255f4afcfecb488d5bd70d427ac8d604-merged.mount: Succeeded. Nov 16 15:11:13.855501 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-36de8d21b6129135613f9f913c526269255f4afcfecb488d5bd70d427ac8d604-merged.mount: Consumed 0 CPU time Nov 16 15:11:13.906694 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00270|connmgr|INFO|br0<->unix#611: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:14.000018 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00271|connmgr|INFO|br0<->unix#614: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:14.031922 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00272|bridge|INFO|bridge br0: deleted interface vethf2aca57a on port 22 Nov 16 15:11:14.042325 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethf2aca57a left promiscuous mode Nov 16 15:11:14.049728 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:13Z [verbose] Del: migration-upgrade-50425:mydep-50425-85f87ccbc6-wgxqg:4e58fe69-e225-425e-a6ca-0a7f136dbdca:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:14.062074 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-f236704e\x2d396e\x2d4bc2\x2db833\x2d20144735ffde.mount: Succeeded. Nov 16 15:11:14.062963 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-f236704e\x2d396e\x2d4bc2\x2db833\x2d20144735ffde.mount: Consumed 0 CPU time Nov 16 15:11:14.076533 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-f236704e\x2d396e\x2d4bc2\x2db833\x2d20144735ffde.mount: Succeeded. Nov 16 15:11:14.077427 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-f236704e\x2d396e\x2d4bc2\x2db833\x2d20144735ffde.mount: Consumed 0 CPU time Nov 16 15:11:14.087580 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-c921c21e3588dc27f80162098f569d7a617eb692f979853d7979535564778d48-merged.mount: Succeeded. Nov 16 15:11:14.088494 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-c921c21e3588dc27f80162098f569d7a617eb692f979853d7979535564778d48-merged.mount: Consumed 0 CPU time Nov 16 15:11:14.104258 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:14.103715790Z" level=info msg="Stopped container a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/oauth-proxy" id=9599626e-a3de-47e6-99ce-170bc272eb7d name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:14.169815 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:14.169758 1563 generic.go:296] "Generic (PLEG): container finished" podID=17691c72-c4d7-4e7f-b345-a0447ae7d4c6 containerID="9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30" exitCode=0 Nov 16 15:11:14.169815 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:14.169833 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" event=&{ID:17691c72-c4d7-4e7f-b345-a0447ae7d4c6 Type:ContainerDied Data:9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30} Nov 16 15:11:14.273606 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:14.273512733Z" level=info msg="Stopped container d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-state-metrics" id=01c05030-f52a-4b04-ae43-7eb8fb9a5487 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:14.332610 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:14.332512335Z" level=info msg="Stopped container 94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/thanos-query" id=6097dcae-59c7-4742-bc58-b26564928284 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:14.431571 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:14.431349075Z" level=info msg="Stopped container e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a: openshift-monitoring/prometheus-k8s-1/config-reloader" id=896640ed-852c-4d58-878c-0f3dab152fa5 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:14.491663 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:14.491574487Z" level=info msg="Stopped container fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy" id=3767722d-fe8f-4c5a-b14b-d41e7980928b name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:14.510741 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:14.510686 1563 patch_prober.go:29] interesting pod/thanos-querier-5b6b7d7fd9-d95wm container/oauth-proxy namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.129.2.33:9091/-/ready\": dial tcp 10.129.2.33:9091: connect: connection refused" start-of-body= Nov 16 15:11:14.510931 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:14.510759 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" podUID=a776019a-1ce1-4328-aed5-a5962a4f775d containerName="oauth-proxy" probeResult=failure output="Get \"https://10.129.2.33:9091/-/ready\": dial tcp 10.129.2.33:9091: connect: connection refused" Nov 16 15:11:14.581340 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:14.581217649Z" level=info msg="Stopped container f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09: openshift-monitoring/prometheus-k8s-1/thanos-sidecar" id=77d9c072-1221-4f11-971d-6d4ef13804a9 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:14.632627 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:14.632521785Z" level=info msg="Stopped container 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae: openshift-monitoring/prometheus-k8s-1/prometheus" id=7557d587-deb2-49b5-8742-1bf5b6102464 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:14.750291 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:14.750050184Z" level=info msg="Stopped container c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651: openshift-monitoring/alertmanager-main-0/alertmanager-proxy" id=1818c301-c5fa-474f-8dbb-e8db85a675d7 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:14.788389 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:14.788335109Z" level=info msg="Stopped container 27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518: openshift-monitoring/alertmanager-main-0/config-reloader" id=a5cbe5a1-c52d-47ff-9496-af7d1dba7288 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:14.831159 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6dc2dcc3eda878b9018f81cc68c0807d4b0ad5f5569509c705012e4cf342a8da-merged.mount: Succeeded. Nov 16 15:11:14.832351 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6dc2dcc3eda878b9018f81cc68c0807d4b0ad5f5569509c705012e4cf342a8da-merged.mount: Consumed 0 CPU time Nov 16 15:11:14.832374 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-d589ca5bd73fcb376473edf52f9887e36ac65160b7e638ffe5d5d03591ab321e-merged.mount: Succeeded. Nov 16 15:11:14.833470 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-d589ca5bd73fcb376473edf52f9887e36ac65160b7e638ffe5d5d03591ab321e-merged.mount: Consumed 0 CPU time Nov 16 15:11:14.833490 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-93a28c5c43ba0161228eb1deb937cb1d7d784b4ec2503e9fc27eb27bd24e4a8d-merged.mount: Succeeded. Nov 16 15:11:14.834287 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-93a28c5c43ba0161228eb1deb937cb1d7d784b4ec2503e9fc27eb27bd24e4a8d-merged.mount: Consumed 0 CPU time Nov 16 15:11:14.834349 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-c4e78179722aecc68f34fe922b56a3ed8c84dcc738ed61a27d8d4b32555a98b0-merged.mount: Succeeded. Nov 16 15:11:14.835243 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-c4e78179722aecc68f34fe922b56a3ed8c84dcc738ed61a27d8d4b32555a98b0-merged.mount: Consumed 0 CPU time Nov 16 15:11:14.835285 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-9860cdb7e49828fec78f539fdc6260badaa50a5b8a6a9235bd0a1b291f6138b1-merged.mount: Succeeded. Nov 16 15:11:14.836040 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-9860cdb7e49828fec78f539fdc6260badaa50a5b8a6a9235bd0a1b291f6138b1-merged.mount: Consumed 0 CPU time Nov 16 15:11:14.836062 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6706773c7ae4852aa18cf6f3dba1b03bb3b0724f6e9c9efd673eac8fe4b34c0c-merged.mount: Succeeded. Nov 16 15:11:14.836900 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6706773c7ae4852aa18cf6f3dba1b03bb3b0724f6e9c9efd673eac8fe4b34c0c-merged.mount: Consumed 0 CPU time Nov 16 15:11:14.836928 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6e6c82cf59e3ead089acbcc5152c4c515853d2ed6b116b1a9a1518b932185c94-merged.mount: Succeeded. Nov 16 15:11:14.837746 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6e6c82cf59e3ead089acbcc5152c4c515853d2ed6b116b1a9a1518b932185c94-merged.mount: Consumed 0 CPU time Nov 16 15:11:14.837764 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-772e9ccbe1324dd4be0540f4c2bec6fd9d75457a6f3a1051d530c8a0c97e81d0-merged.mount: Succeeded. Nov 16 15:11:14.838521 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-772e9ccbe1324dd4be0540f4c2bec6fd9d75457a6f3a1051d530c8a0c97e81d0-merged.mount: Consumed 0 CPU time Nov 16 15:11:14.838562 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-f236704e\x2d396e\x2d4bc2\x2db833\x2d20144735ffde.mount: Succeeded. Nov 16 15:11:14.839346 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-f236704e\x2d396e\x2d4bc2\x2db833\x2d20144735ffde.mount: Consumed 0 CPU time Nov 16 15:11:14.839364 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78-userdata-shm.mount: Succeeded. Nov 16 15:11:14.840048 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:14.854515 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:14.854402570Z" level=info msg="Stopped container 3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d: openshift-monitoring/telemeter-client-55c5556665-7vs77/telemeter-client" id=9cfd00b5-109a-4dc7-9901-991b7dc5da95 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:14.856293 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-97df0909026e3e64145ba7e020bb473db19414e1a050ab6be15175eaf8a60518-merged.mount: Succeeded. Nov 16 15:11:14.857059 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-97df0909026e3e64145ba7e020bb473db19414e1a050ab6be15175eaf8a60518-merged.mount: Consumed 0 CPU time Nov 16 15:11:15.092779 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.092725496Z" level=info msg="Stopped container d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487: openshift-monitoring/telemeter-client-55c5556665-7vs77/reload" id=81d4fbd8-0f68-41dc-8ed2-3a00d4d452e1 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:15.093895 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6c5d5bf7e864f520b9085ab279a6b2193944f99f6dfa6f523ae43dca65367614-merged.mount: Succeeded. Nov 16 15:11:15.094823 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6c5d5bf7e864f520b9085ab279a6b2193944f99f6dfa6f523ae43dca65367614-merged.mount: Consumed 0 CPU time Nov 16 15:11:15.183030 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-9b9bff67e4510ad4672776f84475070ca8b7e1d29dfa9bab8175692ebf5b23f8-merged.mount: Succeeded. Nov 16 15:11:15.184253 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-9b9bff67e4510ad4672776f84475070ca8b7e1d29dfa9bab8175692ebf5b23f8-merged.mount: Consumed 0 CPU time Nov 16 15:11:15.193250 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.193203250Z" level=info msg="Stopped container eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5: openshift-monitoring/alertmanager-main-0/prom-label-proxy" id=e6d52273-dc20-46cc-ae9d-cef02805e30f name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:15.287551 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.287450851Z" level=info msg="Stopped container aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy" id=0ddb3a1b-d6ce-49eb-b829-4aa611b8a936 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:15.288002 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-e424ab1c46c8248f34e50f1d42f56bf0ce5380db70aad26c28db42fbbc4e18ef-merged.mount: Succeeded. Nov 16 15:11:15.288992 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-e424ab1c46c8248f34e50f1d42f56bf0ce5380db70aad26c28db42fbbc4e18ef-merged.mount: Consumed 0 CPU time Nov 16 15:11:15.389385 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.389233416Z" level=info msg="Stopped container ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/openshift-state-metrics" id=ce96b784-0075-40c0-9f9a-cd3c1a17a266 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:15.495085 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.494944914Z" level=info msg="Stopped container d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c: openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf/cluster-logging-operator" id=4b406790-cf1f-4934-add8-34eb3a2d136f name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:15.495719 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.495681721Z" level=info msg="Stopping pod sandbox: 6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d" id=22a621e5-92f8-406d-8d67-01d1c19703a4 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:15.496016 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.495958123Z" level=info msg="Got pod network &{Name:cluster-logging-operator-7f6c6bb67c-hmvsf Namespace:openshift-logging ID:6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d UID:b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd NetNS:/var/run/netns/98721d7b-dd12-41b3-993b-3bfd49aba10f Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:15.496199 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.496171225Z" level=info msg="Deleting pod openshift-logging_cluster-logging-operator-7f6c6bb67c-hmvsf from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:15.538188 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.538084581Z" level=info msg="Stopped container 2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb: openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv/kube-rbac-proxy" id=01b98aba-5a81-4687-b304-ec9c8238d15c name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:15.538807 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.538771987Z" level=info msg="Stopping pod sandbox: 0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7" id=a85b5b2a-a4dd-40a7-b03b-21ca7045c38e name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:15.539117 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.539086490Z" level=info msg="Got pod network &{Name:loki-operator-controller-manager-67cc5866c9-m67tv Namespace:openshift-operators-redhat ID:0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7 UID:9502a499-9f8b-4a15-99d8-714a0cc37bea NetNS:/var/run/netns/4d6a1d55-cb65-4ac7-b4f6-0ccbae6022e5 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:15.539276 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.539248791Z" level=info msg="Deleting pod openshift-operators-redhat_loki-operator-controller-manager-67cc5866c9-m67tv from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:15.570121 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00273|connmgr|INFO|br0<->unix#617: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:15.602417 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00274|connmgr|INFO|br0<->unix#620: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:15.631667 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00275|bridge|INFO|bridge br0: deleted interface veth83bb3ade on port 11 Nov 16 15:11:15.634616 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.634519201Z" level=info msg="Stopped container 6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-rbac-proxy-self" id=373b7569-7efc-4051-847e-07a964711226 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:15.638201 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth83bb3ade left promiscuous mode Nov 16 15:11:15.647367 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:15Z [verbose] Del: openshift-logging:cluster-logging-operator-7f6c6bb67c-hmvsf:b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:15.668385 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00276|connmgr|INFO|br0<->unix#623: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:15.705327 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00277|connmgr|INFO|br0<->unix#626: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:15.711877 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:15.711832 1563 remote_runtime.go:704] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae is running failed: container process not found" containerID="6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae" cmd=[sh -c if [ -x "$(command -v curl)" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x "$(command -v wget)" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi] Nov 16 15:11:15.712533 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:15.712434 1563 remote_runtime.go:704] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae is running failed: container process not found" containerID="6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae" cmd=[sh -c if [ -x "$(command -v curl)" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x "$(command -v wget)" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi] Nov 16 15:11:15.712785 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:15.712757 1563 remote_runtime.go:704] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae is running failed: container process not found" containerID="6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae" cmd=[sh -c if [ -x "$(command -v curl)" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x "$(command -v wget)" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi] Nov 16 15:11:15.712864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:15.712811 1563 prober.go:118] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae is running failed: container process not found" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-1" podUID=3670081b-e385-4685-8dda-b3a3ee01f233 containerName="prometheus" Nov 16 15:11:15.729916 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.729868711Z" level=info msg="Stopped container 4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-rbac-proxy-main" id=eb0f9944-bf25-4bf3-abb7-c88797c494b8 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:15.730387 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.730314415Z" level=info msg="Stopping pod sandbox: 5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3" id=62796e2e-258a-4e35-811b-addc53d3cbd7 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:15.730658 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.730624018Z" level=info msg="Got pod network &{Name:kube-state-metrics-68667ff848-f92nw Namespace:openshift-monitoring ID:5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3 UID:454b3a37-199c-4abc-a79a-8de93eab2423 NetNS:/var/run/netns/e936529a-7e5f-4ca3-b47c-4441e6fa4a10 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:15.730819 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.730792219Z" level=info msg="Deleting pod openshift-monitoring_kube-state-metrics-68667ff848-f92nw from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:15.735238 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00278|bridge|INFO|bridge br0: deleted interface veth5cab1de5 on port 12 Nov 16 15:11:15.819164 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth5cab1de5 left promiscuous mode Nov 16 15:11:15.829664 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:15Z [verbose] Del: openshift-operators-redhat:loki-operator-controller-manager-67cc5866c9-m67tv:9502a499-9f8b-4a15-99d8-714a0cc37bea:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:15.831340 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-8fd2c62d3c8c9f67dd3263cf6a6b5c2cf1deb00df71db2c414656a0bfad8e9ee-merged.mount: Succeeded. Nov 16 15:11:15.832285 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-8fd2c62d3c8c9f67dd3263cf6a6b5c2cf1deb00df71db2c414656a0bfad8e9ee-merged.mount: Consumed 0 CPU time Nov 16 15:11:15.832305 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-f234c32e2dee67395c720dea288e9c0a4f64aa70944b8944aecb7ed30a38d64a-merged.mount: Succeeded. Nov 16 15:11:15.833078 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-f234c32e2dee67395c720dea288e9c0a4f64aa70944b8944aecb7ed30a38d64a-merged.mount: Consumed 0 CPU time Nov 16 15:11:15.833119 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-9421602f00a85b3f4210aeacad1490d01d73b28c358b10c0b13adbe01b611675-merged.mount: Succeeded. Nov 16 15:11:15.833975 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-9421602f00a85b3f4210aeacad1490d01d73b28c358b10c0b13adbe01b611675-merged.mount: Consumed 0 CPU time Nov 16 15:11:15.833995 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-28a0ff8488ede62d9566da7b3a9d906a67404981a9d5333882e5ebdaf1d5e8f0-merged.mount: Succeeded. Nov 16 15:11:15.834699 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-28a0ff8488ede62d9566da7b3a9d906a67404981a9d5333882e5ebdaf1d5e8f0-merged.mount: Consumed 0 CPU time Nov 16 15:11:15.834716 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-24ab82d0a13cda3b194c1627116f49b856204110e0942658bdc145a94fdf4ec7-merged.mount: Succeeded. Nov 16 15:11:15.835431 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-24ab82d0a13cda3b194c1627116f49b856204110e0942658bdc145a94fdf4ec7-merged.mount: Consumed 0 CPU time Nov 16 15:11:15.835453 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-98721d7b\x2ddd12\x2d41b3\x2d993b\x2d3bfd49aba10f.mount: Succeeded. Nov 16 15:11:15.836121 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-98721d7b\x2ddd12\x2d41b3\x2d993b\x2d3bfd49aba10f.mount: Consumed 0 CPU time Nov 16 15:11:15.836173 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-98721d7b\x2ddd12\x2d41b3\x2d993b\x2d3bfd49aba10f.mount: Succeeded. Nov 16 15:11:15.836840 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-98721d7b\x2ddd12\x2d41b3\x2d993b\x2d3bfd49aba10f.mount: Consumed 0 CPU time Nov 16 15:11:15.836855 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-98721d7b\x2ddd12\x2d41b3\x2d993b\x2d3bfd49aba10f.mount: Succeeded. Nov 16 15:11:15.837564 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-98721d7b\x2ddd12\x2d41b3\x2d993b\x2d3bfd49aba10f.mount: Consumed 0 CPU time Nov 16 15:11:15.837580 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d-userdata-shm.mount: Succeeded. Nov 16 15:11:15.838279 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:15.847613 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-4d6a1d55\x2dcb65\x2d4ac7\x2db4f6\x2d0ccbae6022e5.mount: Succeeded. Nov 16 15:11:15.849059 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-4d6a1d55\x2dcb65\x2d4ac7\x2db4f6\x2d0ccbae6022e5.mount: Consumed 0 CPU time Nov 16 15:11:15.850716 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00279|connmgr|INFO|br0<->unix#629: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:15.876226 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:15.876111554Z" level=info msg="Stopped pod sandbox: 11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30" id=6eff3a9c-6ebd-40de-9fbb-232b88e39c10 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:15.882080 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00280|connmgr|INFO|br0<->unix#632: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:15.885563 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-c6379871b11256a3b0d4f387a3632b1d54080da1c2269327a163a65caa7242d4-merged.mount: Succeeded. Nov 16 15:11:15.886428 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-c6379871b11256a3b0d4f387a3632b1d54080da1c2269327a163a65caa7242d4-merged.mount: Consumed 0 CPU time Nov 16 15:11:15.895543 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-4d6a1d55\x2dcb65\x2d4ac7\x2db4f6\x2d0ccbae6022e5.mount: Succeeded. Nov 16 15:11:15.896344 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-4d6a1d55\x2dcb65\x2d4ac7\x2db4f6\x2d0ccbae6022e5.mount: Consumed 0 CPU time Nov 16 15:11:15.910955 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00281|bridge|INFO|bridge br0: deleted interface vethb0280b20 on port 31 Nov 16 15:11:15.917219 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethb0280b20 left promiscuous mode Nov 16 15:11:15.929828 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:15Z [verbose] Del: openshift-monitoring:kube-state-metrics-68667ff848-f92nw:454b3a37-199c-4abc-a79a-8de93eab2423:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:15.934105 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-4d6a1d55\x2dcb65\x2d4ac7\x2db4f6\x2d0ccbae6022e5.mount: Succeeded. Nov 16 15:11:15.935136 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-4d6a1d55\x2dcb65\x2d4ac7\x2db4f6\x2d0ccbae6022e5.mount: Consumed 0 CPU time Nov 16 15:11:16.074691 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.074596 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95fd9\" (UniqueName: \"kubernetes.io/projected/5df14924-dd8b-4deb-a83b-53ff611d5e47-kube-api-access-95fd9\") pod \"5df14924-dd8b-4deb-a83b-53ff611d5e47\" (UID: \"5df14924-dd8b-4deb-a83b-53ff611d5e47\") " Nov 16 15:11:16.074882 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.074726 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\") pod \"5df14924-dd8b-4deb-a83b-53ff611d5e47\" (UID: \"5df14924-dd8b-4deb-a83b-53ff611d5e47\") " Nov 16 15:11:16.095920 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:11:16.096470 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.096433 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5df14924-dd8b-4deb-a83b-53ff611d5e47-kube-api-access-95fd9" (OuterVolumeSpecName: "kube-api-access-95fd9") pod "5df14924-dd8b-4deb-a83b-53ff611d5e47" (UID: "5df14924-dd8b-4deb-a83b-53ff611d5e47"). InnerVolumeSpecName "kube-api-access-95fd9". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:16.096846 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.096814 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:11:16.101370 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r5a14ea3416ba49e8ba39852f7fe1139b.scope: Succeeded. Nov 16 15:11:16.101879 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r5a14ea3416ba49e8ba39852f7fe1139b.scope: Consumed 805us CPU time Nov 16 15:11:16.117341 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.117273 1563 operation_generator.go:1363] UnmapVolume succeeded for volume "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e" (OuterVolumeSpecName: "local") pod "5df14924-dd8b-4deb-a83b-53ff611d5e47" (UID: "5df14924-dd8b-4deb-a83b-53ff611d5e47"). InnerVolumeSpecName "pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e". PluginName "kubernetes.io/azure-disk", VolumeGidValue "" Nov 16 15:11:16.127858 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.127769393Z" level=info msg="Stopped container 0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy-metrics" id=450decc4-623a-4ece-b6e1-6240f917c44e name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:16.128454 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.128417398Z" level=info msg="Stopping pod sandbox: 4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c" id=b1dc4077-d082-4811-87f3-5b67036600b3 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:16.128786 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.128720901Z" level=info msg="Got pod network &{Name:thanos-querier-5b6b7d7fd9-d95wm Namespace:openshift-monitoring ID:4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c UID:a776019a-1ce1-4328-aed5-a5962a4f775d NetNS:/var/run/netns/7a914dca-3986-42a0-ab4c-ab19848f2047 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:16.128924 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.128900502Z" level=info msg="Deleting pod openshift-monitoring_thanos-querier-5b6b7d7fd9-d95wm from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:16.176758 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.176482 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-95fd9\" (UniqueName: \"kubernetes.io/projected/5df14924-dd8b-4deb-a83b-53ff611d5e47-kube-api-access-95fd9\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:16.176758 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.176675 1563 reconciler.go:293] "operationExecutor.UnmountDevice started for volume \"pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" " Nov 16 15:11:16.177010 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.176884 1563 operation_generator.go:1492] UnmapDevice succeeded for volume "pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e") on node "ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 15:11:16.181936 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.181908 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-48669/mydep-48669-79647ff996-g97kh" event=&{ID:5df14924-dd8b-4deb-a83b-53ff611d5e47 Type:ContainerDied Data:11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30} Nov 16 15:11:16.182067 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.181960 1563 scope.go:110] "RemoveContainer" containerID="2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a" Nov 16 15:11:16.183047 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.182968362Z" level=info msg="Removing container: 2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a" id=16bb5147-1938-4a75-91ea-47665d74ac8e name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:16.189943 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod5df14924_dd8b_4deb_a83b_53ff611d5e47.slice. Nov 16 15:11:16.190005 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-besteffort-pod5df14924_dd8b_4deb_a83b_53ff611d5e47.slice: Consumed 93ms CPU time Nov 16 15:11:16.222102 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00282|connmgr|INFO|br0<->unix#635: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:16.236989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.236959 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[migration-upgrade-48669/mydep-48669-79647ff996-g97kh] Nov 16 15:11:16.250357 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.250294 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[migration-upgrade-48669/mydep-48669-79647ff996-g97kh] Nov 16 15:11:16.253967 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00283|connmgr|INFO|br0<->unix#638: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:16.277806 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.277695 1563 reconciler.go:300] "Volume detached for volume \"pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-eb8a8acd-a586-4d92-bb1b-190c4432313e\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"/dev/sdc\"" Nov 16 15:11:16.284446 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00284|bridge|INFO|bridge br0: deleted interface veth7b8f5bbe on port 34 Nov 16 15:11:16.291191 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth7b8f5bbe left promiscuous mode Nov 16 15:11:16.299736 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:16Z [verbose] Del: openshift-monitoring:thanos-querier-5b6b7d7fd9-d95wm:a776019a-1ce1-4328-aed5-a5962a4f775d:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:16.348009 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.347877 1563 patch_prober.go:29] interesting pod/router-default-cff655768-nrbzb container/router namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[+]backend-http ok Nov 16 15:11:16.348009 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: [+]has-synced ok Nov 16 15:11:16.348009 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: [-]process-running failed: reason withheld Nov 16 15:11:16.348009 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: healthz check failed Nov 16 15:11:16.348009 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.348016 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-ingress/router-default-cff655768-nrbzb" podUID=03363f87-776c-47c3-9bd8-8c0de24a4de5 containerName="router" probeResult=failure output="HTTP probe failed with statuscode: 500" Nov 16 15:11:16.348352 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.348300767Z" level=info msg="Stopped container db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95: openshift-monitoring/prometheus-k8s-1/kube-rbac-proxy-thanos" id=24e88f0a-354e-44f0-b118-c495b37c7816 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:16.420412 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.420373 1563 kubelet_node_status.go:590] "Recording event message for node" node="ropatil1611az-gs6zj-worker-westus-j4ngg" event="NodeNotSchedulable" Nov 16 15:11:16.578682 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.578475323Z" level=info msg="Stopped container 9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2: openshift-monitoring/prometheus-k8s-1/kube-rbac-proxy" id=856688bc-6cee-4e17-9d8a-a0b536059296 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:16.579102 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.579059928Z" level=info msg="Stopping pod sandbox: 0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a" id=226e3ae3-b5f4-45e7-85c0-0df245c5dfe9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:16.579416 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.579365431Z" level=info msg="Got pod network &{Name:prometheus-k8s-1 Namespace:openshift-monitoring ID:0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a UID:3670081b-e385-4685-8dda-b3a3ee01f233 NetNS:/var/run/netns/8defb3e9-54f6-49a9-a23b-266a54369420 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:16.579558 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.579526832Z" level=info msg="Deleting pod openshift-monitoring_prometheus-k8s-1 from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:16.593442 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.593399850Z" level=info msg="Stopping pod sandbox: 11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30" id=62334f13-5db6-4dc9-a652-24b68bda4d31 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:16.593442 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.593444651Z" level=info msg="Stopped pod sandbox (already stopped): 11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30" id=62334f13-5db6-4dc9-a652-24b68bda4d31 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:16.595123 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.595087 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=5df14924-dd8b-4deb-a83b-53ff611d5e47 path="/var/lib/kubelet/pods/5df14924-dd8b-4deb-a83b-53ff611d5e47/volumes" Nov 16 15:11:16.645613 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00285|connmgr|INFO|br0<->unix#641: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:16.672253 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.672181120Z" level=info msg="Stopped container 96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972: openshift-monitoring/alertmanager-main-0/alertmanager" id=0f8c39cc-15fc-4d76-8995-51000bf70d73 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:16.731793 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.731731326Z" level=info msg="Stopped container dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf: openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt/elasticsearch-operator" id=90941735-5f36-4363-9de4-13b860e2e32b name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:16.744666 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00286|connmgr|INFO|br0<->unix#644: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:16.776700 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00287|bridge|INFO|bridge br0: deleted interface vethb55d967f on port 43 Nov 16 15:11:16.783153 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethb55d967f left promiscuous mode Nov 16 15:11:16.791834 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:16Z [verbose] Del: openshift-monitoring:prometheus-k8s-1:3670081b-e385-4685-8dda-b3a3ee01f233:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:16.828411 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.828348147Z" level=info msg="Stopped container cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778: openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj/prometheus-operator-admission-webhook" id=f8f4987a-8c69-4ebb-9d80-d7112f7a2304 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:16.829037 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.828898052Z" level=info msg="Stopping pod sandbox: c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844" id=e1321e69-8448-4998-b7e6-53c150b3bd1f name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:16.829230 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.829194654Z" level=info msg="Got pod network &{Name:prometheus-operator-admission-webhook-6c69dfd44-hxqbj Namespace:openshift-monitoring ID:c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844 UID:35f6c830-e480-446f-a5cc-a6aaf007ea9a NetNS:/var/run/netns/b34f6c7c-5574-4659-bd55-848e4e74be11 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:16.829371 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:16.829342056Z" level=info msg="Deleting pod openshift-monitoring_prometheus-operator-admission-webhook-6c69dfd44-hxqbj from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:16.831001 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-de4853d3773bd067f79f7932c1b01506960d4b32b874fad4babf316a11fbb145-merged.mount: Succeeded. Nov 16 15:11:16.831977 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-de4853d3773bd067f79f7932c1b01506960d4b32b874fad4babf316a11fbb145-merged.mount: Consumed 0 CPU time Nov 16 15:11:16.832001 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-129af2e4c1bba52d64bc3c8311d98ad19eb0fc7c8a2c4446e9eb96328f22c244-merged.mount: Succeeded. Nov 16 15:11:16.833775 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-129af2e4c1bba52d64bc3c8311d98ad19eb0fc7c8a2c4446e9eb96328f22c244-merged.mount: Consumed 0 CPU time Nov 16 15:11:16.833799 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-8defb3e9\x2d54f6\x2d49a9\x2da23b\x2d266a54369420.mount: Succeeded. Nov 16 15:11:16.834499 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-8defb3e9\x2d54f6\x2d49a9\x2da23b\x2d266a54369420.mount: Consumed 0 CPU time Nov 16 15:11:16.834668 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-8defb3e9\x2d54f6\x2d49a9\x2da23b\x2d266a54369420.mount: Succeeded. Nov 16 15:11:16.835498 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-8defb3e9\x2d54f6\x2d49a9\x2da23b\x2d266a54369420.mount: Consumed 0 CPU time Nov 16 15:11:16.835557 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-e0847a6425fdb8de314fec9141f5d9014eacd1487103c85261cc4ea31acc4d3a-merged.mount: Succeeded. Nov 16 15:11:16.836255 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-e0847a6425fdb8de314fec9141f5d9014eacd1487103c85261cc4ea31acc4d3a-merged.mount: Consumed 0 CPU time Nov 16 15:11:16.836266 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-8aae9578d418ad361f490e2147f5c78d96750b770a9a1e7edeb893bb66fcf8b9-merged.mount: Succeeded. Nov 16 15:11:16.836925 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-8aae9578d418ad361f490e2147f5c78d96750b770a9a1e7edeb893bb66fcf8b9-merged.mount: Consumed 0 CPU time Nov 16 15:11:16.836967 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-7a914dca\x2d3986\x2d42a0\x2dab4c\x2dab19848f2047.mount: Succeeded. Nov 16 15:11:16.838157 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-7a914dca\x2d3986\x2d42a0\x2dab4c\x2dab19848f2047.mount: Consumed 0 CPU time Nov 16 15:11:16.838182 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-7a914dca\x2d3986\x2d42a0\x2dab4c\x2dab19848f2047.mount: Succeeded. Nov 16 15:11:16.838916 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-7a914dca\x2d3986\x2d42a0\x2dab4c\x2dab19848f2047.mount: Consumed 0 CPU time Nov 16 15:11:16.839404 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-7a914dca\x2d3986\x2d42a0\x2dab4c\x2dab19848f2047.mount: Succeeded. Nov 16 15:11:16.840690 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-7a914dca\x2d3986\x2d42a0\x2dab4c\x2dab19848f2047.mount: Consumed 0 CPU time Nov 16 15:11:16.840713 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c-userdata-shm.mount: Succeeded. Nov 16 15:11:16.841500 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:16.841530 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-e936529a\x2d7e5f\x2d4ca3\x2db47c\x2d4441e6fa4a10.mount: Succeeded. Nov 16 15:11:16.842276 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-e936529a\x2d7e5f\x2d4ca3\x2db47c\x2d4441e6fa4a10.mount: Consumed 0 CPU time Nov 16 15:11:16.842295 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-e936529a\x2d7e5f\x2d4ca3\x2db47c\x2d4441e6fa4a10.mount: Succeeded. Nov 16 15:11:16.843181 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-e936529a\x2d7e5f\x2d4ca3\x2db47c\x2d4441e6fa4a10.mount: Consumed 0 CPU time Nov 16 15:11:16.843199 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-e936529a\x2d7e5f\x2d4ca3\x2db47c\x2d4441e6fa4a10.mount: Succeeded. Nov 16 15:11:16.843969 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-e936529a\x2d7e5f\x2d4ca3\x2db47c\x2d4441e6fa4a10.mount: Consumed 0 CPU time Nov 16 15:11:16.843987 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3-userdata-shm.mount: Succeeded. Nov 16 15:11:16.845488 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:16.845517 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-421d5c3ebbdf391262e93c3b9e2cb26418b84dcec35d53e34e33ba0a9b231ffc-merged.mount: Succeeded. Nov 16 15:11:16.846481 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-421d5c3ebbdf391262e93c3b9e2cb26418b84dcec35d53e34e33ba0a9b231ffc-merged.mount: Consumed 0 CPU time Nov 16 15:11:16.846513 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-plugins-kubernetes.io-azure\x2ddisk-volumeDevices-ropatil1611az\x2dgs6zj\x2ddynamic\x2dpvc\x2deb8a8acd\x2da586\x2d4d92\x2dbb1b\x2d190c4432313e-5df14924\x2ddd8b\x2d4deb\x2da83b\x2d53ff611d5e47.mount: Succeeded. Nov 16 15:11:16.847375 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-plugins-kubernetes.io-azure\x2ddisk-volumeDevices-ropatil1611az\x2dgs6zj\x2ddynamic\x2dpvc\x2deb8a8acd\x2da586\x2d4d92\x2dbb1b\x2d190c4432313e-5df14924\x2ddd8b\x2d4deb\x2da83b\x2d53ff611d5e47.mount: Consumed 0 CPU time Nov 16 15:11:16.847409 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-5df14924\x2ddd8b\x2d4deb\x2da83b\x2d53ff611d5e47-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d95fd9.mount: Succeeded. Nov 16 15:11:16.848093 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-5df14924\x2ddd8b\x2d4deb\x2da83b\x2d53ff611d5e47-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d95fd9.mount: Consumed 0 CPU time Nov 16 15:11:16.848112 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-a2939f2260baa3dddf730cda9fcd0cf0662da9c6a5beffdf070bacb17c267b42-merged.mount: Succeeded. Nov 16 15:11:16.850101 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-a2939f2260baa3dddf730cda9fcd0cf0662da9c6a5beffdf070bacb17c267b42-merged.mount: Consumed 0 CPU time Nov 16 15:11:16.850143 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7-userdata-shm.mount: Succeeded. Nov 16 15:11:16.850912 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:16.852424 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.852368 1563 patch_prober.go:29] interesting pod/loki-operator-controller-manager-67cc5866c9-m67tv container/manager namespace/openshift-operators-redhat: Readiness probe status=failure output="Get \"http://10.129.2.11:8081/readyz\": dial tcp 10.129.2.11:8081: i/o timeout (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 16 15:11:16.852939 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:16.852464 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" podUID=9502a499-9f8b-4a15-99d8-714a0cc37bea containerName="manager" probeResult=failure output="Get \"http://10.129.2.11:8081/readyz\": dial tcp 10.129.2.11:8081: i/o timeout (Client.Timeout exceeded while awaiting headers)" Nov 16 15:11:16.862426 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-8defb3e9\x2d54f6\x2d49a9\x2da23b\x2d266a54369420.mount: Succeeded. Nov 16 15:11:16.863514 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-8defb3e9\x2d54f6\x2d49a9\x2da23b\x2d266a54369420.mount: Consumed 0 CPU time Nov 16 15:11:16.863531 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a-userdata-shm.mount: Succeeded. Nov 16 15:11:16.864256 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:16.964884 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00288|connmgr|INFO|br0<->unix#647: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:16.998210 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00289|connmgr|INFO|br0<->unix#650: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:17.018978 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:17.018937 1563 patch_prober.go:29] interesting pod/elasticsearch-operator-66b6655fcb-ksdmt container/elasticsearch-operator namespace/openshift-operators-redhat: Readiness probe status=failure output="Get \"http://10.129.2.12:8081/readyz\": dial tcp 10.129.2.12:8081: connect: connection refused" start-of-body= Nov 16 15:11:17.019156 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:17.019013 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" podUID=220ce491-f766-41ee-ab02-aa1546eb7ed3 containerName="elasticsearch-operator" probeResult=failure output="Get \"http://10.129.2.12:8081/readyz\": dial tcp 10.129.2.12:8081: connect: connection refused" Nov 16 15:11:17.027765 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00290|bridge|INFO|bridge br0: deleted interface veth93dc054e on port 27 Nov 16 15:11:17.034461 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth93dc054e left promiscuous mode Nov 16 15:11:17.037074 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-3d0de90cb684f2061da39325b8169902adfe9c9070ab78662fa6f96b92906935-merged.mount: Succeeded. Nov 16 15:11:17.038142 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-3d0de90cb684f2061da39325b8169902adfe9c9070ab78662fa6f96b92906935-merged.mount: Consumed 0 CPU time Nov 16 15:11:17.041214 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.041162956Z" level=info msg="Stopped container d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric" id=91af6143-6266-4a92-8c13-8ac5acecff01 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:17.041749 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.041716361Z" level=info msg="Stopping pod sandbox: a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe" id=1730505e-df20-4a2a-bff6-81aec2c88c4b name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:17.042009 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.041959963Z" level=info msg="Got pod network &{Name:alertmanager-main-0 Namespace:openshift-monitoring ID:a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe UID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 NetNS:/var/run/netns/da68e262-d66c-49ee-9218-827c778cb004 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:17.042245 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.042217065Z" level=info msg="Deleting pod openshift-monitoring_alertmanager-main-0 from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:17.042501 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:16Z [verbose] Del: openshift-monitoring:prometheus-operator-admission-webhook-6c69dfd44-hxqbj:35f6c830-e480-446f-a5cc-a6aaf007ea9a:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:17.051528 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-b34f6c7c\x2d5574\x2d4659\x2dbd55\x2d848e4e74be11.mount: Succeeded. Nov 16 15:11:17.052406 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-b34f6c7c\x2d5574\x2d4659\x2dbd55\x2d848e4e74be11.mount: Consumed 0 CPU time Nov 16 15:11:17.064409 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-b34f6c7c\x2d5574\x2d4659\x2dbd55\x2d848e4e74be11.mount: Succeeded. Nov 16 15:11:17.065358 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-b34f6c7c\x2d5574\x2d4659\x2dbd55\x2d848e4e74be11.mount: Consumed 0 CPU time Nov 16 15:11:17.107217 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00291|connmgr|INFO|br0<->unix#653: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:17.139932 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00292|connmgr|INFO|br0<->unix#656: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:17.237779 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00293|bridge|INFO|bridge br0: deleted interface vethb22da9eb on port 38 Nov 16 15:11:17.245168 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethb22da9eb left promiscuous mode Nov 16 15:11:17.253899 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:17Z [verbose] Del: openshift-monitoring:alertmanager-main-0:e0a5ccca-d055-4b4c-820e-515fbf3f92d5:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:17.380357 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.380102836Z" level=info msg="Stopped container c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e: openshift-monitoring/telemeter-client-55c5556665-7vs77/kube-rbac-proxy" id=5dc6d760-b384-4192-9ad4-d08265fd0c51 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:17.380992 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.380910343Z" level=info msg="Stopping pod sandbox: a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145" id=db6290ff-ada1-4e20-9f50-8724d61aa2e5 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:17.381231 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.381198146Z" level=info msg="Got pod network &{Name:telemeter-client-55c5556665-7vs77 Namespace:openshift-monitoring ID:a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145 UID:add06140-663d-48c0-ac00-643bd58a3f7d NetNS:/var/run/netns/52f1a65d-b622-4276-b4a0-e16b4f43bd50 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:17.381372 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.381346047Z" level=info msg="Deleting pod openshift-monitoring_telemeter-client-55c5556665-7vs77 from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:17.445559 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00294|connmgr|INFO|br0<->unix#659: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:17.476187 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00295|connmgr|INFO|br0<->unix#662: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:17.505821 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00296|bridge|INFO|bridge br0: deleted interface veth442b6033 on port 32 Nov 16 15:11:17.512258 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth442b6033 left promiscuous mode Nov 16 15:11:17.521172 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:17Z [verbose] Del: openshift-monitoring:telemeter-client-55c5556665-7vs77:add06140-663d-48c0-ac00-643bd58a3f7d:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:17.526964 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.526876384Z" level=info msg="Stopped container 83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/kube-rbac-proxy-self" id=fc4d9ec5-cc20-4bca-999c-ee8181f4ca3b name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:17.680473 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.680307088Z" level=info msg="Stopped container 89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/kube-rbac-proxy-main" id=17470c70-5853-4b7f-8990-ebe08eaa13d5 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:17.681063 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.681020794Z" level=info msg="Stopping pod sandbox: 9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c" id=b59bac2c-a9bf-48e7-aa54-ceb84db7d308 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:17.681309 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.681273496Z" level=info msg="Got pod network &{Name:openshift-state-metrics-6959bf55f6-ppwrv Namespace:openshift-monitoring ID:9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c UID:c8013b4f-15b6-4bac-bfc9-a45301362f0c NetNS:/var/run/netns/7ac5f633-b1e2-412c-8e55-6972cfdae313 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:17.681456 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.681427397Z" level=info msg="Deleting pod openshift-monitoring_openshift-state-metrics-6959bf55f6-ppwrv from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:17.726443 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:17.726393 1563 patch_prober.go:29] interesting pod/prometheus-adapter-6fb766b4f6-jcwvx container/prometheus-adapter namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.129.2.32:6443/readyz\": dial tcp 10.129.2.32:6443: connect: connection refused" start-of-body= Nov 16 15:11:17.726443 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:17.726474 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" podUID=17691c72-c4d7-4e7f-b345-a0447ae7d4c6 containerName="prometheus-adapter" probeResult=failure output="Get \"https://10.129.2.32:6443/readyz\": dial tcp 10.129.2.32:6443: connect: connection refused" Nov 16 15:11:17.788837 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:17.788751710Z" level=info msg="Stopped pod sandbox: 18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4" id=86c38137-de95-4196-9d99-fab879cf7ea5 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:17.830848 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-da68e262\x2dd66c\x2d49ee\x2d9218\x2d827c778cb004.mount: Succeeded. Nov 16 15:11:17.832169 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-da68e262\x2dd66c\x2d49ee\x2d9218\x2d827c778cb004.mount: Consumed 0 CPU time Nov 16 15:11:17.832187 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-da68e262\x2dd66c\x2d49ee\x2d9218\x2d827c778cb004.mount: Succeeded. Nov 16 15:11:17.832977 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-da68e262\x2dd66c\x2d49ee\x2d9218\x2d827c778cb004.mount: Consumed 0 CPU time Nov 16 15:11:17.832990 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-da68e262\x2dd66c\x2d49ee\x2d9218\x2d827c778cb004.mount: Succeeded. Nov 16 15:11:17.833692 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-da68e262\x2dd66c\x2d49ee\x2d9218\x2d827c778cb004.mount: Consumed 0 CPU time Nov 16 15:11:17.833704 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe-userdata-shm.mount: Succeeded. Nov 16 15:11:17.834372 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:17.834399 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6eee94934f242368215d1479181caec6d9eeeb54bbe8fc758458c3be70c2178e-merged.mount: Succeeded. Nov 16 15:11:17.835064 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-6eee94934f242368215d1479181caec6d9eeeb54bbe8fc758458c3be70c2178e-merged.mount: Consumed 0 CPU time Nov 16 15:11:17.835074 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-51e2adaa12fcebc6542c6409442f96697bcf3a632427065193df2993eebe9679-merged.mount: Succeeded. Nov 16 15:11:17.835788 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-51e2adaa12fcebc6542c6409442f96697bcf3a632427065193df2993eebe9679-merged.mount: Consumed 0 CPU time Nov 16 15:11:17.835810 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-52f1a65d\x2db622\x2d4276\x2db4a0\x2de16b4f43bd50.mount: Succeeded. Nov 16 15:11:17.836550 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-52f1a65d\x2db622\x2d4276\x2db4a0\x2de16b4f43bd50.mount: Consumed 0 CPU time Nov 16 15:11:17.836563 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-52f1a65d\x2db622\x2d4276\x2db4a0\x2de16b4f43bd50.mount: Succeeded. Nov 16 15:11:17.837354 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-52f1a65d\x2db622\x2d4276\x2db4a0\x2de16b4f43bd50.mount: Consumed 0 CPU time Nov 16 15:11:17.837369 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-52f1a65d\x2db622\x2d4276\x2db4a0\x2de16b4f43bd50.mount: Succeeded. Nov 16 15:11:17.838355 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-52f1a65d\x2db622\x2d4276\x2db4a0\x2de16b4f43bd50.mount: Consumed 0 CPU time Nov 16 15:11:17.838373 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145-userdata-shm.mount: Succeeded. Nov 16 15:11:17.839118 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:17.839178 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-b34f6c7c\x2d5574\x2d4659\x2dbd55\x2d848e4e74be11.mount: Succeeded. Nov 16 15:11:17.839878 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-b34f6c7c\x2d5574\x2d4659\x2dbd55\x2d848e4e74be11.mount: Consumed 0 CPU time Nov 16 15:11:17.839892 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844-userdata-shm.mount: Succeeded. Nov 16 15:11:17.840691 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:17.840707 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-d8cb7fb587b6235471fdaacc4642964e055b81169ed0efc21f1b858ecc5ebaae-merged.mount: Succeeded. Nov 16 15:11:17.841503 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-d8cb7fb587b6235471fdaacc4642964e055b81169ed0efc21f1b858ecc5ebaae-merged.mount: Consumed 0 CPU time Nov 16 15:11:17.841540 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-20b75ab222d1cd1922393198992c06c0fbc8fc3c53b429afa9ee4821225c2d6d-merged.mount: Succeeded. Nov 16 15:11:17.842308 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-20b75ab222d1cd1922393198992c06c0fbc8fc3c53b429afa9ee4821225c2d6d-merged.mount: Consumed 0 CPU time Nov 16 15:11:17.856529 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00297|connmgr|INFO|br0<->unix#665: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:17.889816 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00298|connmgr|INFO|br0<->unix#668: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:17.893582 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:17.893536 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-e53f0905-53ec-4da6-829f-022ba78b9629\") pod \"e3d0df21-083b-4e97-a997-e9a7e992eda4\" (UID: \"e3d0df21-083b-4e97-a997-e9a7e992eda4\") " Nov 16 15:11:17.894008 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:17.893609 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k78d2\" (UniqueName: \"kubernetes.io/projected/e3d0df21-083b-4e97-a997-e9a7e992eda4-kube-api-access-k78d2\") pod \"e3d0df21-083b-4e97-a997-e9a7e992eda4\" (UID: \"e3d0df21-083b-4e97-a997-e9a7e992eda4\") " Nov 16 15:11:17.904500 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e3d0df21\x2d083b\x2d4e97\x2da997\x2de9a7e992eda4-volumes-kubernetes.io\x7eazure\x2ddisk-pvc\x2de53f0905\x2d53ec\x2d4da6\x2d829f\x2d022ba78b9629.mount: Succeeded. Nov 16 15:11:17.906335 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e3d0df21\x2d083b\x2d4e97\x2da997\x2de9a7e992eda4-volumes-kubernetes.io\x7eazure\x2ddisk-pvc\x2de53f0905\x2d53ec\x2d4da6\x2d829f\x2d022ba78b9629.mount: Consumed 0 CPU time Nov 16 15:11:17.906365 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e3d0df21\x2d083b\x2d4e97\x2da997\x2de9a7e992eda4-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dk78d2.mount: Succeeded. Nov 16 15:11:17.907742 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e3d0df21\x2d083b\x2d4e97\x2da997\x2de9a7e992eda4-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dk78d2.mount: Consumed 0 CPU time Nov 16 15:11:17.922743 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00299|bridge|INFO|bridge br0: deleted interface veth36d169af on port 30 Nov 16 15:11:17.929146 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth36d169af left promiscuous mode Nov 16 15:11:17.936448 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:17.936313 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-e53f0905-53ec-4da6-829f-022ba78b9629" (OuterVolumeSpecName: "local") pod "e3d0df21-083b-4e97-a997-e9a7e992eda4" (UID: "e3d0df21-083b-4e97-a997-e9a7e992eda4"). InnerVolumeSpecName "pvc-e53f0905-53ec-4da6-829f-022ba78b9629". PluginName "kubernetes.io/azure-disk", VolumeGidValue "" Nov 16 15:11:17.936448 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:17.936430 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3d0df21-083b-4e97-a997-e9a7e992eda4-kube-api-access-k78d2" (OuterVolumeSpecName: "kube-api-access-k78d2") pod "e3d0df21-083b-4e97-a997-e9a7e992eda4" (UID: "e3d0df21-083b-4e97-a997-e9a7e992eda4"). InnerVolumeSpecName "kube-api-access-k78d2". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:17.938298 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:17Z [verbose] Del: openshift-monitoring:openshift-state-metrics-6959bf55f6-ppwrv:c8013b4f-15b6-4bac-bfc9-a45301362f0c:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:17.947997 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-7ac5f633\x2db1e2\x2d412c\x2d8e55\x2d6972cfdae313.mount: Succeeded. Nov 16 15:11:17.948937 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-7ac5f633\x2db1e2\x2d412c\x2d8e55\x2d6972cfdae313.mount: Consumed 0 CPU time Nov 16 15:11:17.993824 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-7ac5f633\x2db1e2\x2d412c\x2d8e55\x2d6972cfdae313.mount: Succeeded. Nov 16 15:11:17.994536 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:17.994493 1563 reconciler.go:293] "operationExecutor.UnmountDevice started for volume \"pvc-e53f0905-53ec-4da6-829f-022ba78b9629\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-e53f0905-53ec-4da6-829f-022ba78b9629\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" " Nov 16 15:11:17.994536 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:17.994557 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-k78d2\" (UniqueName: \"kubernetes.io/projected/e3d0df21-083b-4e97-a997-e9a7e992eda4-kube-api-access-k78d2\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:17.994811 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-7ac5f633\x2db1e2\x2d412c\x2d8e55\x2d6972cfdae313.mount: Consumed 0 CPU time Nov 16 15:11:18.012637 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:18.012555 1563 mount_helper_common.go:133] Warning: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/m2408838648" is not a mountpoint, deleting Nov 16 15:11:18.031644 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.031572 1563 attacher.go:300] azureDisk - Device /var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/m2408838648 was unmounted Nov 16 15:11:18.031763 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.031671 1563 operation_generator.go:1024] UnmountDevice succeeded for volume "pvc-e53f0905-53ec-4da6-829f-022ba78b9629" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-e53f0905-53ec-4da6-829f-022ba78b9629") on node "ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 15:11:18.036206 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:18.036104112Z" level=info msg="Stopped container 881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667: openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt/kube-rbac-proxy" id=055def66-9486-45dd-b8ea-05fb121c2577 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:18.036732 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:18.036700417Z" level=info msg="Stopping pod sandbox: 59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f" id=8941aaf1-1935-4a33-a42a-d98caf8785be name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:18.036974 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:18.036940419Z" level=info msg="Got pod network &{Name:elasticsearch-operator-66b6655fcb-ksdmt Namespace:openshift-operators-redhat ID:59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f UID:220ce491-f766-41ee-ab02-aa1546eb7ed3 NetNS:/var/run/netns/19b123c6-9795-499e-8d37-fae8e0b17a13 Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:18.037180 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:18.037117720Z" level=info msg="Deleting pod openshift-operators-redhat_elasticsearch-operator-66b6655fcb-ksdmt from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:18.096180 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.096089 1563 reconciler.go:300] "Volume detached for volume \"pvc-e53f0905-53ec-4da6-829f-022ba78b9629\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-e53f0905-53ec-4da6-829f-022ba78b9629\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"/dev/disk/azure/scsi1/lun3\"" Nov 16 15:11:18.173598 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00300|connmgr|INFO|br0<->unix#671: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:18.190383 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.190271 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx" event=&{ID:e3d0df21-083b-4e97-a997-e9a7e992eda4 Type:ContainerDied Data:18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4} Nov 16 15:11:18.197652 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-besteffort-pode3d0df21_083b_4e97_a997_e9a7e992eda4.slice. Nov 16 15:11:18.197716 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-besteffort-pode3d0df21_083b_4e97_a997_e9a7e992eda4.slice: Consumed 151ms CPU time Nov 16 15:11:18.207668 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00301|connmgr|INFO|br0<->unix#674: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:18.220692 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.220639 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx] Nov 16 15:11:18.227708 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.227678 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx] Nov 16 15:11:18.243529 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00302|bridge|INFO|bridge br0: deleted interface vetha69937be on port 13 Nov 16 15:11:18.250224 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vetha69937be left promiscuous mode Nov 16 15:11:18.259828 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:18Z [verbose] Del: openshift-operators-redhat:elasticsearch-operator-66b6655fcb-ksdmt:220ce491-f766-41ee-ab02-aa1546eb7ed3:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:18.266479 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:18.266436169Z" level=info msg="Stopped pod sandbox: 94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b" id=c099f0b9-70db-42bd-99fa-e648797f0250 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:18.399688 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.399592 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:11:18.399688 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.399712 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8r4g\" (UniqueName: \"kubernetes.io/projected/f5df478d-fc7b-4fda-b15e-1a9a05fdf611-kube-api-access-s8r4g\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:11:18.400031 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:18.399896 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:11:18.410847 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:11:18.412306 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.412268 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:11:18.416667 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r05239c35ad044a408b299e37f80a8567.scope: Succeeded. Nov 16 15:11:18.416904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:18.416875 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:11:18.916846248 +0000 UTC m=+11883.853621870 (durationBeforeRetry 500ms). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:11:18.416904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:11:18.416904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:11:18.417226 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r05239c35ad044a408b299e37f80a8567.scope: Consumed 801us CPU time Nov 16 15:11:18.460665 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.460408 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5df478d-fc7b-4fda-b15e-1a9a05fdf611-kube-api-access-s8r4g" (OuterVolumeSpecName: "kube-api-access-s8r4g") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611"). InnerVolumeSpecName "kube-api-access-s8r4g". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:18.480085 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:18.479999985Z" level=info msg="Stopped container 9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30: openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx/prometheus-adapter" id=34ede7bb-9b27-4d59-bcb7-66df895fc143 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:18.480674 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:18.480635990Z" level=info msg="Stopping pod sandbox: 9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486" id=200564d4-a53d-41c4-be80-886e9a728819 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:18.480910 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:18.480866192Z" level=info msg="Got pod network &{Name:prometheus-adapter-6fb766b4f6-jcwvx Namespace:openshift-monitoring ID:9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486 UID:17691c72-c4d7-4e7f-b345-a0447ae7d4c6 NetNS:/var/run/netns/e966bd8c-2d2d-4176-8a8d-7d70130b4c3b Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:18.481058 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:18.481027293Z" level=info msg="Deleting pod openshift-monitoring_prometheus-adapter-6fb766b4f6-jcwvx from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:18.501671 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.501624 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-s8r4g\" (UniqueName: \"kubernetes.io/projected/f5df478d-fc7b-4fda-b15e-1a9a05fdf611-kube-api-access-s8r4g\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:18.550039 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00303|connmgr|INFO|br0<->unix#677: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:18.582004 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00304|connmgr|INFO|br0<->unix#680: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:18.596405 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.596364 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=e3d0df21-083b-4e97-a997-e9a7e992eda4 path="/var/lib/kubelet/pods/e3d0df21-083b-4e97-a997-e9a7e992eda4/volumes" Nov 16 15:11:18.610675 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00305|bridge|INFO|bridge br0: deleted interface veth74ac3f97 on port 33 Nov 16 15:11:18.617209 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device veth74ac3f97 left promiscuous mode Nov 16 15:11:18.624693 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:18.624548013Z" level=info msg="Stopped pod sandbox: 9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9" id=b2f6af93-0a75-475b-a081-cabc28f85153 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:18.626999 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:18Z [verbose] Del: openshift-monitoring:prometheus-adapter-6fb766b4f6-jcwvx:17691c72-c4d7-4e7f-b345-a0447ae7d4c6:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:18.735338 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:18.735120853Z" level=info msg="Stopped pod sandbox: 5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78" id=5c0fff1b-ece6-424b-8539-ae56c1708535 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:18.804582 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.804524 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxtpb\" (UniqueName: \"kubernetes.io/projected/5eb5be5f-46bd-425a-a486-e2641bb0ad88-kube-api-access-mxtpb\") pod \"5eb5be5f-46bd-425a-a486-e2641bb0ad88\" (UID: \"5eb5be5f-46bd-425a-a486-e2641bb0ad88\") " Nov 16 15:11:18.821754 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:18.821704089Z" level=info msg="Stopped pod sandbox: 6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d" id=22a621e5-92f8-406d-8d67-01d1c19703a4 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:18.830311 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-33a19a71280dcabb655c4b3c69ddb1fcde79e83c7ff9d6dac64eb9d7a54593a1-merged.mount: Succeeded. Nov 16 15:11:18.831352 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-33a19a71280dcabb655c4b3c69ddb1fcde79e83c7ff9d6dac64eb9d7a54593a1-merged.mount: Consumed 0 CPU time Nov 16 15:11:18.831385 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-5eb5be5f\x2d46bd\x2d425a\x2da486\x2de2641bb0ad88-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dmxtpb.mount: Succeeded. Nov 16 15:11:18.832120 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-5eb5be5f\x2d46bd\x2d425a\x2da486\x2de2641bb0ad88-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dmxtpb.mount: Consumed 0 CPU time Nov 16 15:11:18.832160 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-8a53b37f47d2ee38b2ea90dd2c2ed2a70bece6cf923987f7049446df537d9caf-merged.mount: Succeeded. Nov 16 15:11:18.832849 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-8a53b37f47d2ee38b2ea90dd2c2ed2a70bece6cf923987f7049446df537d9caf-merged.mount: Consumed 0 CPU time Nov 16 15:11:18.832889 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-e966bd8c\x2d2d2d\x2d4176\x2d8a8d\x2d7d70130b4c3b.mount: Succeeded. Nov 16 15:11:18.833589 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-e966bd8c\x2d2d2d\x2d4176\x2d8a8d\x2d7d70130b4c3b.mount: Consumed 0 CPU time Nov 16 15:11:18.833604 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-e966bd8c\x2d2d2d\x2d4176\x2d8a8d\x2d7d70130b4c3b.mount: Succeeded. Nov 16 15:11:18.834306 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-e966bd8c\x2d2d2d\x2d4176\x2d8a8d\x2d7d70130b4c3b.mount: Consumed 0 CPU time Nov 16 15:11:18.834321 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-e966bd8c\x2d2d2d\x2d4176\x2d8a8d\x2d7d70130b4c3b.mount: Succeeded. Nov 16 15:11:18.834989 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-e966bd8c\x2d2d2d\x2d4176\x2d8a8d\x2d7d70130b4c3b.mount: Consumed 0 CPU time Nov 16 15:11:18.835002 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486-userdata-shm.mount: Succeeded. Nov 16 15:11:18.835686 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:18.835702 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-7ac5f633\x2db1e2\x2d412c\x2d8e55\x2d6972cfdae313.mount: Succeeded. Nov 16 15:11:18.836399 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-7ac5f633\x2db1e2\x2d412c\x2d8e55\x2d6972cfdae313.mount: Consumed 0 CPU time Nov 16 15:11:18.836415 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c-userdata-shm.mount: Succeeded. Nov 16 15:11:18.837096 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:18.837172 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-plugins-kubernetes.io-azure\x2ddisk-mounts-m2408838648.mount: Succeeded. Nov 16 15:11:18.837898 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-plugins-kubernetes.io-azure\x2ddisk-mounts-m2408838648.mount: Consumed 0 CPU time Nov 16 15:11:18.837913 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-fef5388373736384b39998f53b36c41a1b31fa6bba3bcd32cb3de253d6f30c10-merged.mount: Succeeded. Nov 16 15:11:18.838618 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-fef5388373736384b39998f53b36c41a1b31fa6bba3bcd32cb3de253d6f30c10-merged.mount: Consumed 0 CPU time Nov 16 15:11:18.838633 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-052885ead7d8ef654201328b98de5a2115d1f262a0ca2041fcb7d908fdd932f6-merged.mount: Succeeded. Nov 16 15:11:18.839432 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-052885ead7d8ef654201328b98de5a2115d1f262a0ca2041fcb7d908fdd932f6-merged.mount: Consumed 0 CPU time Nov 16 15:11:18.839494 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-f5df478d\x2dfc7b\x2d4fda\x2db15e\x2d1a9a05fdf611-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2ds8r4g.mount: Succeeded. Nov 16 15:11:18.840241 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-f5df478d\x2dfc7b\x2d4fda\x2db15e\x2d1a9a05fdf611-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2ds8r4g.mount: Consumed 0 CPU time Nov 16 15:11:18.840259 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-a5607b8d613458e9aad5ffdba90c7176a48f2b7228a918d5c8b8063832debd0b-merged.mount: Succeeded. Nov 16 15:11:18.841140 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-a5607b8d613458e9aad5ffdba90c7176a48f2b7228a918d5c8b8063832debd0b-merged.mount: Consumed 0 CPU time Nov 16 15:11:18.841158 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-19b123c6\x2d9795\x2d499e\x2d8d37\x2dfae8e0b17a13.mount: Succeeded. Nov 16 15:11:18.841838 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-19b123c6\x2d9795\x2d499e\x2d8d37\x2dfae8e0b17a13.mount: Consumed 0 CPU time Nov 16 15:11:18.841852 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-19b123c6\x2d9795\x2d499e\x2d8d37\x2dfae8e0b17a13.mount: Succeeded. Nov 16 15:11:18.842546 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-19b123c6\x2d9795\x2d499e\x2d8d37\x2dfae8e0b17a13.mount: Consumed 0 CPU time Nov 16 15:11:18.842561 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-19b123c6\x2d9795\x2d499e\x2d8d37\x2dfae8e0b17a13.mount: Succeeded. Nov 16 15:11:18.843250 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-19b123c6\x2d9795\x2d499e\x2d8d37\x2dfae8e0b17a13.mount: Consumed 0 CPU time Nov 16 15:11:18.843266 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f-userdata-shm.mount: Succeeded. Nov 16 15:11:18.843972 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:18.843989 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-aac32ec3ce68ca5b5567d7e94702bbfe2ba3c6952da9ad1b373f5ac0a81e11cb-merged.mount: Succeeded. Nov 16 15:11:18.844680 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-aac32ec3ce68ca5b5567d7e94702bbfe2ba3c6952da9ad1b373f5ac0a81e11cb-merged.mount: Consumed 0 CPU time Nov 16 15:11:18.848578 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.848523 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb5be5f-46bd-425a-a486-e2641bb0ad88-kube-api-access-mxtpb" (OuterVolumeSpecName: "kube-api-access-mxtpb") pod "5eb5be5f-46bd-425a-a486-e2641bb0ad88" (UID: "5eb5be5f-46bd-425a-a486-e2641bb0ad88"). InnerVolumeSpecName "kube-api-access-mxtpb". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:18.905724 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.905604 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnznm\" (UniqueName: \"kubernetes.io/projected/4e58fe69-e225-425e-a6ca-0a7f136dbdca-kube-api-access-wnznm\") pod \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\" (UID: \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\") " Nov 16 15:11:18.906330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.905757 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-4618b52d-ff9e-4972-becd-c0da785f9912\") pod \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\" (UID: \"4e58fe69-e225-425e-a6ca-0a7f136dbdca\") " Nov 16 15:11:18.906330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.905962 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-mxtpb\" (UniqueName: \"kubernetes.io/projected/5eb5be5f-46bd-425a-a486-e2641bb0ad88-kube-api-access-mxtpb\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:18.915685 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-4e58fe69\x2de225\x2d425e\x2da6ca\x2d0a7f136dbdca-volumes-kubernetes.io\x7eazure\x2ddisk-pvc\x2d4618b52d\x2dff9e\x2d4972\x2dbecd\x2dc0da785f9912.mount: Succeeded. Nov 16 15:11:18.916583 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-4e58fe69\x2de225\x2d425e\x2da6ca\x2d0a7f136dbdca-volumes-kubernetes.io\x7eazure\x2ddisk-pvc\x2d4618b52d\x2dff9e\x2d4972\x2dbecd\x2dc0da785f9912.mount: Consumed 0 CPU time Nov 16 15:11:18.916606 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-4e58fe69\x2de225\x2d425e\x2da6ca\x2d0a7f136dbdca-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dwnznm.mount: Succeeded. Nov 16 15:11:18.917306 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-4e58fe69\x2de225\x2d425e\x2da6ca\x2d0a7f136dbdca-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dwnznm.mount: Consumed 0 CPU time Nov 16 15:11:18.928834 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.928759 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e58fe69-e225-425e-a6ca-0a7f136dbdca-kube-api-access-wnznm" (OuterVolumeSpecName: "kube-api-access-wnznm") pod "4e58fe69-e225-425e-a6ca-0a7f136dbdca" (UID: "4e58fe69-e225-425e-a6ca-0a7f136dbdca"). InnerVolumeSpecName "kube-api-access-wnznm". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:18.930302 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:18.930248 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-4618b52d-ff9e-4972-becd-c0da785f9912" (OuterVolumeSpecName: "local") pod "4e58fe69-e225-425e-a6ca-0a7f136dbdca" (UID: "4e58fe69-e225-425e-a6ca-0a7f136dbdca"). InnerVolumeSpecName "pvc-4618b52d-ff9e-4972-becd-c0da785f9912". PluginName "kubernetes.io/azure-disk", VolumeGidValue "" Nov 16 15:11:19.007758 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.007467 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6zxl\" (UniqueName: \"kubernetes.io/projected/b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd-kube-api-access-z6zxl\") pod \"b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd\" (UID: \"b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd\") " Nov 16 15:11:19.007758 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.007643 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:11:19.007989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.007904 1563 reconciler.go:293] "operationExecutor.UnmountDevice started for volume \"pvc-4618b52d-ff9e-4972-becd-c0da785f9912\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-4618b52d-ff9e-4972-becd-c0da785f9912\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" " Nov 16 15:11:19.007989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.007927 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-wnznm\" (UniqueName: \"kubernetes.io/projected/4e58fe69-e225-425e-a6ca-0a7f136dbdca-kube-api-access-wnznm\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:19.007989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:19.007929 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:11:19.007989 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:19.007969 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:11:19.017559 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-b6d86f0b\x2de1d0\x2d4ffe\x2db9c6\x2d85f4a93f08dd-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dz6zxl.mount: Succeeded. Nov 16 15:11:19.018474 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-b6d86f0b\x2de1d0\x2d4ffe\x2db9c6\x2d85f4a93f08dd-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dz6zxl.mount: Consumed 0 CPU time Nov 16 15:11:19.021468 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:19.021433 1563 mount_helper_common.go:133] Warning: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/m3413458433" is not a mountpoint, deleting Nov 16 15:11:19.025294 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-plugins-kubernetes.io-azure\x2ddisk-mounts-m3413458433.mount: Succeeded. Nov 16 15:11:19.026108 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-plugins-kubernetes.io-azure\x2ddisk-mounts-m3413458433.mount: Consumed 0 CPU time Nov 16 15:11:19.028434 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.028397 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd-kube-api-access-z6zxl" (OuterVolumeSpecName: "kube-api-access-z6zxl") pod "b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd" (UID: "b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd"). InnerVolumeSpecName "kube-api-access-z6zxl". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:19.029971 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:11:19.031008 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.030978 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:11:19.035355 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:19.035320 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:11:20.035292704 +0000 UTC m=+11884.972068326 (durationBeforeRetry 1s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:11:19.035355 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:11:19.035355 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:11:19.035795 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r8b68d93ed06e414a9b4f03bea972ccae.scope: Succeeded. Nov 16 15:11:19.036350 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r8b68d93ed06e414a9b4f03bea972ccae.scope: Consumed 829us CPU time Nov 16 15:11:19.042115 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.042044 1563 attacher.go:300] azureDisk - Device /var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/m3413458433 was unmounted Nov 16 15:11:19.042235 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.042123 1563 operation_generator.go:1024] UnmountDevice succeeded for volume "pvc-4618b52d-ff9e-4972-becd-c0da785f9912" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-4618b52d-ff9e-4972-becd-c0da785f9912") on node "ropatil1611az-gs6zj-worker-westus-j4ngg" Nov 16 15:11:19.109632 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.109530 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-z6zxl\" (UniqueName: \"kubernetes.io/projected/b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd-kube-api-access-z6zxl\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:19.109632 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.109615 1563 reconciler.go:300] "Volume detached for volume \"pvc-4618b52d-ff9e-4972-becd-c0da785f9912\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-4618b52d-ff9e-4972-becd-c0da785f9912\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"/dev/disk/azure/scsi1/lun1\"" Nov 16 15:11:19.173553 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:19.173447478Z" level=info msg="Stopped pod sandbox: 0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7" id=a85b5b2a-a4dd-40a7-b03b-21ca7045c38e name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:19.181724 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-fa65d40f2125ac549f6b6c601db1274f0c77e70f0c4c74fc6c1aae3f27bfc74a-merged.mount: Succeeded. Nov 16 15:11:19.182655 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-fa65d40f2125ac549f6b6c601db1274f0c77e70f0c4c74fc6c1aae3f27bfc74a-merged.mount: Consumed 0 CPU time Nov 16 15:11:19.196471 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.196434 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/qe-app-registry-9hthz" event=&{ID:5eb5be5f-46bd-425a-a486-e2641bb0ad88 Type:ContainerDied Data:9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9} Nov 16 15:11:19.198021 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.197991 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv" event=&{ID:9502a499-9f8b-4a15-99d8-714a0cc37bea Type:ContainerDied Data:0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7} Nov 16 15:11:19.199194 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.199167 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4" event=&{ID:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Type:ContainerDied Data:94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b} Nov 16 15:11:19.200273 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.200249 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf" event=&{ID:b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd Type:ContainerDied Data:6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d} Nov 16 15:11:19.203120 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.203090 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg" event=&{ID:4e58fe69-e225-425e-a6ca-0a7f136dbdca Type:ContainerDied Data:5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78} Nov 16 15:11:19.205918 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod5eb5be5f_46bd_425a_a486_e2641bb0ad88.slice. Nov 16 15:11:19.206100 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod5eb5be5f_46bd_425a_a486_e2641bb0ad88.slice: Consumed 6.630s CPU time Nov 16 15:11:19.209399 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-besteffort-podf5df478d_fc7b_4fda_b15e_1a9a05fdf611.slice. Nov 16 15:11:19.209458 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-besteffort-podf5df478d_fc7b_4fda_b15e_1a9a05fdf611.slice: Consumed 94ms CPU time Nov 16 15:11:19.212427 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-besteffort-podb6d86f0b_e1d0_4ffe_b9c6_85f4a93f08dd.slice. Nov 16 15:11:19.212496 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-besteffort-podb6d86f0b_e1d0_4ffe_b9c6_85f4a93f08dd.slice: Consumed 11.005s CPU time Nov 16 15:11:19.227583 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod4e58fe69_e225_425e_a6ca_0a7f136dbdca.slice. Nov 16 15:11:19.227652 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-besteffort-pod4e58fe69_e225_425e_a6ca_0a7f136dbdca.slice: Consumed 115ms CPU time Nov 16 15:11:19.235278 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.235233 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-marketplace/qe-app-registry-9hthz] Nov 16 15:11:19.256243 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.256206 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-marketplace/qe-app-registry-9hthz] Nov 16 15:11:19.312779 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.312615 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"loki-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-loki-operator-metrics-cert\") pod \"9502a499-9f8b-4a15-99d8-714a0cc37bea\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " Nov 16 15:11:19.312779 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.312696 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cf6kf\" (UniqueName: \"kubernetes.io/projected/9502a499-9f8b-4a15-99d8-714a0cc37bea-kube-api-access-cf6kf\") pod \"9502a499-9f8b-4a15-99d8-714a0cc37bea\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " Nov 16 15:11:19.312779 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.312769 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-webhook-cert\") pod \"9502a499-9f8b-4a15-99d8-714a0cc37bea\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " Nov 16 15:11:19.312990 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.312808 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-apiservice-cert\") pod \"9502a499-9f8b-4a15-99d8-714a0cc37bea\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " Nov 16 15:11:19.312990 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.312843 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9502a499-9f8b-4a15-99d8-714a0cc37bea-manager-config\") pod \"9502a499-9f8b-4a15-99d8-714a0cc37bea\" (UID: \"9502a499-9f8b-4a15-99d8-714a0cc37bea\") " Nov 16 15:11:19.330402 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.330356 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9502a499-9f8b-4a15-99d8-714a0cc37bea-kube-api-access-cf6kf" (OuterVolumeSpecName: "kube-api-access-cf6kf") pod "9502a499-9f8b-4a15-99d8-714a0cc37bea" (UID: "9502a499-9f8b-4a15-99d8-714a0cc37bea"). InnerVolumeSpecName "kube-api-access-cf6kf". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:19.331418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.331367 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-loki-operator-metrics-cert" (OuterVolumeSpecName: "loki-operator-metrics-cert") pod "9502a499-9f8b-4a15-99d8-714a0cc37bea" (UID: "9502a499-9f8b-4a15-99d8-714a0cc37bea"). InnerVolumeSpecName "loki-operator-metrics-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:19.333056 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:19.333017 1563 mount_helper_common.go:133] Warning: "/var/lib/kubelet/pods/9502a499-9f8b-4a15-99d8-714a0cc37bea/volume-subpaths/manager-config/manager/1" is not a mountpoint, deleting Nov 16 15:11:19.339537 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:19.339452 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/9502a499-9f8b-4a15-99d8-714a0cc37bea/volumes/kubernetes.io~configmap/manager-config: clearQuota called, but quotas disabled Nov 16 15:11:19.339706 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.339677 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9502a499-9f8b-4a15-99d8-714a0cc37bea-manager-config" (OuterVolumeSpecName: "manager-config") pod "9502a499-9f8b-4a15-99d8-714a0cc37bea" (UID: "9502a499-9f8b-4a15-99d8-714a0cc37bea"). InnerVolumeSpecName "manager-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:19.345459 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.345408 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9502a499-9f8b-4a15-99d8-714a0cc37bea" (UID: "9502a499-9f8b-4a15-99d8-714a0cc37bea"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:19.351418 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.351379 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-apiservice-cert" (OuterVolumeSpecName: "apiservice-cert") pod "9502a499-9f8b-4a15-99d8-714a0cc37bea" (UID: "9502a499-9f8b-4a15-99d8-714a0cc37bea"). InnerVolumeSpecName "apiservice-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:19.358842 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.358809 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf] Nov 16 15:11:19.368327 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.368297 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf] Nov 16 15:11:19.401239 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.401161 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg] Nov 16 15:11:19.409799 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.409695 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg] Nov 16 15:11:19.414063 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.413992 1563 reconciler.go:300] "Volume detached for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9502a499-9f8b-4a15-99d8-714a0cc37bea-manager-config\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:19.414063 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.414032 1563 reconciler.go:300] "Volume detached for volume \"loki-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-loki-operator-metrics-cert\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:19.414063 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.414062 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-cf6kf\" (UniqueName: \"kubernetes.io/projected/9502a499-9f8b-4a15-99d8-714a0cc37bea-kube-api-access-cf6kf\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:19.414249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.414080 1563 reconciler.go:300] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-webhook-cert\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:19.414249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.414096 1563 reconciler.go:300] "Volume detached for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9502a499-9f8b-4a15-99d8-714a0cc37bea-apiservice-cert\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:19.490998 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:19.490898176Z" level=info msg="Stopped pod sandbox: 5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3" id=62796e2e-258a-4e35-811b-addc53d3cbd7 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:19.511422 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod9502a499_9f8b_4a15_99d8_714a0cc37bea.slice. Nov 16 15:11:19.511492 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-besteffort-pod9502a499_9f8b_4a15_99d8_714a0cc37bea.slice: Consumed 37.023s CPU time Nov 16 15:11:19.531707 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.531638 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv] Nov 16 15:11:19.544979 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.544439 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv] Nov 16 15:11:19.554185 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:19.554096513Z" level=info msg="Removed container 2266674a49b5037a21e78844fd64e350e804bc58e3d291fd60f73250c583737a: migration-upgrade-48669/mydep-48669-79647ff996-g97kh/mydep-48669" id=16bb5147-1938-4a75-91ea-47665d74ac8e name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:19.554501 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.554469 1563 scope.go:110] "RemoveContainer" containerID="e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8" Nov 16 15:11:19.556338 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:19.556161231Z" level=info msg="Removing container: e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8" id=4001f1f5-7f64-413e-aa52-d9ef4b1bb5dc name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:19.616142 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.616048 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/454b3a37-199c-4abc-a79a-8de93eab2423-kube-state-metrics-kube-rbac-proxy-config\") pod \"454b3a37-199c-4abc-a79a-8de93eab2423\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " Nov 16 15:11:19.616142 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.616140 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/454b3a37-199c-4abc-a79a-8de93eab2423-metrics-client-ca\") pod \"454b3a37-199c-4abc-a79a-8de93eab2423\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " Nov 16 15:11:19.616416 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.616180 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/454b3a37-199c-4abc-a79a-8de93eab2423-volume-directive-shadow\") pod \"454b3a37-199c-4abc-a79a-8de93eab2423\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " Nov 16 15:11:19.616416 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.616226 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/454b3a37-199c-4abc-a79a-8de93eab2423-kube-state-metrics-tls\") pod \"454b3a37-199c-4abc-a79a-8de93eab2423\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " Nov 16 15:11:19.616416 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.616268 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcfpr\" (UniqueName: \"kubernetes.io/projected/454b3a37-199c-4abc-a79a-8de93eab2423-kube-api-access-vcfpr\") pod \"454b3a37-199c-4abc-a79a-8de93eab2423\" (UID: \"454b3a37-199c-4abc-a79a-8de93eab2423\") " Nov 16 15:11:19.616416 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:19.616367 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/454b3a37-199c-4abc-a79a-8de93eab2423/volumes/kubernetes.io~configmap/metrics-client-ca: clearQuota called, but quotas disabled Nov 16 15:11:19.616416 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:19.616398 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/454b3a37-199c-4abc-a79a-8de93eab2423/volumes/kubernetes.io~empty-dir/volume-directive-shadow: clearQuota called, but quotas disabled Nov 16 15:11:19.616579 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.616461 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/454b3a37-199c-4abc-a79a-8de93eab2423-volume-directive-shadow" (OuterVolumeSpecName: "volume-directive-shadow") pod "454b3a37-199c-4abc-a79a-8de93eab2423" (UID: "454b3a37-199c-4abc-a79a-8de93eab2423"). InnerVolumeSpecName "volume-directive-shadow". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Nov 16 15:11:19.616579 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.616560 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/454b3a37-199c-4abc-a79a-8de93eab2423-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "454b3a37-199c-4abc-a79a-8de93eab2423" (UID: "454b3a37-199c-4abc-a79a-8de93eab2423"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:19.636389 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.636348 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/454b3a37-199c-4abc-a79a-8de93eab2423-kube-state-metrics-tls" (OuterVolumeSpecName: "kube-state-metrics-tls") pod "454b3a37-199c-4abc-a79a-8de93eab2423" (UID: "454b3a37-199c-4abc-a79a-8de93eab2423"). InnerVolumeSpecName "kube-state-metrics-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:19.641423 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.641386 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/454b3a37-199c-4abc-a79a-8de93eab2423-kube-api-access-vcfpr" (OuterVolumeSpecName: "kube-api-access-vcfpr") pod "454b3a37-199c-4abc-a79a-8de93eab2423" (UID: "454b3a37-199c-4abc-a79a-8de93eab2423"). InnerVolumeSpecName "kube-api-access-vcfpr". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:19.645419 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.645375 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/454b3a37-199c-4abc-a79a-8de93eab2423-kube-state-metrics-kube-rbac-proxy-config" (OuterVolumeSpecName: "kube-state-metrics-kube-rbac-proxy-config") pod "454b3a37-199c-4abc-a79a-8de93eab2423" (UID: "454b3a37-199c-4abc-a79a-8de93eab2423"). InnerVolumeSpecName "kube-state-metrics-kube-rbac-proxy-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:19.717684 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.717638 1563 reconciler.go:300] "Volume detached for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/454b3a37-199c-4abc-a79a-8de93eab2423-kube-state-metrics-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:19.717684 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.717687 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-vcfpr\" (UniqueName: \"kubernetes.io/projected/454b3a37-199c-4abc-a79a-8de93eab2423-kube-api-access-vcfpr\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:19.717684 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.717707 1563 reconciler.go:300] "Volume detached for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/454b3a37-199c-4abc-a79a-8de93eab2423-kube-state-metrics-kube-rbac-proxy-config\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:19.717980 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.717728 1563 reconciler.go:300] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/454b3a37-199c-4abc-a79a-8de93eab2423-metrics-client-ca\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:19.717980 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.717746 1563 reconciler.go:300] "Volume detached for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/454b3a37-199c-4abc-a79a-8de93eab2423-volume-directive-shadow\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:19.830570 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-e40782a5c971d41fac30df9927b3be27052e28fdcf5b8a475caf47296ba22119-merged.mount: Succeeded. Nov 16 15:11:19.831486 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-e40782a5c971d41fac30df9927b3be27052e28fdcf5b8a475caf47296ba22119-merged.mount: Consumed 0 CPU time Nov 16 15:11:19.831513 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-454b3a37\x2d199c\x2d4abc\x2da79a\x2d8de93eab2423-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dvcfpr.mount: Succeeded. Nov 16 15:11:19.832288 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-454b3a37\x2d199c\x2d4abc\x2da79a\x2d8de93eab2423-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dvcfpr.mount: Consumed 0 CPU time Nov 16 15:11:19.832318 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-454b3a37\x2d199c\x2d4abc\x2da79a\x2d8de93eab2423-volumes-kubernetes.io\x7esecret-kube\x2dstate\x2dmetrics\x2dkube\x2drbac\x2dproxy\x2dconfig.mount: Succeeded. Nov 16 15:11:19.833060 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-454b3a37\x2d199c\x2d4abc\x2da79a\x2d8de93eab2423-volumes-kubernetes.io\x7esecret-kube\x2dstate\x2dmetrics\x2dkube\x2drbac\x2dproxy\x2dconfig.mount: Consumed 0 CPU time Nov 16 15:11:19.833076 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-454b3a37\x2d199c\x2d4abc\x2da79a\x2d8de93eab2423-volumes-kubernetes.io\x7esecret-kube\x2dstate\x2dmetrics\x2dtls.mount: Succeeded. Nov 16 15:11:19.833763 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-454b3a37\x2d199c\x2d4abc\x2da79a\x2d8de93eab2423-volumes-kubernetes.io\x7esecret-kube\x2dstate\x2dmetrics\x2dtls.mount: Consumed 0 CPU time Nov 16 15:11:19.833802 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-9502a499\x2d9f8b\x2d4a15\x2d99d8\x2d714a0cc37bea-volume\x2dsubpaths-manager\x2dconfig-manager-1.mount: Succeeded. Nov 16 15:11:19.834503 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-9502a499\x2d9f8b\x2d4a15\x2d99d8\x2d714a0cc37bea-volume\x2dsubpaths-manager\x2dconfig-manager-1.mount: Consumed 0 CPU time Nov 16 15:11:19.834522 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-9502a499\x2d9f8b\x2d4a15\x2d99d8\x2d714a0cc37bea-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dcf6kf.mount: Succeeded. Nov 16 15:11:19.835205 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-9502a499\x2d9f8b\x2d4a15\x2d99d8\x2d714a0cc37bea-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dcf6kf.mount: Consumed 0 CPU time Nov 16 15:11:19.835220 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-9502a499\x2d9f8b\x2d4a15\x2d99d8\x2d714a0cc37bea-volumes-kubernetes.io\x7esecret-apiservice\x2dcert.mount: Succeeded. Nov 16 15:11:19.835878 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-9502a499\x2d9f8b\x2d4a15\x2d99d8\x2d714a0cc37bea-volumes-kubernetes.io\x7esecret-apiservice\x2dcert.mount: Consumed 0 CPU time Nov 16 15:11:19.835892 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-9502a499\x2d9f8b\x2d4a15\x2d99d8\x2d714a0cc37bea-volumes-kubernetes.io\x7esecret-loki\x2doperator\x2dmetrics\x2dcert.mount: Succeeded. Nov 16 15:11:19.836599 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-9502a499\x2d9f8b\x2d4a15\x2d99d8\x2d714a0cc37bea-volumes-kubernetes.io\x7esecret-loki\x2doperator\x2dmetrics\x2dcert.mount: Consumed 0 CPU time Nov 16 15:11:19.836616 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-9502a499\x2d9f8b\x2d4a15\x2d99d8\x2d714a0cc37bea-volumes-kubernetes.io\x7esecret-webhook\x2dcert.mount: Succeeded. Nov 16 15:11:19.837326 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-9502a499\x2d9f8b\x2d4a15\x2d99d8\x2d714a0cc37bea-volumes-kubernetes.io\x7esecret-webhook\x2dcert.mount: Consumed 0 CPU time Nov 16 15:11:19.950179 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:19.949874156Z" level=info msg="Stopped pod sandbox: 4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c" id=b1dc4077-d082-4811-87f3-5b67036600b3 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:19.958981 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.958952 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b6b7d7fd9-d95wm_a776019a-1ce1-4328-aed5-a5962a4f775d/oauth-proxy/0.log" Nov 16 15:11:19.972391 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:19.972340037Z" level=info msg="Removed container e6863b838ce672298f2ed24b9f2728bbe887d7046a25b7f83517a9488dbf4ac8: migration-upgrade-49496/mydep-49496-65f7bb956d-stlkx/mydep-49496" id=4001f1f5-7f64-413e-aa52-d9ef4b1bb5dc name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:19.972567 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:19.972539 1563 scope.go:110] "RemoveContainer" containerID="ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454" Nov 16 15:11:19.973596 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:19.973540447Z" level=info msg="Removing container: ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454" id=60ba4332-f17f-4470-b776-74acbf4136c6 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:19.980900 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-ebf676002e451c2fcbe1103aaa2a11e981c6a7ce79d33318c4ca5e5eac4c83b4-merged.mount: Succeeded. Nov 16 15:11:19.981749 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-ebf676002e451c2fcbe1103aaa2a11e981c6a7ce79d33318c4ca5e5eac4c83b4-merged.mount: Consumed 0 CPU time Nov 16 15:11:20.121465 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.121419 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy\") pod \"a776019a-1ce1-4328-aed5-a5962a4f775d\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " Nov 16 15:11:20.121465 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.121491 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a776019a-1ce1-4328-aed5-a5962a4f775d-metrics-client-ca\") pod \"a776019a-1ce1-4328-aed5-a5962a4f775d\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " Nov 16 15:11:20.121821 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.121534 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdtt2\" (UniqueName: \"kubernetes.io/projected/a776019a-1ce1-4328-aed5-a5962a4f775d-kube-api-access-pdtt2\") pod \"a776019a-1ce1-4328-aed5-a5962a4f775d\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " Nov 16 15:11:20.121821 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.121576 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"a776019a-1ce1-4328-aed5-a5962a4f775d\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " Nov 16 15:11:20.121821 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.121608 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"a776019a-1ce1-4328-aed5-a5962a4f775d\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " Nov 16 15:11:20.121821 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.121641 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"thanos-querier-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a776019a-1ce1-4328-aed5-a5962a4f775d-thanos-querier-trusted-ca-bundle\") pod \"a776019a-1ce1-4328-aed5-a5962a4f775d\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " Nov 16 15:11:20.121821 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.121679 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-tls\") pod \"a776019a-1ce1-4328-aed5-a5962a4f775d\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " Nov 16 15:11:20.121821 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.121714 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-grpc-tls\") pod \"a776019a-1ce1-4328-aed5-a5962a4f775d\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " Nov 16 15:11:20.121821 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.121772 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:11:20.121821 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.121783 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/a776019a-1ce1-4328-aed5-a5962a4f775d/volumes/kubernetes.io~configmap/metrics-client-ca: clearQuota called, but quotas disabled Nov 16 15:11:20.122148 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.122007 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a776019a-1ce1-4328-aed5-a5962a4f775d-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a776019a-1ce1-4328-aed5-a5962a4f775d" (UID: "a776019a-1ce1-4328-aed5-a5962a4f775d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:20.122148 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.121806 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-thanos-querier-oauth-cookie\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-oauth-cookie\") pod \"a776019a-1ce1-4328-aed5-a5962a4f775d\" (UID: \"a776019a-1ce1-4328-aed5-a5962a4f775d\") " Nov 16 15:11:20.122231 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.122147 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/a776019a-1ce1-4328-aed5-a5962a4f775d/volumes/kubernetes.io~configmap/thanos-querier-trusted-ca-bundle: clearQuota called, but quotas disabled Nov 16 15:11:20.122273 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.122230 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:11:20.122273 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.122256 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:11:20.122340 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.122275 1563 reconciler.go:300] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a776019a-1ce1-4328-aed5-a5962a4f775d-metrics-client-ca\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.122423 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.122389 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a776019a-1ce1-4328-aed5-a5962a4f775d-thanos-querier-trusted-ca-bundle" (OuterVolumeSpecName: "thanos-querier-trusted-ca-bundle") pod "a776019a-1ce1-4328-aed5-a5962a4f775d" (UID: "a776019a-1ce1-4328-aed5-a5962a4f775d"). InnerVolumeSpecName "thanos-querier-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:20.132855 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dpdtt2.mount: Succeeded. Nov 16 15:11:20.133745 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dpdtt2.mount: Consumed 0 CPU time Nov 16 15:11:20.133769 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7esecret-secret\x2dthanos\x2dquerier\x2dkube\x2drbac\x2dproxy\x2dmetrics.mount: Succeeded. Nov 16 15:11:20.134612 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7esecret-secret\x2dthanos\x2dquerier\x2dkube\x2drbac\x2dproxy\x2dmetrics.mount: Consumed 0 CPU time Nov 16 15:11:20.134631 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7esecret-secret\x2dgrpc\x2dtls.mount: Succeeded. Nov 16 15:11:20.135380 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7esecret-secret\x2dgrpc\x2dtls.mount: Consumed 0 CPU time Nov 16 15:11:20.135396 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7esecret-secret\x2dthanos\x2dquerier\x2dkube\x2drbac\x2dproxy.mount: Succeeded. Nov 16 15:11:20.136067 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7esecret-secret\x2dthanos\x2dquerier\x2dkube\x2drbac\x2dproxy.mount: Consumed 0 CPU time Nov 16 15:11:20.136088 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7esecret-secret\x2dthanos\x2dquerier\x2dtls.mount: Succeeded. Nov 16 15:11:20.136776 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7esecret-secret\x2dthanos\x2dquerier\x2dtls.mount: Consumed 0 CPU time Nov 16 15:11:20.136794 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7esecret-secret\x2dthanos\x2dquerier\x2dkube\x2drbac\x2dproxy\x2drules.mount: Succeeded. Nov 16 15:11:20.137594 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7esecret-secret\x2dthanos\x2dquerier\x2dkube\x2drbac\x2dproxy\x2drules.mount: Consumed 0 CPU time Nov 16 15:11:20.137610 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7esecret-secret\x2dthanos\x2dquerier\x2doauth\x2dcookie.mount: Succeeded. Nov 16 15:11:20.138281 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-a776019a\x2d1ce1\x2d4328\x2daed5\x2da5962a4f775d-volumes-kubernetes.io\x7esecret-secret\x2dthanos\x2dquerier\x2doauth\x2dcookie.mount: Consumed 0 CPU time Nov 16 15:11:20.139458 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.139412 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy" (OuterVolumeSpecName: "secret-thanos-querier-kube-rbac-proxy") pod "a776019a-1ce1-4328-aed5-a5962a4f775d" (UID: "a776019a-1ce1-4328-aed5-a5962a4f775d"). InnerVolumeSpecName "secret-thanos-querier-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.140412 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.140371 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a776019a-1ce1-4328-aed5-a5962a4f775d-kube-api-access-pdtt2" (OuterVolumeSpecName: "kube-api-access-pdtt2") pod "a776019a-1ce1-4328-aed5-a5962a4f775d" (UID: "a776019a-1ce1-4328-aed5-a5962a4f775d"). InnerVolumeSpecName "kube-api-access-pdtt2". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:20.150330 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:11:20.151403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.151368 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:11:20.155732 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:20.155668 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:11:22.155636919 +0000 UTC m=+11887.092412641 (durationBeforeRetry 2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:11:20.155732 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:11:20.155732 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:11:20.156274 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r848dc1b49b1745e58eaedaaaf37000f7.scope: Succeeded. Nov 16 15:11:20.156420 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.156381 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-oauth-cookie" (OuterVolumeSpecName: "secret-thanos-querier-oauth-cookie") pod "a776019a-1ce1-4328-aed5-a5962a4f775d" (UID: "a776019a-1ce1-4328-aed5-a5962a4f775d"). InnerVolumeSpecName "secret-thanos-querier-oauth-cookie". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.156866 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r848dc1b49b1745e58eaedaaaf37000f7.scope: Consumed 947us CPU time Nov 16 15:11:20.162389 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.162346 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy-rules" (OuterVolumeSpecName: "secret-thanos-querier-kube-rbac-proxy-rules") pod "a776019a-1ce1-4328-aed5-a5962a4f775d" (UID: "a776019a-1ce1-4328-aed5-a5962a4f775d"). InnerVolumeSpecName "secret-thanos-querier-kube-rbac-proxy-rules". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.167355 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.167312 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy-metrics" (OuterVolumeSpecName: "secret-thanos-querier-kube-rbac-proxy-metrics") pod "a776019a-1ce1-4328-aed5-a5962a4f775d" (UID: "a776019a-1ce1-4328-aed5-a5962a4f775d"). InnerVolumeSpecName "secret-thanos-querier-kube-rbac-proxy-metrics". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.176415 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.176348 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a776019a-1ce1-4328-aed5-a5962a4f775d" (UID: "a776019a-1ce1-4328-aed5-a5962a4f775d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.178204 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.178108600Z" level=info msg="Stopped pod sandbox: 0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a" id=226e3ae3-b5f4-45e7-85c0-0df245c5dfe9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.185345 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-4fce4c0d153d19a304d3bd797391d039680aa362400764583a8e0bffd82f1046-merged.mount: Succeeded. Nov 16 15:11:20.186220 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-4fce4c0d153d19a304d3bd797391d039680aa362400764583a8e0bffd82f1046-merged.mount: Consumed 0 CPU time Nov 16 15:11:20.188454 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.188405 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-tls" (OuterVolumeSpecName: "secret-thanos-querier-tls") pod "a776019a-1ce1-4328-aed5-a5962a4f775d" (UID: "a776019a-1ce1-4328-aed5-a5962a4f775d"). InnerVolumeSpecName "secret-thanos-querier-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.197915 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.197884 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-1_3670081b-e385-4685-8dda-b3a3ee01f233/prometheus-proxy/0.log" Nov 16 15:11:20.198607 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.198582 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-1_3670081b-e385-4685-8dda-b3a3ee01f233/config-reloader/0.log" Nov 16 15:11:20.207912 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.207803 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-68667ff848-f92nw" event=&{ID:454b3a37-199c-4abc-a79a-8de93eab2423 Type:ContainerDied Data:5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3} Nov 16 15:11:20.215291 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.215262 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-1_3670081b-e385-4685-8dda-b3a3ee01f233/prometheus-proxy/0.log" Nov 16 15:11:20.216055 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.216031 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-1_3670081b-e385-4685-8dda-b3a3ee01f233/config-reloader/0.log" Nov 16 15:11:20.216719 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod454b3a37_199c_4abc_a79a_8de93eab2423.slice. Nov 16 15:11:20.216778 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod454b3a37_199c_4abc_a79a_8de93eab2423.slice: Consumed 7.773s CPU time Nov 16 15:11:20.216983 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.216813 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-1" event=&{ID:3670081b-e385-4685-8dda-b3a3ee01f233 Type:ContainerDied Data:0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a} Nov 16 15:11:20.220587 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.220561 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b6b7d7fd9-d95wm_a776019a-1ce1-4328-aed5-a5962a4f775d/oauth-proxy/0.log" Nov 16 15:11:20.221020 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.220996 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" event=&{ID:a776019a-1ce1-4328-aed5-a5962a4f775d Type:ContainerDied Data:4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c} Nov 16 15:11:20.222692 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.222602 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-config\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.222692 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.222692 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-serving-certs-ca-bundle\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.222829 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.222729 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-web-config\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.222829 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.222759 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-metrics-client-certs\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.222829 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.222789 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-k8s-db\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.222829 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.222815 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-kube-etcd-client-certs\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-kube-etcd-client-certs\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.222959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.222840 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-grpc-tls\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.222959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.222874 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-metrics-client-ca\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.222959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.222902 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-proxy\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-proxy\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.222959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.222932 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-tls\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.222959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.222960 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-kubelet-serving-ca-bundle\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.223118 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.222962 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/3670081b-e385-4685-8dda-b3a3ee01f233/volumes/kubernetes.io~configmap/configmap-serving-certs-ca-bundle: clearQuota called, but quotas disabled Nov 16 15:11:20.223118 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.223018 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnqkh\" (UniqueName: \"kubernetes.io/projected/3670081b-e385-4685-8dda-b3a3ee01f233-kube-api-access-vnqkh\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.223118 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.223045 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-kube-rbac-proxy\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.223118 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.223046 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/3670081b-e385-4685-8dda-b3a3ee01f233/volumes/kubernetes.io~empty-dir/prometheus-k8s-db: clearQuota called, but quotas disabled Nov 16 15:11:20.223118 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.223076 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-k8s-rulefiles-0\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.223118 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.223112 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3670081b-e385-4685-8dda-b3a3ee01f233-config-out\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.223339 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.223161 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.223339 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.223190 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-trusted-ca-bundle\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.223339 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.223199 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:20.223339 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.223234 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3670081b-e385-4685-8dda-b3a3ee01f233-tls-assets\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.223339 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.223263 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-metrics-client-ca\") pod \"3670081b-e385-4685-8dda-b3a3ee01f233\" (UID: \"3670081b-e385-4685-8dda-b3a3ee01f233\") " Nov 16 15:11:20.223581 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.223362 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/3670081b-e385-4685-8dda-b3a3ee01f233/volumes/kubernetes.io~configmap/configmap-kubelet-serving-ca-bundle: clearQuota called, but quotas disabled Nov 16 15:11:20.223581 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.223385 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/3670081b-e385-4685-8dda-b3a3ee01f233/volumes/kubernetes.io~empty-dir/config-out: clearQuota called, but quotas disabled Nov 16 15:11:20.223581 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.223488 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/3670081b-e385-4685-8dda-b3a3ee01f233/volumes/kubernetes.io~configmap/configmap-metrics-client-ca: clearQuota called, but quotas disabled Nov 16 15:11:20.223581 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.223539 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3670081b-e385-4685-8dda-b3a3ee01f233-config-out" (OuterVolumeSpecName: "config-out") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Nov 16 15:11:20.223581 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.223548 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:20.223736 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.223690 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:20.224257 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.224221 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/3670081b-e385-4685-8dda-b3a3ee01f233/volumes/kubernetes.io~configmap/prometheus-trusted-ca-bundle: clearQuota called, but quotas disabled Nov 16 15:11:20.224566 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.224407 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/3670081b-e385-4685-8dda-b3a3ee01f233/volumes/kubernetes.io~configmap/metrics-client-ca: clearQuota called, but quotas disabled Nov 16 15:11:20.224566 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.224487 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/3670081b-e385-4685-8dda-b3a3ee01f233/volumes/kubernetes.io~configmap/prometheus-k8s-rulefiles-0: clearQuota called, but quotas disabled Nov 16 15:11:20.224715 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.224590 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:20.224715 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.224603 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225540 1563 reconciler.go:300] "Volume detached for volume \"secret-thanos-querier-oauth-cookie\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-oauth-cookie\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225586 1563 reconciler.go:300] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3670081b-e385-4685-8dda-b3a3ee01f233-config-out\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225606 1563 reconciler.go:300] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-trusted-ca-bundle\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225632 1563 reconciler.go:300] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-metrics-client-ca\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225651 1563 reconciler.go:300] "Volume detached for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225677 1563 reconciler.go:300] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-serving-certs-ca-bundle\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225697 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-pdtt2\" (UniqueName: \"kubernetes.io/projected/a776019a-1ce1-4328-aed5-a5962a4f775d-kube-api-access-pdtt2\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225724 1563 reconciler.go:300] "Volume detached for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy-metrics\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225742 1563 reconciler.go:300] "Volume detached for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-kube-rbac-proxy-rules\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225758 1563 reconciler.go:300] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-metrics-client-ca\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225774 1563 reconciler.go:300] "Volume detached for volume \"thanos-querier-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a776019a-1ce1-4328-aed5-a5962a4f775d-thanos-querier-trusted-ca-bundle\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225790 1563 reconciler.go:300] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-grpc-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225791 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225807 1563 reconciler.go:300] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-configmap-kubelet-serving-ca-bundle\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.226906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.225849 1563 reconciler.go:300] "Volume detached for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a776019a-1ce1-4328-aed5-a5962a4f775d-secret-thanos-querier-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.230238 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.230196 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:20.231904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.231800 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-config" (OuterVolumeSpecName: "config") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.237794 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.237735 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3670081b-e385-4685-8dda-b3a3ee01f233-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:20.237886 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.237869 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.237977 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.237944 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-proxy" (OuterVolumeSpecName: "secret-prometheus-k8s-proxy") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "secret-prometheus-k8s-proxy". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.240649 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.240577 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.240649 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.240614 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-kube-etcd-client-certs" (OuterVolumeSpecName: "secret-kube-etcd-client-certs") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "secret-kube-etcd-client-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.240794 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.240684 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3670081b-e385-4685-8dda-b3a3ee01f233-kube-api-access-vnqkh" (OuterVolumeSpecName: "kube-api-access-vnqkh") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "kube-api-access-vnqkh". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:20.244750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.244654 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.244750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.244724 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.246618 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.246580 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.247003 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-poda776019a_1ce1_4328_aed5_a5962a4f775d.slice. Nov 16 15:11:20.247208 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-poda776019a_1ce1_4328_aed5_a5962a4f775d.slice: Consumed 7.065s CPU time Nov 16 15:11:20.249713 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.249674 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/kube-state-metrics-68667ff848-f92nw] Nov 16 15:11:20.252904 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.252831 1563 mount_helper_common.go:133] Warning: "/var/lib/kubelet/pods/3670081b-e385-4685-8dda-b3a3ee01f233/volume-subpaths/web-config/prometheus/5" is not a mountpoint, deleting Nov 16 15:11:20.261770 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.261669 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-web-config" (OuterVolumeSpecName: "web-config") pod "3670081b-e385-4685-8dda-b3a3ee01f233" (UID: "3670081b-e385-4685-8dda-b3a3ee01f233"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.269322 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.269291 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-monitoring/kube-state-metrics-68667ff848-f92nw] Nov 16 15:11:20.326489 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.326359 1563 reconciler.go:300] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-config\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.326489 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.326455 1563 reconciler.go:300] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-web-config\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.326489 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.326478 1563 reconciler.go:300] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-metrics-client-certs\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.326489 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.326494 1563 reconciler.go:300] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-k8s-db\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.326489 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.326512 1563 reconciler.go:300] "Volume detached for volume \"secret-kube-etcd-client-certs\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-kube-etcd-client-certs\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.326884 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.326527 1563 reconciler.go:300] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-grpc-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.326884 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.326544 1563 reconciler.go:300] "Volume detached for volume \"secret-prometheus-k8s-proxy\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-proxy\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.326884 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.326560 1563 reconciler.go:300] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.326884 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.326576 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-vnqkh\" (UniqueName: \"kubernetes.io/projected/3670081b-e385-4685-8dda-b3a3ee01f233-kube-api-access-vnqkh\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.326884 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.326591 1563 reconciler.go:300] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-kube-rbac-proxy\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.326884 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.326607 1563 reconciler.go:300] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3670081b-e385-4685-8dda-b3a3ee01f233-prometheus-k8s-rulefiles-0\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.326884 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.326625 1563 reconciler.go:300] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3670081b-e385-4685-8dda-b3a3ee01f233-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.326884 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.326640 1563 reconciler.go:300] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3670081b-e385-4685-8dda-b3a3ee01f233-tls-assets\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.352218 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.352141 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm] Nov 16 15:11:20.361923 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.361851 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm] Nov 16 15:11:20.486344 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.486204090Z" level=info msg="Stopped pod sandbox: c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844" id=e1321e69-8448-4998-b7e6-53c150b3bd1f name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.510474 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.510402 1563 patch_prober.go:29] interesting pod/thanos-querier-5b6b7d7fd9-d95wm container/oauth-proxy namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.129.2.33:9091/-/ready\": context deadline exceeded" start-of-body= Nov 16 15:11:20.510684 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.510508 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm" podUID=a776019a-1ce1-4328-aed5-a5962a4f775d containerName="oauth-proxy" probeResult=failure output="Get \"https://10.129.2.33:9091/-/ready\": context deadline exceeded" Nov 16 15:11:20.527637 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.527557 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/35f6c830-e480-446f-a5cc-a6aaf007ea9a-tls-certificates\") pod \"35f6c830-e480-446f-a5cc-a6aaf007ea9a\" (UID: \"35f6c830-e480-446f-a5cc-a6aaf007ea9a\") " Nov 16 15:11:20.538665 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod3670081b_e385_4685_8dda_b3a3ee01f233.slice. Nov 16 15:11:20.538737 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod3670081b_e385_4685_8dda_b3a3ee01f233.slice: Consumed 43.683s CPU time Nov 16 15:11:20.547408 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.547369 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35f6c830-e480-446f-a5cc-a6aaf007ea9a-tls-certificates" (OuterVolumeSpecName: "tls-certificates") pod "35f6c830-e480-446f-a5cc-a6aaf007ea9a" (UID: "35f6c830-e480-446f-a5cc-a6aaf007ea9a"). InnerVolumeSpecName "tls-certificates". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.563810 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.563775 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/prometheus-k8s-1] Nov 16 15:11:20.570027 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.569963 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-monitoring/prometheus-k8s-1] Nov 16 15:11:20.592613 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592459049Z" level=info msg="Stopping pod sandbox: 4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c" id=a50444a6-734b-4a04-9037-d304289b8157 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.592613 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592564149Z" level=info msg="Stopping pod sandbox: 9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9" id=876c310f-2ae6-4ac3-b2db-fcf413b29f37 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.592613 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592589350Z" level=info msg="Stopping pod sandbox: 5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3" id=17573f03-2803-4f14-8584-a0a4a6b79467 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.593070 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592622450Z" level=info msg="Stopped pod sandbox (already stopped): 9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9" id=876c310f-2ae6-4ac3-b2db-fcf413b29f37 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.593070 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592620950Z" level=info msg="Stopping pod sandbox: 0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7" id=0bd3f4f2-7075-4b83-891e-6bc0abaaeeb4 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.593070 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592588950Z" level=info msg="Stopped pod sandbox (already stopped): 4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c" id=a50444a6-734b-4a04-9037-d304289b8157 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.593070 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592633550Z" level=info msg="Stopped pod sandbox (already stopped): 5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3" id=17573f03-2803-4f14-8584-a0a4a6b79467 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.593070 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592461149Z" level=info msg="Stopping pod sandbox: 5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78" id=b0e60779-cdf0-484d-aaa0-28318ddbd352 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.593070 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592628950Z" level=info msg="Stopping pod sandbox: 0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a" id=c4eeb097-9556-4af9-a530-64e9bb6c755f name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.593070 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592647450Z" level=info msg="Stopping pod sandbox: 6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d" id=97d89f99-1643-4bd4-9722-324aff71e948 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.593070 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592745051Z" level=info msg="Stopped pod sandbox (already stopped): 6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d" id=97d89f99-1643-4bd4-9722-324aff71e948 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.593345 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592731551Z" level=info msg="Stopped pod sandbox (already stopped): 0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a" id=c4eeb097-9556-4af9-a530-64e9bb6c755f name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.593345 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592661450Z" level=info msg="Stopped pod sandbox (already stopped): 0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7" id=0bd3f4f2-7075-4b83-891e-6bc0abaaeeb4 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.593345 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.592720651Z" level=info msg="Stopped pod sandbox (already stopped): 5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78" id=b0e60779-cdf0-484d-aaa0-28318ddbd352 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.594565 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.594488 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=3670081b-e385-4685-8dda-b3a3ee01f233 path="/var/lib/kubelet/pods/3670081b-e385-4685-8dda-b3a3ee01f233/volumes" Nov 16 15:11:20.595726 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.595625 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=454b3a37-199c-4abc-a79a-8de93eab2423 path="/var/lib/kubelet/pods/454b3a37-199c-4abc-a79a-8de93eab2423/volumes" Nov 16 15:11:20.596612 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.596543 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=4e58fe69-e225-425e-a6ca-0a7f136dbdca path="/var/lib/kubelet/pods/4e58fe69-e225-425e-a6ca-0a7f136dbdca/volumes" Nov 16 15:11:20.597078 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.597050 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=5eb5be5f-46bd-425a-a486-e2641bb0ad88 path="/var/lib/kubelet/pods/5eb5be5f-46bd-425a-a486-e2641bb0ad88/volumes" Nov 16 15:11:20.597746 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.597688 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=9502a499-9f8b-4a15-99d8-714a0cc37bea path="/var/lib/kubelet/pods/9502a499-9f8b-4a15-99d8-714a0cc37bea/volumes" Nov 16 15:11:20.598507 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.598478 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=a776019a-1ce1-4328-aed5-a5962a4f775d path="/var/lib/kubelet/pods/a776019a-1ce1-4328-aed5-a5962a4f775d/volumes" Nov 16 15:11:20.599441 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.599406 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd path="/var/lib/kubelet/pods/b6d86f0b-e1d0-4ffe-b9c6-85f4a93f08dd/volumes" Nov 16 15:11:20.609061 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod35f6c830_e480_446f_a5cc_a6aaf007ea9a.slice. Nov 16 15:11:20.609118 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod35f6c830_e480_446f_a5cc_a6aaf007ea9a.slice: Consumed 2.250s CPU time Nov 16 15:11:20.628180 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.628114 1563 reconciler.go:300] "Volume detached for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/35f6c830-e480-446f-a5cc-a6aaf007ea9a-tls-certificates\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.783928 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.783715794Z" level=info msg="Stopped pod sandbox: a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe" id=1730505e-df20-4a2a-bff6-81aec2c88c4b name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.792347 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.792318 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a5ccca-d055-4b4c-820e-515fbf3f92d5/alertmanager-proxy/0.log" Nov 16 15:11:20.792835 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.792747 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a5ccca-d055-4b4c-820e-515fbf3f92d5/config-reloader/0.log" Nov 16 15:11:20.828381 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volume\x2dsubpaths-web\x2dconfig-prometheus-5.mount: Succeeded. Nov 16 15:11:20.829059 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.829011 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-config-volume\") pod \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " Nov 16 15:11:20.829233 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.829110 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7lr9\" (UniqueName: \"kubernetes.io/projected/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-kube-api-access-n7lr9\") pod \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " Nov 16 15:11:20.829233 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.829211 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-metrics-client-ca\") pod \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " Nov 16 15:11:20.829308 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.829246 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-main-tls\") pod \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " Nov 16 15:11:20.829308 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.829272 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-tls-assets\") pod \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " Nov 16 15:11:20.829413 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.829323 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-alertmanager-trusted-ca-bundle\") pod \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " Nov 16 15:11:20.829413 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.829393 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-main-proxy\") pod \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " Nov 16 15:11:20.829511 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.829464 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " Nov 16 15:11:20.829511 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.829491 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-kube-rbac-proxy\") pod \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " Nov 16 15:11:20.829465 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volume\x2dsubpaths-web\x2dconfig-prometheus-5.mount: Consumed 0 CPU time Nov 16 15:11:20.829650 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.829520 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-alertmanager-main-db\") pod \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\" (UID: \"e0a5ccca-d055-4b4c-820e-515fbf3f92d5\") " Nov 16 15:11:20.829650 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.829558 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/e0a5ccca-d055-4b4c-820e-515fbf3f92d5/volumes/kubernetes.io~configmap/metrics-client-ca: clearQuota called, but quotas disabled Nov 16 15:11:20.829480 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dvnqkh.mount: Succeeded. Nov 16 15:11:20.829811 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.829786 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "e0a5ccca-d055-4b4c-820e-515fbf3f92d5" (UID: "e0a5ccca-d055-4b4c-820e-515fbf3f92d5"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:20.829851 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.829815 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/e0a5ccca-d055-4b4c-820e-515fbf3f92d5/volumes/kubernetes.io~empty-dir/alertmanager-main-db: clearQuota called, but quotas disabled Nov 16 15:11:20.829955 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:20.829905 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/e0a5ccca-d055-4b4c-820e-515fbf3f92d5/volumes/kubernetes.io~configmap/alertmanager-trusted-ca-bundle: clearQuota called, but quotas disabled Nov 16 15:11:20.830009 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.829961 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "e0a5ccca-d055-4b4c-820e-515fbf3f92d5" (UID: "e0a5ccca-d055-4b4c-820e-515fbf3f92d5"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Nov 16 15:11:20.830201 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.830166 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "e0a5ccca-d055-4b4c-820e-515fbf3f92d5" (UID: "e0a5ccca-d055-4b4c-820e-515fbf3f92d5"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:20.830189 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dvnqkh.mount: Consumed 0 CPU time Nov 16 15:11:20.830201 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dprometheus\x2dk8s\x2dthanos\x2dsidecar\x2dtls.mount: Succeeded. Nov 16 15:11:20.831048 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dprometheus\x2dk8s\x2dthanos\x2dsidecar\x2dtls.mount: Consumed 0 CPU time Nov 16 15:11:20.831081 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7eprojected-tls\x2dassets.mount: Succeeded. Nov 16 15:11:20.831798 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7eprojected-tls\x2dassets.mount: Consumed 0 CPU time Nov 16 15:11:20.831809 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dkube\x2detcd\x2dclient\x2dcerts.mount: Succeeded. Nov 16 15:11:20.832671 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dkube\x2detcd\x2dclient\x2dcerts.mount: Consumed 0 CPU time Nov 16 15:11:20.832685 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dprometheus\x2dk8s\x2dtls.mount: Succeeded. Nov 16 15:11:20.833398 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dprometheus\x2dk8s\x2dtls.mount: Consumed 0 CPU time Nov 16 15:11:20.833412 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-config.mount: Succeeded. Nov 16 15:11:20.834043 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-config.mount: Consumed 0 CPU time Nov 16 15:11:20.834056 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dmetrics\x2dclient\x2dcerts.mount: Succeeded. Nov 16 15:11:20.834752 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dmetrics\x2dclient\x2dcerts.mount: Consumed 0 CPU time Nov 16 15:11:20.834765 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dprometheus\x2dk8s\x2dproxy.mount: Succeeded. Nov 16 15:11:20.835451 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dprometheus\x2dk8s\x2dproxy.mount: Consumed 0 CPU time Nov 16 15:11:20.835465 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dgrpc\x2dtls.mount: Succeeded. Nov 16 15:11:20.836085 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dgrpc\x2dtls.mount: Consumed 0 CPU time Nov 16 15:11:20.836097 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dkube\x2drbac\x2dproxy.mount: Succeeded. Nov 16 15:11:20.836842 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-secret\x2dkube\x2drbac\x2dproxy.mount: Consumed 0 CPU time Nov 16 15:11:20.836855 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-web\x2dconfig.mount: Succeeded. Nov 16 15:11:20.837582 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-3670081b\x2de385\x2d4685\x2d8dda\x2db3a3ee01f233-volumes-kubernetes.io\x7esecret-web\x2dconfig.mount: Consumed 0 CPU time Nov 16 15:11:20.837628 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-955d20ab3ad2bd242fa3069c275a5042deab6422702eed715be2277eb0537e61-merged.mount: Succeeded. Nov 16 15:11:20.838333 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-955d20ab3ad2bd242fa3069c275a5042deab6422702eed715be2277eb0537e61-merged.mount: Consumed 0 CPU time Nov 16 15:11:20.838368 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-167aec1b7eb1792f5d9afd0622d683fe853c8338f259792361dd241c2a1ded49-merged.mount: Succeeded. Nov 16 15:11:20.839063 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-167aec1b7eb1792f5d9afd0622d683fe853c8338f259792361dd241c2a1ded49-merged.mount: Consumed 0 CPU time Nov 16 15:11:20.839082 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-35f6c830\x2de480\x2d446f\x2da5cc\x2da6aaf007ea9a-volumes-kubernetes.io\x7esecret-tls\x2dcertificates.mount: Succeeded. Nov 16 15:11:20.839816 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-35f6c830\x2de480\x2d446f\x2da5cc\x2da6aaf007ea9a-volumes-kubernetes.io\x7esecret-tls\x2dcertificates.mount: Consumed 0 CPU time Nov 16 15:11:20.848403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.848366 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-kube-api-access-n7lr9" (OuterVolumeSpecName: "kube-api-access-n7lr9") pod "e0a5ccca-d055-4b4c-820e-515fbf3f92d5" (UID: "e0a5ccca-d055-4b4c-820e-515fbf3f92d5"). InnerVolumeSpecName "kube-api-access-n7lr9". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:20.849857 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dn7lr9.mount: Succeeded. Nov 16 15:11:20.850666 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dn7lr9.mount: Consumed 0 CPU time Nov 16 15:11:20.850683 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7eprojected-tls\x2dassets.mount: Succeeded. Nov 16 15:11:20.851522 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7eprojected-tls\x2dassets.mount: Consumed 0 CPU time Nov 16 15:11:20.851542 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dmain\x2dproxy.mount: Succeeded. Nov 16 15:11:20.852342 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dmain\x2dproxy.mount: Consumed 0 CPU time Nov 16 15:11:20.852358 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7esecret-config\x2dvolume.mount: Succeeded. Nov 16 15:11:20.853049 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7esecret-config\x2dvolume.mount: Consumed 0 CPU time Nov 16 15:11:20.853064 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dmain\x2dtls.mount: Succeeded. Nov 16 15:11:20.853772 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dmain\x2dtls.mount: Consumed 0 CPU time Nov 16 15:11:20.853786 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dkube\x2drbac\x2dproxy.mount: Succeeded. Nov 16 15:11:20.854479 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dkube\x2drbac\x2dproxy.mount: Consumed 0 CPU time Nov 16 15:11:20.854494 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dkube\x2drbac\x2dproxy\x2dmetric.mount: Succeeded. Nov 16 15:11:20.855115 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-e0a5ccca\x2dd055\x2d4b4c\x2d820e\x2d515fbf3f92d5-volumes-kubernetes.io\x7esecret-secret\x2dalertmanager\x2dkube\x2drbac\x2dproxy\x2dmetric.mount: Consumed 0 CPU time Nov 16 15:11:20.856395 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.856356 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "e0a5ccca-d055-4b4c-820e-515fbf3f92d5" (UID: "e0a5ccca-d055-4b4c-820e-515fbf3f92d5"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.860662 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.860581 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e0a5ccca-d055-4b4c-820e-515fbf3f92d5" (UID: "e0a5ccca-d055-4b4c-820e-515fbf3f92d5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:20.863591 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.863462 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-config-volume" (OuterVolumeSpecName: "config-volume") pod "e0a5ccca-d055-4b4c-820e-515fbf3f92d5" (UID: "e0a5ccca-d055-4b4c-820e-515fbf3f92d5"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.872450 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.872384 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-main-proxy" (OuterVolumeSpecName: "secret-alertmanager-main-proxy") pod "e0a5ccca-d055-4b4c-820e-515fbf3f92d5" (UID: "e0a5ccca-d055-4b4c-820e-515fbf3f92d5"). InnerVolumeSpecName "secret-alertmanager-main-proxy". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.882553 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.882430 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "e0a5ccca-d055-4b4c-820e-515fbf3f92d5" (UID: "e0a5ccca-d055-4b4c-820e-515fbf3f92d5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.896517 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.896428 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "e0a5ccca-d055-4b4c-820e-515fbf3f92d5" (UID: "e0a5ccca-d055-4b4c-820e-515fbf3f92d5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:20.929932 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.929857 1563 reconciler.go:300] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-config-volume\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.929932 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.929912 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-n7lr9\" (UniqueName: \"kubernetes.io/projected/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-kube-api-access-n7lr9\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.929932 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.929929 1563 reconciler.go:300] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-metrics-client-ca\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.930122 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.929950 1563 reconciler.go:300] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-main-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.930122 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.929981 1563 reconciler.go:300] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-tls-assets\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.930122 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.930026 1563 reconciler.go:300] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-alertmanager-trusted-ca-bundle\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.930122 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.930042 1563 reconciler.go:300] "Volume detached for volume \"secret-alertmanager-main-proxy\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-main-proxy\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.930122 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.930059 1563 reconciler.go:300] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.930122 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.930074 1563 reconciler.go:300] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-secret-alertmanager-kube-rbac-proxy\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.930122 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:20.930089 1563 reconciler.go:300] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e0a5ccca-d055-4b4c-820e-515fbf3f92d5-alertmanager-main-db\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:20.971976 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:20.971917115Z" level=info msg="Stopped pod sandbox: a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145" id=db6290ff-ada1-4e20-9f50-8724d61aa2e5 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:20.978693 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-02478a52b938335ecc8b2f4fe8b6b4a47985c1adc1d3a130664aeb0f66d79fb0-merged.mount: Succeeded. Nov 16 15:11:20.979672 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-02478a52b938335ecc8b2f4fe8b6b4a47985c1adc1d3a130664aeb0f66d79fb0-merged.mount: Consumed 0 CPU time Nov 16 15:11:21.131372 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.131327 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-metrics-client-ca\") pod \"add06140-663d-48c0-ac00-643bd58a3f7d\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " Nov 16 15:11:21.131372 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.131395 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-telemeter-trusted-ca-bundle\") pod \"add06140-663d-48c0-ac00-643bd58a3f7d\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " Nov 16 15:11:21.132015 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.131462 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"add06140-663d-48c0-ac00-643bd58a3f7d\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " Nov 16 15:11:21.132015 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.131498 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-serving-certs-ca-bundle\") pod \"add06140-663d-48c0-ac00-643bd58a3f7d\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " Nov 16 15:11:21.132015 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.131533 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-secret-telemeter-client\") pod \"add06140-663d-48c0-ac00-643bd58a3f7d\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " Nov 16 15:11:21.132015 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.131571 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-telemeter-client-tls\") pod \"add06140-663d-48c0-ac00-643bd58a3f7d\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " Nov 16 15:11:21.132015 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.131602 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-547nc\" (UniqueName: \"kubernetes.io/projected/add06140-663d-48c0-ac00-643bd58a3f7d-kube-api-access-547nc\") pod \"add06140-663d-48c0-ac00-643bd58a3f7d\" (UID: \"add06140-663d-48c0-ac00-643bd58a3f7d\") " Nov 16 15:11:21.132015 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:21.131720 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/add06140-663d-48c0-ac00-643bd58a3f7d/volumes/kubernetes.io~configmap/metrics-client-ca: clearQuota called, but quotas disabled Nov 16 15:11:21.132015 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:21.131726 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/add06140-663d-48c0-ac00-643bd58a3f7d/volumes/kubernetes.io~configmap/telemeter-trusted-ca-bundle: clearQuota called, but quotas disabled Nov 16 15:11:21.132015 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:21.131794 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/add06140-663d-48c0-ac00-643bd58a3f7d/volumes/kubernetes.io~configmap/serving-certs-ca-bundle: clearQuota called, but quotas disabled Nov 16 15:11:21.132015 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.131955 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "add06140-663d-48c0-ac00-643bd58a3f7d" (UID: "add06140-663d-48c0-ac00-643bd58a3f7d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:21.132015 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.131997 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-telemeter-trusted-ca-bundle" (OuterVolumeSpecName: "telemeter-trusted-ca-bundle") pod "add06140-663d-48c0-ac00-643bd58a3f7d" (UID: "add06140-663d-48c0-ac00-643bd58a3f7d"). InnerVolumeSpecName "telemeter-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:21.132434 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.132037 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-serving-certs-ca-bundle" (OuterVolumeSpecName: "serving-certs-ca-bundle") pod "add06140-663d-48c0-ac00-643bd58a3f7d" (UID: "add06140-663d-48c0-ac00-643bd58a3f7d"). InnerVolumeSpecName "serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:21.140506 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-add06140\x2d663d\x2d48c0\x2dac00\x2d643bd58a3f7d-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d547nc.mount: Succeeded. Nov 16 15:11:21.141376 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-add06140\x2d663d\x2d48c0\x2dac00\x2d643bd58a3f7d-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d547nc.mount: Consumed 0 CPU time Nov 16 15:11:21.141398 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-add06140\x2d663d\x2d48c0\x2dac00\x2d643bd58a3f7d-volumes-kubernetes.io\x7esecret-secret\x2dtelemeter\x2dclient\x2dkube\x2drbac\x2dproxy\x2dconfig.mount: Succeeded. Nov 16 15:11:21.142083 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-add06140\x2d663d\x2d48c0\x2dac00\x2d643bd58a3f7d-volumes-kubernetes.io\x7esecret-secret\x2dtelemeter\x2dclient\x2dkube\x2drbac\x2dproxy\x2dconfig.mount: Consumed 0 CPU time Nov 16 15:11:21.142099 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-add06140\x2d663d\x2d48c0\x2dac00\x2d643bd58a3f7d-volumes-kubernetes.io\x7esecret-telemeter\x2dclient\x2dtls.mount: Succeeded. Nov 16 15:11:21.142792 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-add06140\x2d663d\x2d48c0\x2dac00\x2d643bd58a3f7d-volumes-kubernetes.io\x7esecret-telemeter\x2dclient\x2dtls.mount: Consumed 0 CPU time Nov 16 15:11:21.142809 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-add06140\x2d663d\x2d48c0\x2dac00\x2d643bd58a3f7d-volumes-kubernetes.io\x7esecret-secret\x2dtelemeter\x2dclient.mount: Succeeded. Nov 16 15:11:21.143519 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-add06140\x2d663d\x2d48c0\x2dac00\x2d643bd58a3f7d-volumes-kubernetes.io\x7esecret-secret\x2dtelemeter\x2dclient.mount: Consumed 0 CPU time Nov 16 15:11:21.149459 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.149388 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-secret-telemeter-client" (OuterVolumeSpecName: "secret-telemeter-client") pod "add06140-663d-48c0-ac00-643bd58a3f7d" (UID: "add06140-663d-48c0-ac00-643bd58a3f7d"). InnerVolumeSpecName "secret-telemeter-client". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:21.149459 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.149420 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/add06140-663d-48c0-ac00-643bd58a3f7d-kube-api-access-547nc" (OuterVolumeSpecName: "kube-api-access-547nc") pod "add06140-663d-48c0-ac00-643bd58a3f7d" (UID: "add06140-663d-48c0-ac00-643bd58a3f7d"). InnerVolumeSpecName "kube-api-access-547nc". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:21.157392 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.157339 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-secret-telemeter-client-kube-rbac-proxy-config" (OuterVolumeSpecName: "secret-telemeter-client-kube-rbac-proxy-config") pod "add06140-663d-48c0-ac00-643bd58a3f7d" (UID: "add06140-663d-48c0-ac00-643bd58a3f7d"). InnerVolumeSpecName "secret-telemeter-client-kube-rbac-proxy-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:21.165388 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.165348 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-telemeter-client-tls" (OuterVolumeSpecName: "telemeter-client-tls") pod "add06140-663d-48c0-ac00-643bd58a3f7d" (UID: "add06140-663d-48c0-ac00-643bd58a3f7d"). InnerVolumeSpecName "telemeter-client-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:21.226458 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.226363 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj" event=&{ID:35f6c830-e480-446f-a5cc-a6aaf007ea9a Type:ContainerDied Data:c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844} Nov 16 15:11:21.228315 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.228274 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c5556665-7vs77" event=&{ID:add06140-663d-48c0-ac00-643bd58a3f7d Type:ContainerDied Data:a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145} Nov 16 15:11:21.231905 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.231801 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a5ccca-d055-4b4c-820e-515fbf3f92d5/alertmanager-proxy/0.log" Nov 16 15:11:21.233263 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.233190 1563 reconciler.go:300] "Volume detached for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-secret-telemeter-client-kube-rbac-proxy-config\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.233355 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.233307 1563 reconciler.go:300] "Volume detached for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-serving-certs-ca-bundle\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.233355 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.233334 1563 reconciler.go:300] "Volume detached for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-secret-telemeter-client\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.233433 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.233398 1563 reconciler.go:300] "Volume detached for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/add06140-663d-48c0-ac00-643bd58a3f7d-telemeter-client-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.233433 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.233425 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-547nc\" (UniqueName: \"kubernetes.io/projected/add06140-663d-48c0-ac00-643bd58a3f7d-kube-api-access-547nc\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.233515 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.233440 1563 reconciler.go:300] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-metrics-client-ca\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.233554 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.233501 1563 reconciler.go:300] "Volume detached for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add06140-663d-48c0-ac00-643bd58a3f7d-telemeter-trusted-ca-bundle\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.234223 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.234119 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a5ccca-d055-4b4c-820e-515fbf3f92d5/config-reloader/0.log" Nov 16 15:11:21.235016 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.234988 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event=&{ID:e0a5ccca-d055-4b4c-820e-515fbf3f92d5 Type:ContainerDied Data:a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe} Nov 16 15:11:21.237785 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-podadd06140_663d_48c0_ac00_643bd58a3f7d.slice. Nov 16 15:11:21.237863 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-podadd06140_663d_48c0_ac00_643bd58a3f7d.slice: Consumed 1.615s CPU time Nov 16 15:11:21.266785 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pode0a5ccca_d055_4b4c_820e_515fbf3f92d5.slice. Nov 16 15:11:21.266862 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pode0a5ccca_d055_4b4c_820e_515fbf3f92d5.slice: Consumed 7.317s CPU time Nov 16 15:11:21.274471 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.274118 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj] Nov 16 15:11:21.283658 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.283564 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj] Nov 16 15:11:21.321625 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.321550 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/telemeter-client-55c5556665-7vs77] Nov 16 15:11:21.332008 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.331936 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-monitoring/telemeter-client-55c5556665-7vs77] Nov 16 15:11:21.377968 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.377891 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 15:11:21.385632 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.385444757Z" level=info msg="Stopped pod sandbox: 9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c" id=b59bac2c-a9bf-48e7-aa54-ceb84db7d308 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:21.388957 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.388929 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-monitoring/alertmanager-main-0] Nov 16 15:11:21.434765 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.434726 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8013b4f-15b6-4bac-bfc9-a45301362f0c-openshift-state-metrics-tls\") pod \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\" (UID: \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\") " Nov 16 15:11:21.434987 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.434804 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8z4g\" (UniqueName: \"kubernetes.io/projected/c8013b4f-15b6-4bac-bfc9-a45301362f0c-kube-api-access-b8z4g\") pod \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\" (UID: \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\") " Nov 16 15:11:21.434987 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.434843 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8013b4f-15b6-4bac-bfc9-a45301362f0c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\" (UID: \"c8013b4f-15b6-4bac-bfc9-a45301362f0c\") " Nov 16 15:11:21.452383 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.452342 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8013b4f-15b6-4bac-bfc9-a45301362f0c-openshift-state-metrics-kube-rbac-proxy-config" (OuterVolumeSpecName: "openshift-state-metrics-kube-rbac-proxy-config") pod "c8013b4f-15b6-4bac-bfc9-a45301362f0c" (UID: "c8013b4f-15b6-4bac-bfc9-a45301362f0c"). InnerVolumeSpecName "openshift-state-metrics-kube-rbac-proxy-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:21.452485 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.452395 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8013b4f-15b6-4bac-bfc9-a45301362f0c-kube-api-access-b8z4g" (OuterVolumeSpecName: "kube-api-access-b8z4g") pod "c8013b4f-15b6-4bac-bfc9-a45301362f0c" (UID: "c8013b4f-15b6-4bac-bfc9-a45301362f0c"). InnerVolumeSpecName "kube-api-access-b8z4g". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:21.464392 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.464317 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8013b4f-15b6-4bac-bfc9-a45301362f0c-openshift-state-metrics-tls" (OuterVolumeSpecName: "openshift-state-metrics-tls") pod "c8013b4f-15b6-4bac-bfc9-a45301362f0c" (UID: "c8013b4f-15b6-4bac-bfc9-a45301362f0c"). InnerVolumeSpecName "openshift-state-metrics-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:21.536163 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.536032 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-b8z4g\" (UniqueName: \"kubernetes.io/projected/c8013b4f-15b6-4bac-bfc9-a45301362f0c-kube-api-access-b8z4g\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.536163 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.536087 1563 reconciler.go:300] "Volume detached for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8013b4f-15b6-4bac-bfc9-a45301362f0c-openshift-state-metrics-kube-rbac-proxy-config\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.536163 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.536108 1563 reconciler.go:300] "Volume detached for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8013b4f-15b6-4bac-bfc9-a45301362f0c-openshift-state-metrics-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.631083 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.630995241Z" level=info msg="Stopped pod sandbox: 59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f" id=8941aaf1-1935-4a33-a42a-d98caf8785be name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:21.723706 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.723534189Z" level=info msg="Stopped pod sandbox: 9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486" id=200564d4-a53d-41c4-be80-886e9a728819 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:21.736984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.736917 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-config\") pod \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " Nov 16 15:11:21.736984 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.736967 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-audit-log\") pod \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " Nov 16 15:11:21.737252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737005 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-serving-certs-ca-bundle\") pod \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " Nov 16 15:11:21.737252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737040 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-tls\") pod \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " Nov 16 15:11:21.737252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737075 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6skm9\" (UniqueName: \"kubernetes.io/projected/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-kube-api-access-6skm9\") pod \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " Nov 16 15:11:21.737252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737108 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"prometheus-adapter-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-prometheus-adapter-audit-profiles\") pod \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " Nov 16 15:11:21.737252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737173 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"tmpfs\" (UniqueName: \"kubernetes.io/empty-dir/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-tmpfs\") pod \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " Nov 16 15:11:21.737252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737220 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"elasticsearch-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/220ce491-f766-41ee-ab02-aa1546eb7ed3-elasticsearch-operator-metrics-cert\") pod \"220ce491-f766-41ee-ab02-aa1546eb7ed3\" (UID: \"220ce491-f766-41ee-ab02-aa1546eb7ed3\") " Nov 16 15:11:21.737480 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:21.737222 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/17691c72-c4d7-4e7f-b345-a0447ae7d4c6/volumes/kubernetes.io~configmap/config: clearQuota called, but quotas disabled Nov 16 15:11:21.737480 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:21.737276 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/17691c72-c4d7-4e7f-b345-a0447ae7d4c6/volumes/kubernetes.io~empty-dir/audit-log: clearQuota called, but quotas disabled Nov 16 15:11:21.737480 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:21.737381 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/17691c72-c4d7-4e7f-b345-a0447ae7d4c6/volumes/kubernetes.io~configmap/serving-certs-ca-bundle: clearQuota called, but quotas disabled Nov 16 15:11:21.737480 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:21.737454 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/17691c72-c4d7-4e7f-b345-a0447ae7d4c6/volumes/kubernetes.io~configmap/prometheus-adapter-audit-profiles: clearQuota called, but quotas disabled Nov 16 15:11:21.737480 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737470 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-config" (OuterVolumeSpecName: "config") pod "17691c72-c4d7-4e7f-b345-a0447ae7d4c6" (UID: "17691c72-c4d7-4e7f-b345-a0447ae7d4c6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:21.737651 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:21.737486 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/17691c72-c4d7-4e7f-b345-a0447ae7d4c6/volumes/kubernetes.io~empty-dir/tmpfs: clearQuota called, but quotas disabled Nov 16 15:11:21.737651 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737262 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xj7t8\" (UniqueName: \"kubernetes.io/projected/220ce491-f766-41ee-ab02-aa1546eb7ed3-kube-api-access-xj7t8\") pod \"220ce491-f766-41ee-ab02-aa1546eb7ed3\" (UID: \"220ce491-f766-41ee-ab02-aa1546eb7ed3\") " Nov 16 15:11:21.737651 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737536 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-tmpfs" (OuterVolumeSpecName: "tmpfs") pod "17691c72-c4d7-4e7f-b345-a0447ae7d4c6" (UID: "17691c72-c4d7-4e7f-b345-a0447ae7d4c6"). InnerVolumeSpecName "tmpfs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Nov 16 15:11:21.737651 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737554 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-audit-log" (OuterVolumeSpecName: "audit-log") pod "17691c72-c4d7-4e7f-b345-a0447ae7d4c6" (UID: "17691c72-c4d7-4e7f-b345-a0447ae7d4c6"). InnerVolumeSpecName "audit-log". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Nov 16 15:11:21.737651 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737585 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"prometheus-adapter-prometheus-config\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-prometheus-adapter-prometheus-config\") pod \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\" (UID: \"17691c72-c4d7-4e7f-b345-a0447ae7d4c6\") " Nov 16 15:11:21.737824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737686 1563 reconciler.go:300] "Volume detached for volume \"tmpfs\" (UniqueName: \"kubernetes.io/empty-dir/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-tmpfs\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.737824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737706 1563 reconciler.go:300] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-config\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.737824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737574 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-serving-certs-ca-bundle" (OuterVolumeSpecName: "serving-certs-ca-bundle") pod "17691c72-c4d7-4e7f-b345-a0447ae7d4c6" (UID: "17691c72-c4d7-4e7f-b345-a0447ae7d4c6"). InnerVolumeSpecName "serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:21.737824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737722 1563 reconciler.go:300] "Volume detached for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-audit-log\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.737824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:21.737726 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/17691c72-c4d7-4e7f-b345-a0447ae7d4c6/volumes/kubernetes.io~configmap/prometheus-adapter-prometheus-config: clearQuota called, but quotas disabled Nov 16 15:11:21.737824 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737787 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-prometheus-adapter-audit-profiles" (OuterVolumeSpecName: "prometheus-adapter-audit-profiles") pod "17691c72-c4d7-4e7f-b345-a0447ae7d4c6" (UID: "17691c72-c4d7-4e7f-b345-a0447ae7d4c6"). InnerVolumeSpecName "prometheus-adapter-audit-profiles". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:21.738058 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.737927 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-prometheus-adapter-prometheus-config" (OuterVolumeSpecName: "prometheus-adapter-prometheus-config") pod "17691c72-c4d7-4e7f-b345-a0447ae7d4c6" (UID: "17691c72-c4d7-4e7f-b345-a0447ae7d4c6"). InnerVolumeSpecName "prometheus-adapter-prometheus-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:21.744376 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.744330857Z" level=info msg="Removed container ca695a9e04ca1bc959748b13616af6001f762fff366e780a715d9d18b569f454: openshift-marketplace/qe-app-registry-9hthz/registry-server" id=60ba4332-f17f-4470-b776-74acbf4136c6 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:21.744545 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.744515 1563 scope.go:110] "RemoveContainer" containerID="2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb" Nov 16 15:11:21.745596 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.745494066Z" level=info msg="Removing container: 2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb" id=2f4e691c-a978-4e6a-a654-f29388b7dc4f name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:21.756410 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.756357 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-kube-api-access-6skm9" (OuterVolumeSpecName: "kube-api-access-6skm9") pod "17691c72-c4d7-4e7f-b345-a0447ae7d4c6" (UID: "17691c72-c4d7-4e7f-b345-a0447ae7d4c6"). InnerVolumeSpecName "kube-api-access-6skm9". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:21.756538 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.756411 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/220ce491-f766-41ee-ab02-aa1546eb7ed3-kube-api-access-xj7t8" (OuterVolumeSpecName: "kube-api-access-xj7t8") pod "220ce491-f766-41ee-ab02-aa1546eb7ed3" (UID: "220ce491-f766-41ee-ab02-aa1546eb7ed3"). InnerVolumeSpecName "kube-api-access-xj7t8". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:21.758396 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.758354 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/220ce491-f766-41ee-ab02-aa1546eb7ed3-elasticsearch-operator-metrics-cert" (OuterVolumeSpecName: "elasticsearch-operator-metrics-cert") pod "220ce491-f766-41ee-ab02-aa1546eb7ed3" (UID: "220ce491-f766-41ee-ab02-aa1546eb7ed3"). InnerVolumeSpecName "elasticsearch-operator-metrics-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:21.758396 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.758376 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-tls" (OuterVolumeSpecName: "tls") pod "17691c72-c4d7-4e7f-b345-a0447ae7d4c6" (UID: "17691c72-c4d7-4e7f-b345-a0447ae7d4c6"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:21.781930 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.781847060Z" level=info msg="Removed container 2837cd7588b35c027748a496702696170cb0043735996eebca8f34e80f8d3dbb: openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv/kube-rbac-proxy" id=2f4e691c-a978-4e6a-a654-f29388b7dc4f name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:21.782148 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.782108 1563 scope.go:110] "RemoveContainer" containerID="8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4" Nov 16 15:11:21.783408 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.783375172Z" level=info msg="Removing container: 8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4" id=247b280e-7f5c-48af-852e-92d44612900a name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:21.815935 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.815838934Z" level=info msg="Removed container 8217a98ad0e7f7d99c11bd3afd65343378033b43abafe0f438d49e30b01eebe4: openshift-operators-redhat/loki-operator-controller-manager-67cc5866c9-m67tv/manager" id=247b280e-7f5c-48af-852e-92d44612900a name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:21.816179 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.816119 1563 scope.go:110] "RemoveContainer" containerID="7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d" Nov 16 15:11:21.817425 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.817385347Z" level=info msg="Removing container: 7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d" id=35faaea7-6a4c-4879-8126-bf0e1b513c0a name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:21.828852 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0f97558d6aaf19ef4c74b37369125505242a93c79a4b3922174fcef38d70c3a4-merged.mount: Succeeded. Nov 16 15:11:21.829738 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0f97558d6aaf19ef4c74b37369125505242a93c79a4b3922174fcef38d70c3a4-merged.mount: Consumed 0 CPU time Nov 16 15:11:21.829759 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-17691c72\x2dc4d7\x2d4e7f\x2db345\x2da0447ae7d4c6-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d6skm9.mount: Succeeded. Nov 16 15:11:21.830614 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-17691c72\x2dc4d7\x2d4e7f\x2db345\x2da0447ae7d4c6-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d6skm9.mount: Consumed 0 CPU time Nov 16 15:11:21.830632 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-17691c72\x2dc4d7\x2d4e7f\x2db345\x2da0447ae7d4c6-volumes-kubernetes.io\x7esecret-tls.mount: Succeeded. Nov 16 15:11:21.831297 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-17691c72\x2dc4d7\x2d4e7f\x2db345\x2da0447ae7d4c6-volumes-kubernetes.io\x7esecret-tls.mount: Consumed 0 CPU time Nov 16 15:11:21.831313 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-c8013b4f\x2d15b6\x2d4bac\x2dbfc9\x2da45301362f0c-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2db8z4g.mount: Succeeded. Nov 16 15:11:21.831952 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-c8013b4f\x2d15b6\x2d4bac\x2dbfc9\x2da45301362f0c-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2db8z4g.mount: Consumed 0 CPU time Nov 16 15:11:21.831967 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-c8013b4f\x2d15b6\x2d4bac\x2dbfc9\x2da45301362f0c-volumes-kubernetes.io\x7esecret-openshift\x2dstate\x2dmetrics\x2dtls.mount: Succeeded. Nov 16 15:11:21.832680 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-c8013b4f\x2d15b6\x2d4bac\x2dbfc9\x2da45301362f0c-volumes-kubernetes.io\x7esecret-openshift\x2dstate\x2dmetrics\x2dtls.mount: Consumed 0 CPU time Nov 16 15:11:21.832695 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-c8013b4f\x2d15b6\x2d4bac\x2dbfc9\x2da45301362f0c-volumes-kubernetes.io\x7esecret-openshift\x2dstate\x2dmetrics\x2dkube\x2drbac\x2dproxy\x2dconfig.mount: Succeeded. Nov 16 15:11:21.833546 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-c8013b4f\x2d15b6\x2d4bac\x2dbfc9\x2da45301362f0c-volumes-kubernetes.io\x7esecret-openshift\x2dstate\x2dmetrics\x2dkube\x2drbac\x2dproxy\x2dconfig.mount: Consumed 0 CPU time Nov 16 15:11:21.833564 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-a97a8854f1a563176f909abdf492cd4240b76976dfd4249f0ad26f4683821693-merged.mount: Succeeded. Nov 16 15:11:21.834314 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-a97a8854f1a563176f909abdf492cd4240b76976dfd4249f0ad26f4683821693-merged.mount: Consumed 0 CPU time Nov 16 15:11:21.834329 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-220ce491\x2df766\x2d41ee\x2dab02\x2daa1546eb7ed3-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dxj7t8.mount: Succeeded. Nov 16 15:11:21.834973 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-220ce491\x2df766\x2d41ee\x2dab02\x2daa1546eb7ed3-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dxj7t8.mount: Consumed 0 CPU time Nov 16 15:11:21.834987 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-220ce491\x2df766\x2d41ee\x2dab02\x2daa1546eb7ed3-volumes-kubernetes.io\x7esecret-elasticsearch\x2doperator\x2dmetrics\x2dcert.mount: Succeeded. Nov 16 15:11:21.835663 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-220ce491\x2df766\x2d41ee\x2dab02\x2daa1546eb7ed3-volumes-kubernetes.io\x7esecret-elasticsearch\x2doperator\x2dmetrics\x2dcert.mount: Consumed 0 CPU time Nov 16 15:11:21.838715 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.838642 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-6skm9\" (UniqueName: \"kubernetes.io/projected/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-kube-api-access-6skm9\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.838715 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.838684 1563 reconciler.go:300] "Volume detached for volume \"prometheus-adapter-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-prometheus-adapter-audit-profiles\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.838715 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.838703 1563 reconciler.go:300] "Volume detached for volume \"elasticsearch-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/220ce491-f766-41ee-ab02-aa1546eb7ed3-elasticsearch-operator-metrics-cert\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.838715 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.838718 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-xj7t8\" (UniqueName: \"kubernetes.io/projected/220ce491-f766-41ee-ab02-aa1546eb7ed3-kube-api-access-xj7t8\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.838909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.838734 1563 reconciler.go:300] "Volume detached for volume \"prometheus-adapter-prometheus-config\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-prometheus-adapter-prometheus-config\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.838909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.838751 1563 reconciler.go:300] "Volume detached for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-serving-certs-ca-bundle\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.838909 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.838765 1563 reconciler.go:300] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/17691c72-c4d7-4e7f-b345-a0447ae7d4c6-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:21.859264 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.859220685Z" level=info msg="Removed container 7cbc78ab6343be67615183679b15bac5d69628d49302137654538e7fd8d0218d: migration-upgrade-49678/mydep-49678-6c64845f74-kvbp4/mydep-49678" id=35faaea7-6a4c-4879-8126-bf0e1b513c0a name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:21.859508 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.859455 1563 scope.go:110] "RemoveContainer" containerID="d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c" Nov 16 15:11:21.860640 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.860603996Z" level=info msg="Removing container: d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c" id=5c067712-02e3-4fce-8b59-f61a70c0c082 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:21.895121 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.895076475Z" level=info msg="Removed container d06fb8144ed417e58aea67fc8ca555d707157bed76861950bdf73e987b6c038c: openshift-logging/cluster-logging-operator-7f6c6bb67c-hmvsf/cluster-logging-operator" id=5c067712-02e3-4fce-8b59-f61a70c0c082 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:21.895416 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.895381 1563 scope.go:110] "RemoveContainer" containerID="7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858" Nov 16 15:11:21.897004 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.896963590Z" level=info msg="Removing container: 7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858" id=cc2a0846-7502-49e3-b2d8-97205a29ee41 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:21.938108 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.938030822Z" level=info msg="Removed container 7682bf118828abbb158b1727a985cd239d17cfbe145d96c3e3adb6c29190c858: migration-upgrade-50425/mydep-50425-85f87ccbc6-wgxqg/mydep-50425" id=cc2a0846-7502-49e3-b2d8-97205a29ee41 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:21.938377 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.938342 1563 scope.go:110] "RemoveContainer" containerID="6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b" Nov 16 15:11:21.939339 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.939305332Z" level=info msg="Removing container: 6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b" id=f25dace8-09d7-4b4d-93a9-72b00a448e97 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:21.973085 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.973007005Z" level=info msg="Removed container 6e0b4dd6b1a3b7557dd82503bfda1f87f9b85c55d9bc074654455ed2319e931b: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-rbac-proxy-self" id=f25dace8-09d7-4b4d-93a9-72b00a448e97 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:21.973278 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:21.973247 1563 scope.go:110] "RemoveContainer" containerID="4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4" Nov 16 15:11:21.974526 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:21.974385216Z" level=info msg="Removing container: 4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4" id=dc49c4f7-e069-46c7-89e9-e6053654cfc3 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.008984 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.008940895Z" level=info msg="Removed container 4dec10cfeccc7f5bb0b4121a02e3b883f33d155a572c3f449a48dbe31221f8a4: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-rbac-proxy-main" id=dc49c4f7-e069-46c7-89e9-e6053654cfc3 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.009205 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.009174 1563 scope.go:110] "RemoveContainer" containerID="d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610" Nov 16 15:11:22.010239 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.010203305Z" level=info msg="Removing container: d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610" id=b440a15f-01d9-4625-928a-34132512e156 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.042179 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.042120363Z" level=info msg="Removed container d7957427c6fca49cb947314733545d5572c4aa424c4583a086d1efcc68bf3610: openshift-monitoring/kube-state-metrics-68667ff848-f92nw/kube-state-metrics" id=b440a15f-01d9-4625-928a-34132512e156 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.042353 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.042324 1563 scope.go:110] "RemoveContainer" containerID="db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95" Nov 16 15:11:22.043419 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.043381773Z" level=info msg="Removing container: db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95" id=b07f00de-90ea-4bd5-9177-4c2ab956bfd6 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.084341 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.084293204Z" level=info msg="Removed container db149a7e72e0799079e69f339eafe7703ae8a76ab43a61ce0941e322f452dd95: openshift-monitoring/prometheus-k8s-1/kube-rbac-proxy-thanos" id=b07f00de-90ea-4bd5-9177-4c2ab956bfd6 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.084663 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.084619 1563 scope.go:110] "RemoveContainer" containerID="9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2" Nov 16 15:11:22.085887 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.085853416Z" level=info msg="Removing container: 9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2" id=f31a84ee-3e34-409b-9759-634bc30746ed name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.124260 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.124092725Z" level=info msg="Removed container 9e0b851ad31801b2b9f30b7cbc320f0541f9bea404d0519fdb5110d8d3125fe2: openshift-monitoring/prometheus-k8s-1/kube-rbac-proxy" id=f31a84ee-3e34-409b-9759-634bc30746ed name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.124491 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.124438 1563 scope.go:110] "RemoveContainer" containerID="526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4" Nov 16 15:11:22.125630 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.125591938Z" level=info msg="Removing container: 526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4" id=0ba65105-649f-4f0f-a026-966e66905310 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.158683 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.158646005Z" level=info msg="Removed container 526da9f1ad1351e831dc123885afec7a4c6550076c5bd51d52885b41028b08b4: openshift-monitoring/prometheus-k8s-1/prometheus-proxy" id=0ba65105-649f-4f0f-a026-966e66905310 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.158950 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.158873 1563 scope.go:110] "RemoveContainer" containerID="f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09" Nov 16 15:11:22.160239 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.160197717Z" level=info msg="Removing container: f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09" id=bba1fd8e-0061-48fb-8d79-bd3f66c1e50e name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.195787 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.195702804Z" level=info msg="Removed container f52c208b233a5246237830fa4f38c96faf833ba5ac20bfbb81ab5917108f1e09: openshift-monitoring/prometheus-k8s-1/thanos-sidecar" id=bba1fd8e-0061-48fb-8d79-bd3f66c1e50e name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.196026 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.195995 1563 scope.go:110] "RemoveContainer" containerID="e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a" Nov 16 15:11:22.197178 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.197091615Z" level=info msg="Removing container: e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a" id=e2511b4b-5f79-44a1-bbe2-7e8671c85f05 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.235508 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.235357625Z" level=info msg="Removed container e9f285ea6d456cbec040bd5af1f6f7d43410e2758afaabe446550ab0b25e152a: openshift-monitoring/prometheus-k8s-1/config-reloader" id=e2511b4b-5f79-44a1-bbe2-7e8671c85f05 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.235627 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.235594 1563 scope.go:110] "RemoveContainer" containerID="6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae" Nov 16 15:11:22.236596 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.236558634Z" level=info msg="Removing container: 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae" id=0d82a358-65e0-4bfa-aaaa-6f524bb3f2b2 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.241087 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.241043 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:11:22.241349 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:22.241323 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:11:22.241416 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:22.241356 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:11:22.241959 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.241931 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt" event=&{ID:220ce491-f766-41ee-ab02-aa1546eb7ed3 Type:ContainerDied Data:59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f} Nov 16 15:11:22.243856 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.243830 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv" event=&{ID:c8013b4f-15b6-4bac-bfc9-a45301362f0c Type:ContainerDied Data:9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c} Nov 16 15:11:22.248471 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.248442 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx" event=&{ID:17691c72-c4d7-4e7f-b345-a0447ae7d4c6 Type:ContainerDied Data:9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486} Nov 16 15:11:22.252567 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:11:22.254459 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.253513 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:11:22.254157 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod220ce491_f766_41ee_ab02_aa1546eb7ed3.slice. Nov 16 15:11:22.254223 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-besteffort-pod220ce491_f766_41ee_ab02_aa1546eb7ed3.slice: Consumed 33.395s CPU time Nov 16 15:11:22.257442 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:22.257409 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:11:26.257386403 +0000 UTC m=+11891.194162025 (durationBeforeRetry 4s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:11:22.257442 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:11:22.257442 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:11:22.258792 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r6cc254f2194646f6991fcbbb5dc3f43d.scope: Succeeded. Nov 16 15:11:22.259790 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r6cc254f2194646f6991fcbbb5dc3f43d.scope: Consumed 847us CPU time Nov 16 15:11:22.263344 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod17691c72_c4d7_4e7f_b345_a0447ae7d4c6.slice. Nov 16 15:11:22.263390 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod17691c72_c4d7_4e7f_b345_a0447ae7d4c6.slice: Consumed 10.734s CPU time Nov 16 15:11:22.265468 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-podc8013b4f_15b6_4bac_bfc9_a45301362f0c.slice. Nov 16 15:11:22.265524 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-podc8013b4f_15b6_4bac_bfc9_a45301362f0c.slice: Consumed 1.148s CPU time Nov 16 15:11:22.270331 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.270288407Z" level=info msg="Removed container 6c9e4f3ef651b9432c4e412d6eed63e59937a7b64796323422d9edb7600e29ae: openshift-monitoring/prometheus-k8s-1/prometheus" id=0d82a358-65e0-4bfa-aaaa-6f524bb3f2b2 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.270591 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.270549 1563 scope.go:110] "RemoveContainer" containerID="55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4" Nov 16 15:11:22.271695 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.271659118Z" level=info msg="Removing container: 55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4" id=a0721929-1c7b-4954-8a26-0843517bd07f name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.283780 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.283749 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt] Nov 16 15:11:22.294531 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-c05813fbf9777e8565a9b584c77779bc6c421abfd6a4e5ff4caf4788045f0526-merged.mount: Succeeded. Nov 16 15:11:22.295439 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-c05813fbf9777e8565a9b584c77779bc6c421abfd6a4e5ff4caf4788045f0526-merged.mount: Consumed 0 CPU time Nov 16 15:11:22.302745 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.302679 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt] Nov 16 15:11:22.334052 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.333976 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv] Nov 16 15:11:22.340592 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.340530 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv] Nov 16 15:11:22.371456 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.371407 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx] Nov 16 15:11:22.382329 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.382296 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx] Nov 16 15:11:22.601191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.597117 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=17691c72-c4d7-4e7f-b345-a0447ae7d4c6 path="/var/lib/kubelet/pods/17691c72-c4d7-4e7f-b345-a0447ae7d4c6/volumes" Nov 16 15:11:22.601191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.597808 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=220ce491-f766-41ee-ab02-aa1546eb7ed3 path="/var/lib/kubelet/pods/220ce491-f766-41ee-ab02-aa1546eb7ed3/volumes" Nov 16 15:11:22.601191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.598395 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=35f6c830-e480-446f-a5cc-a6aaf007ea9a path="/var/lib/kubelet/pods/35f6c830-e480-446f-a5cc-a6aaf007ea9a/volumes" Nov 16 15:11:22.601191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.598887 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=add06140-663d-48c0-ac00-643bd58a3f7d path="/var/lib/kubelet/pods/add06140-663d-48c0-ac00-643bd58a3f7d/volumes" Nov 16 15:11:22.601191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.599518 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=c8013b4f-15b6-4bac-bfc9-a45301362f0c path="/var/lib/kubelet/pods/c8013b4f-15b6-4bac-bfc9-a45301362f0c/volumes" Nov 16 15:11:22.601191 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.600299 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=e0a5ccca-d055-4b4c-820e-515fbf3f92d5 path="/var/lib/kubelet/pods/e0a5ccca-d055-4b4c-820e-515fbf3f92d5/volumes" Nov 16 15:11:22.705697 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.705638325Z" level=info msg="Removed container 55b4a251d46c0a4a66122dd4d2ef68c6b3414a83a02cf1ce4c59052d62cb81c4: openshift-monitoring/prometheus-k8s-1/init-config-reloader" id=a0721929-1c7b-4954-8a26-0843517bd07f name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.706265 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.705994 1563 scope.go:110] "RemoveContainer" containerID="0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0" Nov 16 15:11:22.707240 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.707200937Z" level=info msg="Removing container: 0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0" id=79619387-3d2e-401b-9f12-ca120e49b2bb name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.745664 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.745605648Z" level=info msg="Removed container 0e7c706f1467070e288a3307394a19ae51ad1db3e72894cf66961a7499042ff0: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy-metrics" id=79619387-3d2e-401b-9f12-ca120e49b2bb name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.745913 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.745878 1563 scope.go:110] "RemoveContainer" containerID="f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665" Nov 16 15:11:22.746990 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.746951359Z" level=info msg="Removing container: f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665" id=d1ee4a25-0ff6-4889-bc1a-87b3774c9069 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.786976 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.786889781Z" level=info msg="Removed container f61920116381ef207475fea4268e2ed219c12e24db1acc1abd4f8a4eeaddc665: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy-rules" id=d1ee4a25-0ff6-4889-bc1a-87b3774c9069 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.787337 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.787294 1563 scope.go:110] "RemoveContainer" containerID="8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619" Nov 16 15:11:22.788574 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.788538195Z" level=info msg="Removing container: 8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619" id=0ef76135-a1a0-473d-82ec-bcad401cb0e5 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.826670 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.824193483Z" level=info msg="Removed container 8ff5ec40efb447c9cf5d3ee8fde0acd99e10c3d08bf860e005de253647469619: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/prom-label-proxy" id=0ef76135-a1a0-473d-82ec-bcad401cb0e5 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.826670 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.825537594Z" level=info msg="Removing container: fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf" id=8f657c3e-4cbc-454f-b95c-d8b03dda5b48 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.826903 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.824489 1563 scope.go:110] "RemoveContainer" containerID="fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf" Nov 16 15:11:22.860921 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.860792379Z" level=info msg="Removed container fdb9a7258546c5259b672b9777775b533133a72c93144a3df44bfbc86576daaf: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/kube-rbac-proxy" id=8f657c3e-4cbc-454f-b95c-d8b03dda5b48 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.861085 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.861054 1563 scope.go:110] "RemoveContainer" containerID="a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222" Nov 16 15:11:22.864614 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.864579109Z" level=info msg="Removing container: a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222" id=46827f7b-3c6a-4daf-bb56-44c8538875d2 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.899425 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.899360690Z" level=info msg="Removed container a8003de829cbf5883948a97c8eb1ee2c3a80ffcec41cbeb8c2f65fbea6f60222: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/oauth-proxy" id=46827f7b-3c6a-4daf-bb56-44c8538875d2 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.899758 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.899725 1563 scope.go:110] "RemoveContainer" containerID="94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d" Nov 16 15:11:22.901232 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.901193205Z" level=info msg="Removing container: 94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d" id=6896d458-e9e4-4735-b684-8416089bc60b name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.937049 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.937004294Z" level=info msg="Removed container 94a6f4b75cc5b54cc1cb3e22152d715f3ecb6eba84cbda84b891ec40ab38ea0d: openshift-monitoring/thanos-querier-5b6b7d7fd9-d95wm/thanos-query" id=6896d458-e9e4-4735-b684-8416089bc60b name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.937369 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.937331 1563 scope.go:110] "RemoveContainer" containerID="cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778" Nov 16 15:11:22.938387 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.938352405Z" level=info msg="Removing container: cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778" id=1218ebc6-de26-42df-a2fe-9dade89029e4 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.971504 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.971461173Z" level=info msg="Removed container cc72d4ed768d202b6346608d8896479213a2a3a83e2947750c8529fa57106778: openshift-monitoring/prometheus-operator-admission-webhook-6c69dfd44-hxqbj/prometheus-operator-admission-webhook" id=1218ebc6-de26-42df-a2fe-9dade89029e4 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:22.971840 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:22.971739 1563 scope.go:110] "RemoveContainer" containerID="c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e" Nov 16 15:11:22.972876 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:22.972767583Z" level=info msg="Removing container: c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e" id=e8b3025d-cb08-4b09-afc7-83397d5c56d3 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.005892 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.005795450Z" level=info msg="Removed container c84b1cedc85430aee7f47233a02ba54c751a9cd77e3d8155e5df566e26c3172e: openshift-monitoring/telemeter-client-55c5556665-7vs77/kube-rbac-proxy" id=e8b3025d-cb08-4b09-afc7-83397d5c56d3 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.006054 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.006021 1563 scope.go:110] "RemoveContainer" containerID="d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487" Nov 16 15:11:23.007089 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.007049460Z" level=info msg="Removing container: d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487" id=e27e1641-4493-47fe-820f-e78b552f9ba9 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.041792 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.041744641Z" level=info msg="Removed container d3d8ea72d2f15494d1ed3df4005b2e441ef4a84e2ac2182c9a9840ce4546a487: openshift-monitoring/telemeter-client-55c5556665-7vs77/reload" id=e27e1641-4493-47fe-820f-e78b552f9ba9 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.042100 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.042066 1563 scope.go:110] "RemoveContainer" containerID="3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d" Nov 16 15:11:23.043171 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.043118352Z" level=info msg="Removing container: 3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d" id=73fd3c6c-9d77-4b49-a949-58993f3eb9ab name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.081956 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.081904465Z" level=info msg="Removed container 3f3c6b751ca3a3a7bd370c8f38aacdbe0fa2b691a0f0344a8b4928af2751920d: openshift-monitoring/telemeter-client-55c5556665-7vs77/telemeter-client" id=73fd3c6c-9d77-4b49-a949-58993f3eb9ab name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.082310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.082275 1563 scope.go:110] "RemoveContainer" containerID="eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5" Nov 16 15:11:23.083520 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.083448578Z" level=info msg="Removing container: eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5" id=240d0b37-5afd-4c19-848f-9f84469443ec name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.117750 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.117587754Z" level=info msg="Removed container eced3a4f1bb66ef60f94dbe0ec99682dd69bdcca08c47e6ba33dec35a09907f5: openshift-monitoring/alertmanager-main-0/prom-label-proxy" id=240d0b37-5afd-4c19-848f-9f84469443ec name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.117922 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.117887 1563 scope.go:110] "RemoveContainer" containerID="d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3" Nov 16 15:11:23.119015 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.118935665Z" level=info msg="Removing container: d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3" id=ee57f625-f639-43be-a57d-5688dd83dfaf name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.151714 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.151634129Z" level=info msg="Removed container d782bd56c2a199f6b8948dbfe5fc115733017151d7cd81197a83777919a3f6c3: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy-metric" id=ee57f625-f639-43be-a57d-5688dd83dfaf name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.151971 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.151935 1563 scope.go:110] "RemoveContainer" containerID="aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1" Nov 16 15:11:23.153084 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.153012640Z" level=info msg="Removing container: aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1" id=f796065b-1593-4bb1-a04c-da085e956077 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.185941 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.185902406Z" level=info msg="Removed container aad711d982f002e008e3f5fa02ac6f7be49556f4acfd1db26e7e1511ee413ec1: openshift-monitoring/alertmanager-main-0/kube-rbac-proxy" id=f796065b-1593-4bb1-a04c-da085e956077 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.186233 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.186198 1563 scope.go:110] "RemoveContainer" containerID="c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651" Nov 16 15:11:23.187483 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.187448118Z" level=info msg="Removing container: c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651" id=d81c1cb9-4ff9-474e-9762-4f15cbfad653 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.220410 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.220371384Z" level=info msg="Removed container c33089f3afc5456906d84bc2686916ed04e1f30d7c2ccc19e94e0459d41c9651: openshift-monitoring/alertmanager-main-0/alertmanager-proxy" id=d81c1cb9-4ff9-474e-9762-4f15cbfad653 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.220787 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.220754 1563 scope.go:110] "RemoveContainer" containerID="27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518" Nov 16 15:11:23.222168 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.222058898Z" level=info msg="Removing container: 27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518" id=85d65a20-d8cf-471d-bfe5-920a7bdd1f3d name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.258581 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.258537793Z" level=info msg="Removed container 27cebd4feaa9b3998a3cf0810c05323ca4238445d116f8d560733eea07c34518: openshift-monitoring/alertmanager-main-0/config-reloader" id=85d65a20-d8cf-471d-bfe5-920a7bdd1f3d name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.258855 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.258827 1563 scope.go:110] "RemoveContainer" containerID="96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972" Nov 16 15:11:23.259881 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.259838803Z" level=info msg="Removing container: 96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972" id=284f63cd-9a8b-4f50-bf62-0a2325083895 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.294189 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.294109380Z" level=info msg="Removed container 96af7ce67361b963b1c2b55945fdbeb37a457111fbc72e0214db3faf4b19c972: openshift-monitoring/alertmanager-main-0/alertmanager" id=284f63cd-9a8b-4f50-bf62-0a2325083895 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.294477 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.294435 1563 scope.go:110] "RemoveContainer" containerID="dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf" Nov 16 15:11:23.295551 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.295501391Z" level=info msg="Removing container: dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf" id=83cc149c-4fca-4c3d-aa7f-bf27846f78fa name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.330849 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.330757376Z" level=info msg="Removed container dfb4359dcb6f850a2a898e4b17870e0c9dc2195167f7600b60ac31647ab45dbf: openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt/elasticsearch-operator" id=83cc149c-4fca-4c3d-aa7f-bf27846f78fa name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.331070 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.331037 1563 scope.go:110] "RemoveContainer" containerID="881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667" Nov 16 15:11:23.332235 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.332196488Z" level=info msg="Removing container: 881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667" id=ee0d4774-0b04-477a-88b3-b37a791ea409 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.367303 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.367262371Z" level=info msg="Removed container 881e77027170d0c53e568d5f38f45c84beabb5b2b9c1fc87203b7056a32f6667: openshift-operators-redhat/elasticsearch-operator-66b6655fcb-ksdmt/kube-rbac-proxy" id=ee0d4774-0b04-477a-88b3-b37a791ea409 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.367527 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.367492 1563 scope.go:110] "RemoveContainer" containerID="ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778" Nov 16 15:11:23.368785 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.368691383Z" level=info msg="Removing container: ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778" id=c02eb98d-3c5f-4308-8941-042c06aa7a86 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.401954 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.401857051Z" level=info msg="Removed container ceac845fd85409fbe78345be620f266a381e8c7ddf33ea2a837f3f6c87be1778: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/openshift-state-metrics" id=c02eb98d-3c5f-4308-8941-042c06aa7a86 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.402170 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.402123 1563 scope.go:110] "RemoveContainer" containerID="83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a" Nov 16 15:11:23.403266 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.403227962Z" level=info msg="Removing container: 83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a" id=5a5c1aba-8323-4a48-baf1-b9846b836c72 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.442187 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.442143176Z" level=info msg="Removed container 83da6568017beac515bf3eac7fd943aacc1bc923e4d78f9a761d1565df0e447a: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/kube-rbac-proxy-self" id=5a5c1aba-8323-4a48-baf1-b9846b836c72 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.442438 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.442356 1563 scope.go:110] "RemoveContainer" containerID="89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25" Nov 16 15:11:23.443507 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.443457487Z" level=info msg="Removing container: 89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25" id=68f80d54-73e8-4f72-b559-784e70ccdcaa name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.474066 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.473981234Z" level=info msg="Removed container 89ca11203068d17ac86d55e1c30985ed437edbf1234a2481c1ee9c7a3c56cb25: openshift-monitoring/openshift-state-metrics-6959bf55f6-ppwrv/kube-rbac-proxy-main" id=68f80d54-73e8-4f72-b559-784e70ccdcaa name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.474248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:23.474217 1563 scope.go:110] "RemoveContainer" containerID="9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30" Nov 16 15:11:23.475359 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.475321644Z" level=info msg="Removing container: 9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30" id=b504a570-ad5c-47b5-8e75-b3aa9d8511fd name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:23.506044 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:23.506003692Z" level=info msg="Removed container 9871eebaffc0990f82d2d94c9b4e912c307d0f5d04d1233df1949e73ee829a30: openshift-monitoring/prometheus-adapter-6fb766b4f6-jcwvx/prometheus-adapter" id=b504a570-ad5c-47b5-8e75-b3aa9d8511fd name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:24.403928 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.403874848Z" level=info msg="Stopping pod sandbox: c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844" id=58cc55e8-fc16-4fea-9d4b-59f5ca4548d9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.403928 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.403926248Z" level=info msg="Stopped pod sandbox (already stopped): c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844" id=58cc55e8-fc16-4fea-9d4b-59f5ca4548d9 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.404521 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.404233951Z" level=info msg="Removing pod sandbox: c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844" id=1c171b2f-4394-4697-ac41-2ac3b725795c name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.436898 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.436853914Z" level=info msg="Removed pod sandbox: c22299b8e0a38073acaac46c37aa85b63500e9d599dc5ca297adb46cb220c844" id=1c171b2f-4394-4697-ac41-2ac3b725795c name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.437410 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.437373719Z" level=info msg="Stopping pod sandbox: 59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f" id=b1bf73fd-aacc-4150-81ad-13b9fa057d08 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.437502 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.437416819Z" level=info msg="Stopped pod sandbox (already stopped): 59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f" id=b1bf73fd-aacc-4150-81ad-13b9fa057d08 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.437749 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.437718821Z" level=info msg="Removing pod sandbox: 59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f" id=0ff11b21-0a49-4651-84a1-e21ba22d3016 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.456384 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.456345372Z" level=info msg="Removed pod sandbox: 59858fc7003cbd8718704604f2f74740c850db1fbb71a1204bfb67fff105cb5f" id=0ff11b21-0a49-4651-84a1-e21ba22d3016 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.456943 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.456892676Z" level=info msg="Stopping pod sandbox: a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145" id=cac2ec9e-8823-40c6-991e-52359d7f34e6 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.456943 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.456938577Z" level=info msg="Stopped pod sandbox (already stopped): a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145" id=cac2ec9e-8823-40c6-991e-52359d7f34e6 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.457239 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.457202179Z" level=info msg="Removing pod sandbox: a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145" id=8aafd646-4aae-4b69-a3e5-9df94d675ceb name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.476882 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.476791737Z" level=info msg="Removed pod sandbox: a201e559a72996f00570bad3d1a38a530bbd552ceba651b3f0feb868b5ce2145" id=8aafd646-4aae-4b69-a3e5-9df94d675ceb name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.477338 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.477302641Z" level=info msg="Stopping pod sandbox: 9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c" id=b273ddd3-9515-4156-ad0c-81b3e37f2208 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.477400 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.477345742Z" level=info msg="Stopped pod sandbox (already stopped): 9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c" id=b273ddd3-9515-4156-ad0c-81b3e37f2208 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.477634 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.477598444Z" level=info msg="Removing pod sandbox: 9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c" id=dea09e56-3a6f-4cb5-97bf-056a5ee3d3f0 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.495706 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.495623389Z" level=info msg="Removed pod sandbox: 9ebbcda21cb9797158cb8eacbd0dae9ebb483faff199990627501efc9ab2a19c" id=dea09e56-3a6f-4cb5-97bf-056a5ee3d3f0 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.496265 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.496229594Z" level=info msg="Stopping pod sandbox: 4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c" id=3ca50447-efa3-4b84-8fb8-af15b0b7762b name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.496329 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.496270895Z" level=info msg="Stopped pod sandbox (already stopped): 4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c" id=3ca50447-efa3-4b84-8fb8-af15b0b7762b name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.496750 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.496715898Z" level=info msg="Removing pod sandbox: 4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c" id=5fc1c27f-4c1f-4b80-9b1e-eebc1f156f11 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.515712 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.515668251Z" level=info msg="Removed pod sandbox: 4bb10e4d85effb883eea5dd95d3cdc80db32d4aaca41ad5594ae293666d8e04c" id=5fc1c27f-4c1f-4b80-9b1e-eebc1f156f11 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.516119 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.516081055Z" level=info msg="Stopping pod sandbox: 0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7" id=97db5e8c-d602-4f47-a3b7-48267b3a6f3c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.516282 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.516166055Z" level=info msg="Stopped pod sandbox (already stopped): 0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7" id=97db5e8c-d602-4f47-a3b7-48267b3a6f3c name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.516506 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.516462258Z" level=info msg="Removing pod sandbox: 0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7" id=ed48df0d-e7c2-4c29-8500-bdfb544df825 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.533696 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.533661897Z" level=info msg="Removed pod sandbox: 0fd49ef93ad8544a30f3662238683ba0df13aeff6a868d83bb832a2662a03ba7" id=ed48df0d-e7c2-4c29-8500-bdfb544df825 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.534097 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.534060600Z" level=info msg="Stopping pod sandbox: 6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d" id=df364da0-ca2d-41ba-906e-52a64dacef64 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.534201 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.534101400Z" level=info msg="Stopped pod sandbox (already stopped): 6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d" id=df364da0-ca2d-41ba-906e-52a64dacef64 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.534434 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.534397803Z" level=info msg="Removing pod sandbox: 6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d" id=9f9ab7e4-5859-4780-b2dc-52361ae83a55 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.551715 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.551567441Z" level=info msg="Removed pod sandbox: 6ea30fd9a70f05d1284d5db353059fa758a99a4fbd760a411167fb9d5b5d322d" id=9f9ab7e4-5859-4780-b2dc-52361ae83a55 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.552086 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.552036645Z" level=info msg="Stopping pod sandbox: a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe" id=7328316a-6cf3-4838-bf2f-ce26834885be name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.552086 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.552077446Z" level=info msg="Stopped pod sandbox (already stopped): a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe" id=7328316a-6cf3-4838-bf2f-ce26834885be name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.552353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.552322948Z" level=info msg="Removing pod sandbox: a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe" id=25a6146d-373b-4c6b-9150-2a60ea25127f name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.572406 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.572370110Z" level=info msg="Removed pod sandbox: a3e14752329bbac00d9acec721e707be6b55cd416aa8d5523dcc0c49a47ab5fe" id=25a6146d-373b-4c6b-9150-2a60ea25127f name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.573023 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.572941514Z" level=info msg="Stopping pod sandbox: 94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b" id=864a517c-677d-4299-a5cd-c84d8a618e36 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.573023 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.572984115Z" level=info msg="Stopped pod sandbox (already stopped): 94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b" id=864a517c-677d-4299-a5cd-c84d8a618e36 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.573390 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.573353718Z" level=info msg="Removing pod sandbox: 94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b" id=e67c1b99-e6ae-41fd-bfcc-f5d99d8ca3d8 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.610561 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.610437317Z" level=info msg="Removed pod sandbox: 94b915327969294787da9875cf225d78754e5cd12dcef6b6702e5111449de11b" id=e67c1b99-e6ae-41fd-bfcc-f5d99d8ca3d8 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.610899 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.610861621Z" level=info msg="Stopping pod sandbox: 9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9" id=1caf92b9-6063-4740-ac76-f7e377b806c5 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.610951 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.610904521Z" level=info msg="Stopped pod sandbox (already stopped): 9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9" id=1caf92b9-6063-4740-ac76-f7e377b806c5 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.611245 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.611211623Z" level=info msg="Removing pod sandbox: 9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9" id=7266a09a-f1e9-470e-9b05-f7c4e48044f9 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.629315 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.629238269Z" level=info msg="Removed pod sandbox: 9165866ade480f9daac615fa0f84900fbf433ba42322083a2787442d91c300d9" id=7266a09a-f1e9-470e-9b05-f7c4e48044f9 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.629698 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.629660673Z" level=info msg="Stopping pod sandbox: 9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486" id=fb8908bd-d544-477f-8251-c67caddf6e96 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.629698 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.629694873Z" level=info msg="Stopped pod sandbox (already stopped): 9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486" id=fb8908bd-d544-477f-8251-c67caddf6e96 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.630014 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.629977975Z" level=info msg="Removing pod sandbox: 9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486" id=042907dd-b019-443a-a412-4bfefb024f90 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.658795 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.658631207Z" level=info msg="Removed pod sandbox: 9f4da5d922a0fd0bef151c6877fd2abba7b95404dc74ac4128241bb688325486" id=042907dd-b019-443a-a412-4bfefb024f90 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.659043 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.659006110Z" level=info msg="Stopping pod sandbox: 18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4" id=eaff7735-1026-4c3b-be95-8b513e9252c4 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.659043 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.659046410Z" level=info msg="Stopped pod sandbox (already stopped): 18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4" id=eaff7735-1026-4c3b-be95-8b513e9252c4 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.659410 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.659374313Z" level=info msg="Removing pod sandbox: 18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4" id=2ad16d8b-ba0b-4ce5-8b4d-8b52737224cd name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.676698 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.676616252Z" level=info msg="Removed pod sandbox: 18651ab24daebd67fddb14113442277c6264999566c73c90706f9e90f83ff3c4" id=2ad16d8b-ba0b-4ce5-8b4d-8b52737224cd name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.677103 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.677053056Z" level=info msg="Stopping pod sandbox: 5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78" id=c73acd72-9819-4819-baab-ddfd0bb89777 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.677209 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.677109256Z" level=info msg="Stopped pod sandbox (already stopped): 5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78" id=c73acd72-9819-4819-baab-ddfd0bb89777 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.677530 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.677479259Z" level=info msg="Removing pod sandbox: 5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78" id=02dd4ae2-674a-46f9-b985-2efc8b3ef2c1 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.694845 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.694806199Z" level=info msg="Removed pod sandbox: 5d0744ff54c5fc273d2ca57086b8447bcfab5f29c8f5c059d0322a713139bd78" id=02dd4ae2-674a-46f9-b985-2efc8b3ef2c1 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.695234 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.695200802Z" level=info msg="Stopping pod sandbox: 11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30" id=a102bf7a-5677-421f-8dee-81b4cbad6d3b name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.695305 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.695238202Z" level=info msg="Stopped pod sandbox (already stopped): 11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30" id=a102bf7a-5677-421f-8dee-81b4cbad6d3b name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.695555 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.695519505Z" level=info msg="Removing pod sandbox: 11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30" id=d280ca90-45f3-4d5e-999d-bf0318275e33 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.716206 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.716170572Z" level=info msg="Removed pod sandbox: 11b3fac610c1b6bdb2df9902a757b671deaf05ece2dcdcbbfd13bdc964d38a30" id=d280ca90-45f3-4d5e-999d-bf0318275e33 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.716451 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.716415874Z" level=info msg="Stopping pod sandbox: 5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3" id=10fb7bb2-92be-46d3-813f-fda24c7968b3 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.716451 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.716451274Z" level=info msg="Stopped pod sandbox (already stopped): 5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3" id=10fb7bb2-92be-46d3-813f-fda24c7968b3 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.716787 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.716735776Z" level=info msg="Removing pod sandbox: 5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3" id=70e1a2b4-f79a-4656-b6d3-9d2bd521b808 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.735140 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.735057324Z" level=info msg="Removed pod sandbox: 5eb8d9f2ce5bf3d1c27aebdaa0c0a4d2776fe9f891a0fef98769a5e0c41faef3" id=70e1a2b4-f79a-4656-b6d3-9d2bd521b808 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.735481 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.735446227Z" level=info msg="Stopping pod sandbox: 0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a" id=a5badb7c-1509-4e80-aa22-4bee9be6a64b name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.735541 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.735483928Z" level=info msg="Stopped pod sandbox (already stopped): 0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a" id=a5badb7c-1509-4e80-aa22-4bee9be6a64b name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:24.735791 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.735730230Z" level=info msg="Removing pod sandbox: 0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a" id=aee39851-f751-482e-826b-31098363e3d5 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:24.753852 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:24.753771075Z" level=info msg="Removed pod sandbox: 0bf039484ac010a4de9a64dae9f33ab7667620b50eb9cabbb80afcf23f6e1a3a" id=aee39851-f751-482e-826b-31098363e3d5 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:11:26.273403 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:26.273340 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:11:26.273930 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:26.273617 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:11:26.273930 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:26.273645 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:11:26.284384 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:11:26.285559 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:26.285528 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:11:26.289739 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:26.289676 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:11:34.289650987 +0000 UTC m=+11899.226426609 (durationBeforeRetry 8s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:11:26.289739 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:11:26.289739 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:11:26.290083 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r8ee75dfea5e740acb96217ec51a6f0e2.scope: Succeeded. Nov 16 15:11:26.290696 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r8ee75dfea5e740acb96217ec51a6f0e2.scope: Consumed 977us CPU time Nov 16 15:11:26.348768 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:26.348732 1563 patch_prober.go:29] interesting pod/router-default-cff655768-nrbzb container/router namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[+]backend-http ok Nov 16 15:11:26.348768 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: [+]has-synced ok Nov 16 15:11:26.348768 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: [-]process-running failed: reason withheld Nov 16 15:11:26.348768 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: healthz check failed Nov 16 15:11:26.348953 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:26.348801 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-ingress/router-default-cff655768-nrbzb" podUID=03363f87-776c-47c3-9bd8-8c0de24a4de5 containerName="router" probeResult=failure output="HTTP probe failed with statuscode: 500" Nov 16 15:11:34.344248 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:34.344097 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:11:34.344868 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:34.344475 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:11:34.344868 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:34.344506 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:11:34.354715 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:11:34.355790 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:34.355695 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:11:34.359564 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:34.359454 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:11:50.359428209 +0000 UTC m=+11915.296203831 (durationBeforeRetry 16s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:11:34.359564 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:11:34.359564 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:11:34.360174 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rb00e1e8eca2443dc9bae9d874ee440a5.scope: Succeeded. Nov 16 15:11:34.360668 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rb00e1e8eca2443dc9bae9d874ee440a5.scope: Consumed 855us CPU time Nov 16 15:11:35.569962 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:35.569918 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" podUID=0520c563-12a4-4756-b022-6ec7396a5839 containerName="registry" containerID="cri-o://d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa" gracePeriod=55 Nov 16 15:11:35.570457 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:35.570208522Z" level=info msg="Stopping container: d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa (timeout: 55s)" id=e1f9f3b6-d119-4554-b74c-48a09e44f404 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:35.681272 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:35.681211 1563 patch_prober.go:29] interesting pod/image-registry-944cf86d8-kb2fb container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.129.2.35:5000/healthz\": dial tcp 10.129.2.35:5000: connect: connection refused" start-of-body= Nov 16 15:11:35.681272 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:35.681303 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" podUID=0520c563-12a4-4756-b022-6ec7396a5839 containerName="registry" probeResult=failure output="Get \"https://10.129.2.35:5000/healthz\": dial tcp 10.129.2.35:5000: connect: connection refused" Nov 16 15:11:36.349185 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:36.349074 1563 patch_prober.go:29] interesting pod/router-default-cff655768-nrbzb container/router namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[+]backend-http ok Nov 16 15:11:36.349185 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: [+]has-synced ok Nov 16 15:11:36.349185 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: [-]process-running failed: reason withheld Nov 16 15:11:36.349185 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: healthz check failed Nov 16 15:11:36.349185 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:36.349192 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-ingress/router-default-cff655768-nrbzb" podUID=03363f87-776c-47c3-9bd8-8c0de24a4de5 containerName="router" probeResult=failure output="HTTP probe failed with statuscode: 500" Nov 16 15:11:36.349592 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:36.349287 1563 kubelet.go:2247] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-ingress/router-default-cff655768-nrbzb" Nov 16 15:11:36.641464 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa.scope: Succeeded. Nov 16 15:11:36.642234 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa.scope: Consumed 2.468s CPU time Nov 16 15:11:36.648740 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa.scope: Succeeded. Nov 16 15:11:36.649426 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa.scope: Consumed 70ms CPU time Nov 16 15:11:36.740506 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-f005736cb612af20b2bd9e45190f12a9ff76dc65530e545078fd9d70c2e625ee-merged.mount: Succeeded. Nov 16 15:11:36.741530 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-f005736cb612af20b2bd9e45190f12a9ff76dc65530e545078fd9d70c2e625ee-merged.mount: Consumed 0 CPU time Nov 16 15:11:37.174351 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:37.174292830Z" level=info msg="Stopped container d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa: openshift-image-registry/image-registry-944cf86d8-kb2fb/registry" id=e1f9f3b6-d119-4554-b74c-48a09e44f404 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:37.175034 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:37.174947136Z" level=info msg="Stopping pod sandbox: 4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e" id=f88e5631-cfc9-4aea-bd90-686e5cd904bd name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:37.175273 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:37.175224238Z" level=info msg="Got pod network &{Name:image-registry-944cf86d8-kb2fb Namespace:openshift-image-registry ID:4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e UID:0520c563-12a4-4756-b022-6ec7396a5839 NetNS:/var/run/netns/887d142c-2ca2-4563-b8cc-9b0e806e344e Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:37.175414 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:37.175385940Z" level=info msg="Deleting pod openshift-image-registry_image-registry-944cf86d8-kb2fb from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:37.246605 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00306|connmgr|INFO|br0<->unix#683: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:37.277577 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00307|connmgr|INFO|br0<->unix#686: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:37.296943 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.296904 1563 generic.go:296] "Generic (PLEG): container finished" podID=0520c563-12a4-4756-b022-6ec7396a5839 containerID="d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa" exitCode=0 Nov 16 15:11:37.297458 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.296963 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" event=&{ID:0520c563-12a4-4756-b022-6ec7396a5839 Type:ContainerDied Data:d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa} Nov 16 15:11:37.305204 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00308|bridge|INFO|bridge br0: deleted interface vethf52cb045 on port 36 Nov 16 15:11:37.312156 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethf52cb045 left promiscuous mode Nov 16 15:11:37.320352 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:37Z [verbose] Del: openshift-image-registry:image-registry-944cf86d8-kb2fb:0520c563-12a4-4756-b022-6ec7396a5839:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:37.326878 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-887d142c\x2d2ca2\x2d4563\x2db8cc\x2d9b0e806e344e.mount: Succeeded. Nov 16 15:11:37.327822 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-887d142c\x2d2ca2\x2d4563\x2db8cc\x2d9b0e806e344e.mount: Consumed 0 CPU time Nov 16 15:11:37.343783 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-887d142c\x2d2ca2\x2d4563\x2db8cc\x2d9b0e806e344e.mount: Succeeded. Nov 16 15:11:37.344765 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-887d142c\x2d2ca2\x2d4563\x2db8cc\x2d9b0e806e344e.mount: Consumed 0 CPU time Nov 16 15:11:37.370425 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-887d142c\x2d2ca2\x2d4563\x2db8cc\x2d9b0e806e344e.mount: Succeeded. Nov 16 15:11:37.371191 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-887d142c\x2d2ca2\x2d4563\x2db8cc\x2d9b0e806e344e.mount: Consumed 0 CPU time Nov 16 15:11:37.390306 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e-userdata-shm.mount: Succeeded. Nov 16 15:11:37.391177 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:37.616133 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:37.616076564Z" level=info msg="Stopped pod sandbox: 4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e" id=f88e5631-cfc9-4aea-bd90-686e5cd904bd name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:37.742028 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-540da2c9a1a3492cd357f283b94644c97971ee5f35776c27aff8d2ac7d7258f8-merged.mount: Succeeded. Nov 16 15:11:37.743275 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-540da2c9a1a3492cd357f283b94644c97971ee5f35776c27aff8d2ac7d7258f8-merged.mount: Consumed 0 CPU time Nov 16 15:11:37.772240 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.772090 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0520c563-12a4-4756-b022-6ec7396a5839-ca-trust-extracted\") pod \"0520c563-12a4-4756-b022-6ec7396a5839\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " Nov 16 15:11:37.772240 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.772228 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0520c563-12a4-4756-b022-6ec7396a5839-trusted-ca\") pod \"0520c563-12a4-4756-b022-6ec7396a5839\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " Nov 16 15:11:37.772461 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.772271 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-bound-sa-token\") pod \"0520c563-12a4-4756-b022-6ec7396a5839\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " Nov 16 15:11:37.772461 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.772305 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0520c563-12a4-4756-b022-6ec7396a5839-installation-pull-secrets\") pod \"0520c563-12a4-4756-b022-6ec7396a5839\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " Nov 16 15:11:37.772461 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.772334 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0520c563-12a4-4756-b022-6ec7396a5839-registry-certificates\") pod \"0520c563-12a4-4756-b022-6ec7396a5839\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " Nov 16 15:11:37.772461 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.772416 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfvls\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-kube-api-access-kfvls\") pod \"0520c563-12a4-4756-b022-6ec7396a5839\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " Nov 16 15:11:37.772461 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:37.772408 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/0520c563-12a4-4756-b022-6ec7396a5839/volumes/kubernetes.io~empty-dir/ca-trust-extracted: clearQuota called, but quotas disabled Nov 16 15:11:37.772461 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.772443 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-registry-tls\") pod \"0520c563-12a4-4756-b022-6ec7396a5839\" (UID: \"0520c563-12a4-4756-b022-6ec7396a5839\") " Nov 16 15:11:37.772644 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:37.772551 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/0520c563-12a4-4756-b022-6ec7396a5839/volumes/kubernetes.io~configmap/trusted-ca: clearQuota called, but quotas disabled Nov 16 15:11:37.772682 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:37.772652 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/0520c563-12a4-4756-b022-6ec7396a5839/volumes/kubernetes.io~configmap/registry-certificates: clearQuota called, but quotas disabled Nov 16 15:11:37.772855 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.772829 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0520c563-12a4-4756-b022-6ec7396a5839-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0520c563-12a4-4756-b022-6ec7396a5839" (UID: "0520c563-12a4-4756-b022-6ec7396a5839"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:37.772971 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.772932 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0520c563-12a4-4756-b022-6ec7396a5839-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0520c563-12a4-4756-b022-6ec7396a5839" (UID: "0520c563-12a4-4756-b022-6ec7396a5839"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:37.773123 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.773073 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0520c563-12a4-4756-b022-6ec7396a5839-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0520c563-12a4-4756-b022-6ec7396a5839" (UID: "0520c563-12a4-4756-b022-6ec7396a5839"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Nov 16 15:11:37.780519 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-0520c563\x2d12a4\x2d4756\x2db022\x2d6ec7396a5839-volumes-kubernetes.io\x7eprojected-bound\x2dsa\x2dtoken.mount: Succeeded. Nov 16 15:11:37.781374 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-0520c563\x2d12a4\x2d4756\x2db022\x2d6ec7396a5839-volumes-kubernetes.io\x7eprojected-bound\x2dsa\x2dtoken.mount: Consumed 0 CPU time Nov 16 15:11:37.781399 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-0520c563\x2d12a4\x2d4756\x2db022\x2d6ec7396a5839-volumes-kubernetes.io\x7eprojected-registry\x2dtls.mount: Succeeded. Nov 16 15:11:37.782173 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-0520c563\x2d12a4\x2d4756\x2db022\x2d6ec7396a5839-volumes-kubernetes.io\x7eprojected-registry\x2dtls.mount: Consumed 0 CPU time Nov 16 15:11:37.782193 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-0520c563\x2d12a4\x2d4756\x2db022\x2d6ec7396a5839-volumes-kubernetes.io\x7esecret-installation\x2dpull\x2dsecrets.mount: Succeeded. Nov 16 15:11:37.782926 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-0520c563\x2d12a4\x2d4756\x2db022\x2d6ec7396a5839-volumes-kubernetes.io\x7esecret-installation\x2dpull\x2dsecrets.mount: Consumed 0 CPU time Nov 16 15:11:37.788808 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-0520c563\x2d12a4\x2d4756\x2db022\x2d6ec7396a5839-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dkfvls.mount: Succeeded. Nov 16 15:11:37.789687 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-0520c563\x2d12a4\x2d4756\x2db022\x2d6ec7396a5839-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dkfvls.mount: Consumed 0 CPU time Nov 16 15:11:37.790433 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.790395 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0520c563-12a4-4756-b022-6ec7396a5839" (UID: "0520c563-12a4-4756-b022-6ec7396a5839"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:37.791262 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.791211 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0520c563-12a4-4756-b022-6ec7396a5839-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0520c563-12a4-4756-b022-6ec7396a5839" (UID: "0520c563-12a4-4756-b022-6ec7396a5839"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:37.799463 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.799381 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-kube-api-access-kfvls" (OuterVolumeSpecName: "kube-api-access-kfvls") pod "0520c563-12a4-4756-b022-6ec7396a5839" (UID: "0520c563-12a4-4756-b022-6ec7396a5839"). InnerVolumeSpecName "kube-api-access-kfvls". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:37.807496 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.807452 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0520c563-12a4-4756-b022-6ec7396a5839" (UID: "0520c563-12a4-4756-b022-6ec7396a5839"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:37.873293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.873121 1563 reconciler.go:300] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0520c563-12a4-4756-b022-6ec7396a5839-ca-trust-extracted\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:37.873293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.873200 1563 reconciler.go:300] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0520c563-12a4-4756-b022-6ec7396a5839-trusted-ca\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:37.873293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.873218 1563 reconciler.go:300] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-bound-sa-token\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:37.873293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.873235 1563 reconciler.go:300] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0520c563-12a4-4756-b022-6ec7396a5839-installation-pull-secrets\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:37.873293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.873266 1563 reconciler.go:300] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0520c563-12a4-4756-b022-6ec7396a5839-registry-certificates\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:37.873293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.873281 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-kfvls\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-kube-api-access-kfvls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:37.873293 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:37.873295 1563 reconciler.go:300] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0520c563-12a4-4756-b022-6ec7396a5839-registry-tls\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:38.301374 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:38.301190 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" event=&{ID:0520c563-12a4-4756-b022-6ec7396a5839 Type:ContainerDied Data:4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e} Nov 16 15:11:38.301374 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:38.301247 1563 scope.go:110] "RemoveContainer" containerID="d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa" Nov 16 15:11:38.302571 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:38.302505434Z" level=info msg="Removing container: d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa" id=49fc8aa8-b877-40f2-b75d-775cc13bdba2 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:38.309646 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod0520c563_12a4_4756_b022_6ec7396a5839.slice. Nov 16 15:11:38.309731 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod0520c563_12a4_4756_b022_6ec7396a5839.slice: Consumed 2.538s CPU time Nov 16 15:11:38.340878 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:38.340825 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-image-registry/image-registry-944cf86d8-kb2fb] Nov 16 15:11:38.341326 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:38.341285888Z" level=info msg="Removed container d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa: openshift-image-registry/image-registry-944cf86d8-kb2fb/registry" id=49fc8aa8-b877-40f2-b75d-775cc13bdba2 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:38.353803 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:38.353747 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-image-registry/image-registry-944cf86d8-kb2fb] Nov 16 15:11:38.591580 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:38.591506 1563 kuberuntime_container.go:729] "Killing container with a grace period" pod="openshift-image-registry/image-registry-944cf86d8-kb2fb" podUID=0520c563-12a4-4756-b022-6ec7396a5839 containerName="registry" containerID="cri-o://d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa" gracePeriod=1 Nov 16 15:11:38.591802 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:38.591746775Z" level=info msg="Stopping container: d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa (timeout: 1s)" id=08049137-34b6-40da-a32d-9f4446fb2882 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:38.592031 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:38.592008 1563 remote_runtime.go:479] "StopContainer from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa\": container with ID starting with d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa not found: ID does not exist" containerID="d0b89fcec1a0a33da5c39b2857fe00d6e4c5109b9a0a4c01dd86b6b406e7a6fa" Nov 16 15:11:38.592287 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:38.592252680Z" level=info msg="Stopping pod sandbox: 4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e" id=0240eca7-97e6-4037-9971-4b3f0263d036 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:38.592347 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:38.592293680Z" level=info msg="Stopped pod sandbox (already stopped): 4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e" id=0240eca7-97e6-4037-9971-4b3f0263d036 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:38.593200 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:38.593085 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=0520c563-12a4-4756-b022-6ec7396a5839 path="/var/lib/kubelet/pods/0520c563-12a4-4756-b022-6ec7396a5839/volumes" Nov 16 15:11:46.348454 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:46.348401 1563 patch_prober.go:29] interesting pod/router-default-cff655768-nrbzb container/router namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[+]backend-http ok Nov 16 15:11:46.348454 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: [+]has-synced ok Nov 16 15:11:46.348454 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: [-]process-running failed: reason withheld Nov 16 15:11:46.348454 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: healthz check failed Nov 16 15:11:46.349090 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:46.348486 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-ingress/router-default-cff655768-nrbzb" podUID=03363f87-776c-47c3-9bd8-8c0de24a4de5 containerName="router" probeResult=failure output="HTTP probe failed with statuscode: 500" Nov 16 15:11:50.380406 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:50.380300 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:11:50.380860 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:50.380574 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:11:50.380860 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:50.380606 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:11:50.390167 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:11:50.391144 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:50.391063 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:11:50.394976 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:11:50.394884 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:12:22.394860754 +0000 UTC m=+11947.331636376 (durationBeforeRetry 32s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:11:50.394976 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:11:50.394976 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:11:50.395619 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r2e40e1f07e8741c89e9e03ad17677984.scope: Succeeded. Nov 16 15:11:50.396200 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r2e40e1f07e8741c89e9e03ad17677984.scope: Consumed 787us CPU time Nov 16 15:11:56.347527 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:56.347421 1563 patch_prober.go:29] interesting pod/router-default-cff655768-nrbzb container/router namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Nov 16 15:11:56.347527 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: [+]has-synced ok Nov 16 15:11:56.347527 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: [-]process-running failed: reason withheld Nov 16 15:11:56.347527 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: healthz check failed Nov 16 15:11:56.347527 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:56.347532 1563 prober.go:121] "Probe failed" probeType="Readiness" pod="openshift-ingress/router-default-cff655768-nrbzb" podUID=03363f87-776c-47c3-9bd8-8c0de24a4de5 containerName="router" probeResult=failure output="HTTP probe failed with statuscode: 500" Nov 16 15:11:57.212958 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3.scope: Succeeded. Nov 16 15:11:57.213665 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3.scope: Consumed 1.967s CPU time Nov 16 15:11:57.225434 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3.scope: Succeeded. Nov 16 15:11:57.226209 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: crio-conmon-5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3.scope: Consumed 43ms CPU time Nov 16 15:11:57.296047 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0a851ce41f960679057c8b15fde41eec79bbe24298aaf498865e76c06d06a2f0-merged.mount: Succeeded. Nov 16 15:11:57.296984 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-0a851ce41f960679057c8b15fde41eec79bbe24298aaf498865e76c06d06a2f0-merged.mount: Consumed 0 CPU time Nov 16 15:11:57.349245 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:57.349210 1563 generic.go:296] "Generic (PLEG): container finished" podID=03363f87-776c-47c3-9bd8-8c0de24a4de5 containerID="5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3" exitCode=0 Nov 16 15:11:57.349671 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:57.349267 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-cff655768-nrbzb" event=&{ID:03363f87-776c-47c3-9bd8-8c0de24a4de5 Type:ContainerDied Data:5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3} Nov 16 15:11:57.629555 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:57.629459072Z" level=info msg="Stopped container 5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3: openshift-ingress/router-default-cff655768-nrbzb/router" id=522df1d8-436a-4ad1-9093-8a918a9b9229 name=/runtime.v1.RuntimeService/StopContainer Nov 16 15:11:57.630150 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:57.630067876Z" level=info msg="Stopping pod sandbox: f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2" id=3075dcd9-74f9-447d-bc0d-0b34d4f158a7 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:57.630387 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:57.630334577Z" level=info msg="Got pod network &{Name:router-default-cff655768-nrbzb Namespace:openshift-ingress ID:f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2 UID:03363f87-776c-47c3-9bd8-8c0de24a4de5 NetNS:/var/run/netns/05707f1f-11b9-44c3-9087-b9367f58b73c Networks:[{Name:multus-cni-network Ifname:eth0}] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth: IpRanges:[]}] Aliases:map[]}" Nov 16 15:11:57.630574 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:57.630547979Z" level=info msg="Deleting pod openshift-ingress_router-default-cff655768-nrbzb from CNI network \"multus-cni-network\" (type=multus)" Nov 16 15:11:57.698997 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00309|connmgr|INFO|br0<->unix#689: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:11:57.731122 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00310|connmgr|INFO|br0<->unix#692: 4 flow_mods in the last 0 s (4 deletes) Nov 16 15:11:57.760422 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00311|bridge|INFO|bridge br0: deleted interface vethade753ea on port 42 Nov 16 15:11:57.765184 ropatil1611az-gs6zj-worker-westus-j4ngg kernel: device vethade753ea left promiscuous mode Nov 16 15:11:57.774298 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: 2022-11-16T15:11:57Z [verbose] Del: openshift-ingress:router-default-cff655768-nrbzb:03363f87-776c-47c3-9bd8-8c0de24a4de5:openshift-sdn:eth0 {"cniVersion":"0.3.1","name":"openshift-sdn","type":"openshift-sdn"} Nov 16 15:11:57.780835 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-05707f1f\x2d11b9\x2d44c3\x2d9087\x2db9367f58b73c.mount: Succeeded. Nov 16 15:11:57.781798 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-utsns-05707f1f\x2d11b9\x2d44c3\x2d9087\x2db9367f58b73c.mount: Consumed 0 CPU time Nov 16 15:11:57.787397 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-05707f1f\x2d11b9\x2d44c3\x2d9087\x2db9367f58b73c.mount: Succeeded. Nov 16 15:11:57.788209 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ipcns-05707f1f\x2d11b9\x2d44c3\x2d9087\x2db9367f58b73c.mount: Consumed 0 CPU time Nov 16 15:11:57.811334 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-05707f1f\x2d11b9\x2d44c3\x2d9087\x2db9367f58b73c.mount: Succeeded. Nov 16 15:11:57.812168 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-netns-05707f1f\x2d11b9\x2d44c3\x2d9087\x2db9367f58b73c.mount: Consumed 0 CPU time Nov 16 15:11:57.819486 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2-userdata-shm.mount: Succeeded. Nov 16 15:11:57.820334 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-containers-storage-overlay\x2dcontainers-f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2-userdata-shm.mount: Consumed 0 CPU time Nov 16 15:11:58.037175 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:58.037006234Z" level=info msg="Stopped pod sandbox: f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2" id=3075dcd9-74f9-447d-bc0d-0b34d4f158a7 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:11:58.046773 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.046707 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqcdh\" (UniqueName: \"kubernetes.io/projected/03363f87-776c-47c3-9bd8-8c0de24a4de5-kube-api-access-mqcdh\") pod \"03363f87-776c-47c3-9bd8-8c0de24a4de5\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " Nov 16 15:11:58.046773 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.046763 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-stats-auth\") pod \"03363f87-776c-47c3-9bd8-8c0de24a4de5\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " Nov 16 15:11:58.046946 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.046790 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-default-certificate\") pod \"03363f87-776c-47c3-9bd8-8c0de24a4de5\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " Nov 16 15:11:58.046946 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.046820 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-metrics-certs\") pod \"03363f87-776c-47c3-9bd8-8c0de24a4de5\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " Nov 16 15:11:58.046946 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.046853 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03363f87-776c-47c3-9bd8-8c0de24a4de5-service-ca-bundle\") pod \"03363f87-776c-47c3-9bd8-8c0de24a4de5\" (UID: \"03363f87-776c-47c3-9bd8-8c0de24a4de5\") " Nov 16 15:11:58.047226 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:11:58.047181 1563 empty_dir.go:517] Warning: Failed to clear quota on /var/lib/kubelet/pods/03363f87-776c-47c3-9bd8-8c0de24a4de5/volumes/kubernetes.io~configmap/service-ca-bundle: clearQuota called, but quotas disabled Nov 16 15:11:58.047441 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.047408 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03363f87-776c-47c3-9bd8-8c0de24a4de5-service-ca-bundle" (OuterVolumeSpecName: "service-ca-bundle") pod "03363f87-776c-47c3-9bd8-8c0de24a4de5" (UID: "03363f87-776c-47c3-9bd8-8c0de24a4de5"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Nov 16 15:11:58.055920 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.055802 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-stats-auth" (OuterVolumeSpecName: "stats-auth") pod "03363f87-776c-47c3-9bd8-8c0de24a4de5" (UID: "03363f87-776c-47c3-9bd8-8c0de24a4de5"). InnerVolumeSpecName "stats-auth". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:58.059760 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.059702 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03363f87-776c-47c3-9bd8-8c0de24a4de5-kube-api-access-mqcdh" (OuterVolumeSpecName: "kube-api-access-mqcdh") pod "03363f87-776c-47c3-9bd8-8c0de24a4de5" (UID: "03363f87-776c-47c3-9bd8-8c0de24a4de5"). InnerVolumeSpecName "kube-api-access-mqcdh". PluginName "kubernetes.io/projected", VolumeGidValue "" Nov 16 15:11:58.059760 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.059725 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-default-certificate" (OuterVolumeSpecName: "default-certificate") pod "03363f87-776c-47c3-9bd8-8c0de24a4de5" (UID: "03363f87-776c-47c3-9bd8-8c0de24a4de5"). InnerVolumeSpecName "default-certificate". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:58.060634 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.060578 1563 operation_generator.go:910] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "03363f87-776c-47c3-9bd8-8c0de24a4de5" (UID: "03363f87-776c-47c3-9bd8-8c0de24a4de5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Nov 16 15:11:58.147978 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.147931 1563 reconciler.go:300] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-metrics-certs\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:58.147978 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.147978 1563 reconciler.go:300] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03363f87-776c-47c3-9bd8-8c0de24a4de5-service-ca-bundle\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:58.147978 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.147993 1563 reconciler.go:300] "Volume detached for volume \"kube-api-access-mqcdh\" (UniqueName: \"kubernetes.io/projected/03363f87-776c-47c3-9bd8-8c0de24a4de5-kube-api-access-mqcdh\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:58.148266 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.148011 1563 reconciler.go:300] "Volume detached for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-default-certificate\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:58.148266 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.148026 1563 reconciler.go:300] "Volume detached for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/03363f87-776c-47c3-9bd8-8c0de24a4de5-stats-auth\") on node \"ropatil1611az-gs6zj-worker-westus-j4ngg\" DevicePath \"\"" Nov 16 15:11:58.296980 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-916e6bae79a818822c2ae0b2a3ee4c7eb5c6909132b5053294385cb32d9108df-merged.mount: Succeeded. Nov 16 15:11:58.297894 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-containers-storage-overlay-916e6bae79a818822c2ae0b2a3ee4c7eb5c6909132b5053294385cb32d9108df-merged.mount: Consumed 0 CPU time Nov 16 15:11:58.297916 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-03363f87\x2d776c\x2d47c3\x2d9bd8\x2d8c0de24a4de5-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dmqcdh.mount: Succeeded. Nov 16 15:11:58.298582 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-03363f87\x2d776c\x2d47c3\x2d9bd8\x2d8c0de24a4de5-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dmqcdh.mount: Consumed 0 CPU time Nov 16 15:11:58.298602 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-03363f87\x2d776c\x2d47c3\x2d9bd8\x2d8c0de24a4de5-volumes-kubernetes.io\x7esecret-metrics\x2dcerts.mount: Succeeded. Nov 16 15:11:58.299274 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-03363f87\x2d776c\x2d47c3\x2d9bd8\x2d8c0de24a4de5-volumes-kubernetes.io\x7esecret-metrics\x2dcerts.mount: Consumed 0 CPU time Nov 16 15:11:58.299291 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-03363f87\x2d776c\x2d47c3\x2d9bd8\x2d8c0de24a4de5-volumes-kubernetes.io\x7esecret-default\x2dcertificate.mount: Succeeded. Nov 16 15:11:58.299981 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-03363f87\x2d776c\x2d47c3\x2d9bd8\x2d8c0de24a4de5-volumes-kubernetes.io\x7esecret-default\x2dcertificate.mount: Consumed 0 CPU time Nov 16 15:11:58.299996 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-03363f87\x2d776c\x2d47c3\x2d9bd8\x2d8c0de24a4de5-volumes-kubernetes.io\x7esecret-stats\x2dauth.mount: Succeeded. Nov 16 15:11:58.300673 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: var-lib-kubelet-pods-03363f87\x2d776c\x2d47c3\x2d9bd8\x2d8c0de24a4de5-volumes-kubernetes.io\x7esecret-stats\x2dauth.mount: Consumed 0 CPU time Nov 16 15:11:58.353495 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.353453 1563 kubelet.go:2175] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-cff655768-nrbzb" event=&{ID:03363f87-776c-47c3-9bd8-8c0de24a4de5 Type:ContainerDied Data:f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2} Nov 16 15:11:58.353495 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.353518 1563 scope.go:110] "RemoveContainer" containerID="5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3" Nov 16 15:11:58.361785 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Removed slice libcontainer container kubepods-burstable-pod03363f87_776c_47c3_9bd8_8c0de24a4de5.slice. Nov 16 15:11:58.361848 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: kubepods-burstable-pod03363f87_776c_47c3_9bd8_8c0de24a4de5.slice: Consumed 2.010s CPU time Nov 16 15:11:58.403669 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.403594 1563 kubelet.go:2153] "SyncLoop DELETE" source="api" pods=[openshift-ingress/router-default-cff655768-nrbzb] Nov 16 15:11:58.424832 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.424788 1563 kubelet.go:2147] "SyncLoop REMOVE" source="api" pods=[openshift-ingress/router-default-cff655768-nrbzb] Nov 16 15:11:58.616681 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:58.616627486Z" level=info msg="Removing container: 5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3" id=f502930c-bcbf-4ea1-a5a6-622471a51462 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:11:58.668409 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:11:58.668362 1563 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=03363f87-776c-47c3-9bd8-8c0de24a4de5 path="/var/lib/kubelet/pods/03363f87-776c-47c3-9bd8-8c0de24a4de5/volumes" Nov 16 15:11:58.681706 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:11:58.681598230Z" level=info msg="Removed container 5780a7bbe9ae1522c6abf16f94b82f19880a5d61441ad1e67e12e440afcb65b3: openshift-ingress/router-default-cff655768-nrbzb/router" id=f502930c-bcbf-4ea1-a5a6-622471a51462 name=/runtime.v1.RuntimeService/RemoveContainer Nov 16 15:12:06.746273 ropatil1611az-gs6zj-worker-westus-j4ngg rpm-ostree[212179]: In idle state; will auto-exit in 62 seconds Nov 16 15:12:06.761603 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: rpm-ostreed.service: Succeeded. Nov 16 15:12:06.762276 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: rpm-ostreed.service: Consumed 57ms CPU time Nov 16 15:12:22.453918 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:12:22.453809 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:12:22.454453 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:12:22.454170 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:12:22.454453 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:12:22.454204 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:12:22.463996 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:12:22.470083 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:12:22.465372 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:12:22.470471 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:12:22.470442 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:13:26.470381021 +0000 UTC m=+12011.407156643 (durationBeforeRetry 1m4s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:12:22.470471 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:12:22.470471 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:12:22.470608 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rad77d09e76a84f49be0751f54d7b55f9.scope: Succeeded. Nov 16 15:12:22.471140 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rad77d09e76a84f49be0751f54d7b55f9.scope: Consumed 860us CPU time Nov 16 15:12:24.760121 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:12:24.760049702Z" level=info msg="Stopping pod sandbox: f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2" id=56922c70-e406-4707-bfdf-cb7390aca1ef name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:12:24.760121 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:12:24.760117002Z" level=info msg="Stopped pod sandbox (already stopped): f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2" id=56922c70-e406-4707-bfdf-cb7390aca1ef name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:12:24.760660 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:12:24.760376804Z" level=info msg="Removing pod sandbox: f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2" id=15a01ee5-4661-45b9-ae0f-7288637340e3 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:12:24.785885 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:12:24.785800410Z" level=info msg="Removed pod sandbox: f061c2c9a2154f002a83c167e50f45019a566b001dcda5dd73dd083d1cf5bbe2" id=15a01ee5-4661-45b9-ae0f-7288637340e3 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:12:24.786271 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:12:24.786238414Z" level=info msg="Stopping pod sandbox: 4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e" id=181132b8-73c3-453e-9a92-6bf475d69d13 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:12:24.786324 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:12:24.786276814Z" level=info msg="Stopped pod sandbox (already stopped): 4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e" id=181132b8-73c3-453e-9a92-6bf475d69d13 name=/runtime.v1.RuntimeService/StopPodSandbox Nov 16 15:12:24.786610 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:12:24.786572516Z" level=info msg="Removing pod sandbox: 4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e" id=eb84a3bf-57c0-476c-bac0-c544257df761 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:12:24.803638 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:12:24.803550254Z" level=info msg="Removed pod sandbox: 4d3ffe9187da797c2ef4ab3aa7c4f71d4d3d42142daa556f32d1e2b941fa4f9e" id=eb84a3bf-57c0-476c-bac0-c544257df761 name=/runtime.v1.RuntimeService/RemovePodSandbox Nov 16 15:13:20.514791 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:13:20.514585094Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=42093f1b-ff89-4104-931f-26886fe031a2 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:13:20.535634 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:13:20.514935297Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=42093f1b-ff89-4104-931f-26886fe031a2 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:13:24.445244 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b-runc.6Tmri4.mount: Succeeded. Nov 16 15:13:26.505264 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:13:26.505086 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:13:26.505843 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:13:26.505501 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:13:26.505843 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:13:26.505558 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:13:26.515738 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:13:26.516750 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:13:26.516721 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:13:26.520525 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:13:26.520370 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:15:28.520345096 +0000 UTC m=+12133.457120818 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:13:26.520525 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:13:26.520525 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:13:26.521692 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rda8251fc82494fdc8fc66e7d3f632399.scope: Succeeded. Nov 16 15:13:26.522295 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rda8251fc82494fdc8fc66e7d3f632399.scope: Consumed 828us CPU time Nov 16 15:15:28.533702 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:15:28.533610 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:15:28.534422 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:15:28.533866 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:15:28.534422 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:15:28.533896 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:15:28.544314 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:15:28.545335 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:15:28.545303 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:15:28.549057 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:15:28.548972 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:17:30.548946666 +0000 UTC m=+12255.485722388 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:15:28.549057 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:15:28.549057 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:15:28.549980 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rafba6e8892f6435a8f664e923a676b9a.scope: Succeeded. Nov 16 15:15:28.550597 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rafba6e8892f6435a8f664e923a676b9a.scope: Consumed 835us CPU time Nov 16 15:17:03.237470 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:17:03.237376 1563 logs.go:401] "Finished parsing log file, hit bytes limit" path="/var/log/pods/openshift-sdn_sdn-tqwqj_4dbb404b-812a-4297-b9ee-2cefe58d83ae/sdn/0.log" limit=65536 Nov 16 15:17:03.421832 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:17:03.421793 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-tqwqj_4dbb404b-812a-4297-b9ee-2cefe58d83ae/kube-rbac-proxy/0.log" Nov 16 15:17:03.623455 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:17:03.623370 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-tqwqj_4dbb404b-812a-4297-b9ee-2cefe58d83ae/drop-icmp/0.log" Nov 16 15:17:06.655659 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:17:06.655537 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_machine-config-daemon-q7lkj_29d0f8e3-e33b-4c9f-a290-193626492310/machine-config-daemon/0.log" Nov 16 15:17:06.876850 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:17:06.876807 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_machine-config-daemon-q7lkj_29d0f8e3-e33b-4c9f-a290-193626492310/oauth-proxy/0.log" Nov 16 15:17:30.576896 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:17:30.576634 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:17:30.577414 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:17:30.576997 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:17:30.577414 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:17:30.577028 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:17:30.587921 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:17:30.588998 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:17:30.588929 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:17:30.592492 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:17:30.592459 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:19:32.592436869 +0000 UTC m=+12377.529212491 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:17:30.592492 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:17:30.592492 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:17:30.593352 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r622f7cc9607e4283b5efff64cd6c7c8f.scope: Succeeded. Nov 16 15:17:30.593960 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r622f7cc9607e4283b5efff64cd6c7c8f.scope: Consumed 854us CPU time Nov 16 15:18:20.523470 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:18:20.523305717Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=77c80f4e-fbf3-4458-8223-928b8f3190a0 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:18:20.547216 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:18:20.523578019Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=77c80f4e-fbf3-4458-8223-928b8f3190a0 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:19:32.640768 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:19:32.640681 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:19:32.641591 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:19:32.640914 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:19:32.641591 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:19:32.640941 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:19:32.651600 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:19:32.652768 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:19:32.652682 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:19:32.656627 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:19:32.656500 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:21:34.656473934 +0000 UTC m=+12499.593249556 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:19:32.656627 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:19:32.656627 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:19:32.657389 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-re5a980be5c7946d18054037779814a88.scope: Succeeded. Nov 16 15:19:32.657902 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-re5a980be5c7946d18054037779814a88.scope: Consumed 921us CPU time Nov 16 15:21:34.694604 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:21:34.694454 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:21:34.695388 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:21:34.694801 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:21:34.695388 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:21:34.694826 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:21:34.705755 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:21:34.706792 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:21:34.706759 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:21:34.710992 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:21:34.710914 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:23:36.710870849 +0000 UTC m=+12621.647646471 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:21:34.710992 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:21:34.710992 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:21:34.711611 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r1dc8349a03b548a495f8e7107245feed.scope: Succeeded. Nov 16 15:21:34.712141 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r1dc8349a03b548a495f8e7107245feed.scope: Consumed 858us CPU time Nov 16 15:23:20.532353 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:23:20.532082664Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=33b6f95d-2e6d-40ba-8e8f-1cac6d0e4626 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:23:20.549612 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:23:20.532504168Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=33b6f95d-2e6d-40ba-8e8f-1cac6d0e4626 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:23:36.761297 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:23:36.761196 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:23:36.761901 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:23:36.761476 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:23:36.761901 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:23:36.761508 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:23:36.771991 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:23:36.775401 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:23:36.775364 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:23:36.779210 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:23:36.779169 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:25:38.779099788 +0000 UTC m=+12743.715875410 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:23:36.779210 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:23:36.779210 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:23:36.780169 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rf4e1bdb7f2b04c4581a700a8bd65feda.scope: Succeeded. Nov 16 15:23:36.780713 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rf4e1bdb7f2b04c4581a700a8bd65feda.scope: Consumed 2ms CPU time Nov 16 15:25:38.822570 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:25:38.822526 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:25:38.823310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:25:38.822808 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:25:38.823310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:25:38.822837 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:25:38.833821 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:25:38.835002 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:25:38.834871 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:25:38.839197 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:25:38.839105 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:27:40.839078592 +0000 UTC m=+12865.775854314 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:25:38.839197 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:25:38.839197 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:25:38.839925 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r4c45f16580444f868ef2aef91e3841ce.scope: Succeeded. Nov 16 15:25:38.840650 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r4c45f16580444f868ef2aef91e3841ce.scope: Consumed 878us CPU time Nov 16 15:27:40.876272 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:27:40.876062 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:27:40.877027 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:27:40.876409 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:27:40.877027 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:27:40.876454 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:27:40.886923 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:27:40.888113 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:27:40.888037 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:27:40.892073 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:27:40.892033 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:29:42.891913466 +0000 UTC m=+12987.828689088 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:27:40.892073 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:27:40.892073 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:27:40.892935 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r07cccff241e64a939a0ab594fcdb3a53.scope: Succeeded. Nov 16 15:27:40.893542 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r07cccff241e64a939a0ab594fcdb3a53.scope: Consumed 910us CPU time Nov 16 15:28:20.537812 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:28:20.537635533Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=bc0ac23a-cef6-4a72-90da-7a184adba0d6 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:28:20.558916 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:28:20.537945036Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=bc0ac23a-cef6-4a72-90da-7a184adba0d6 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:29:42.909748 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:29:42.909550 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:29:42.910430 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:29:42.909899 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:29:42.910430 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:29:42.909923 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:29:42.919521 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:29:42.920682 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:29:42.920616 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:29:42.924553 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:29:42.924487 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:31:44.924432679 +0000 UTC m=+13109.861208301 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:29:42.924553 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:29:42.924553 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:29:42.925051 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r7d10c81cc1e3488990ce646477c4bd55.scope: Succeeded. Nov 16 15:29:42.925597 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r7d10c81cc1e3488990ce646477c4bd55.scope: Consumed 917us CPU time Nov 16 15:30:19.446659 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b-runc.LM6Hhf.mount: Succeeded. Nov 16 15:31:44.954663 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:31:44.954568 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:31:44.955313 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:31:44.954801 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:31:44.955313 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:31:44.954830 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:31:44.964297 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:31:44.965246 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:31:44.965209 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:31:44.969086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:31:44.969057 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:33:46.969032996 +0000 UTC m=+13231.905808618 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:31:44.969086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:31:44.969086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:31:44.969591 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rfb9105a735d54ea49e340444f36700c1.scope: Succeeded. Nov 16 15:31:44.970254 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rfb9105a735d54ea49e340444f36700c1.scope: Consumed 811us CPU time Nov 16 15:33:20.543823 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:33:20.543664126Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=82efdf6d-f315-445a-9d7a-a0eb88211555 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:33:20.563310 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:33:20.544004529Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=82efdf6d-f315-445a-9d7a-a0eb88211555 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:33:46.992336 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:33:46.992197 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:33:46.992841 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:33:46.992461 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:33:46.992841 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:33:46.992580 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:33:47.003213 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:33:47.004195 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:33:47.004083 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:33:47.007877 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:33:47.007836 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:35:49.007809588 +0000 UTC m=+13353.944585310 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:33:47.007877 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:33:47.007877 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:33:47.008696 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r913320d1da2f451dbb2fe5ded8ff3584.scope: Succeeded. Nov 16 15:33:47.009242 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r913320d1da2f451dbb2fe5ded8ff3584.scope: Consumed 749us CPU time Nov 16 15:35:49.011267 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:35:49.011202 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:35:49.011952 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:35:49.011495 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:35:49.011952 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:35:49.011526 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:35:49.021268 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:35:49.022365 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:35:49.022329 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:35:49.026195 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:35:49.026140 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:37:51.026100684 +0000 UTC m=+13475.962876406 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:35:49.026195 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:35:49.026195 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:35:49.026793 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r52ef942dae974aed9bd53b434845c37e.scope: Succeeded. Nov 16 15:35:49.027480 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r52ef942dae974aed9bd53b434845c37e.scope: Consumed 823us CPU time Nov 16 15:37:51.109797 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:37:51.109735 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:37:51.110534 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:37:51.110003 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:37:51.110534 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:37:51.110033 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:37:51.120265 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:37:51.121396 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:37:51.121300 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:37:51.125618 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:37:51.125590 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:39:53.125567026 +0000 UTC m=+13598.062342648 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:37:51.125618 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:37:51.125618 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:37:51.126335 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rd8b0970fafb9418ba6157c7b934dc43c.scope: Succeeded. Nov 16 15:37:51.126900 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rd8b0970fafb9418ba6157c7b934dc43c.scope: Consumed 919us CPU time Nov 16 15:38:20.550662 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:38:20.550410891Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=5c803938-8cf5-41c8-8f1d-9cbeca1e5a93 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:38:20.650664 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:38:20.550727493Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=5c803938-8cf5-41c8-8f1d-9cbeca1e5a93 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:39:53.133441 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:39:53.133299 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:39:53.134107 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:39:53.133623 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:39:53.134107 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:39:53.133652 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:39:53.144370 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:39:53.145376 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:39:53.145346 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:39:53.149174 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:39:53.149065 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:41:55.149040838 +0000 UTC m=+13720.085816460 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:39:53.149174 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:39:53.149174 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:39:53.149900 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r0636e7c9a6164d778feac6ea5425bd8c.scope: Succeeded. Nov 16 15:39:53.150593 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r0636e7c9a6164d778feac6ea5425bd8c.scope: Consumed 845us CPU time Nov 16 15:41:55.175237 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:41:55.175100 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:41:55.176005 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:41:55.175431 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:41:55.176005 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:41:55.175479 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:41:55.185822 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:41:55.186906 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:41:55.186815 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:41:55.190619 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:41:55.190552 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:43:57.190526891 +0000 UTC m=+13842.127302513 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:41:55.190619 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:41:55.190619 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:41:55.191370 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r51d843a389e04b7fa37e380ccd8d3625.scope: Succeeded. Nov 16 15:41:55.191937 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r51d843a389e04b7fa37e380ccd8d3625.scope: Consumed 832us CPU time Nov 16 15:41:59.445973 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b-runc.K2xEpV.mount: Succeeded. Nov 16 15:43:20.556151 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:43:20.555943334Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=3b44638d-f4db-4c88-8614-9d08f84de894 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:43:20.578954 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:43:20.556278136Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=3b44638d-f4db-4c88-8614-9d08f84de894 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:43:57.230156 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:43:57.230098 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:43:57.230933 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:43:57.230334 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:43:57.230933 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:43:57.230383 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:43:57.240363 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:43:57.241320 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:43:57.241287 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:43:57.244986 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:43:57.244941 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:45:59.24491603 +0000 UTC m=+13964.181691752 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:43:57.244986 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:43:57.244986 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:43:57.245639 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rb5502d2eefb54e8b920378bbc1b37599.scope: Succeeded. Nov 16 15:43:57.246108 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rb5502d2eefb54e8b920378bbc1b37599.scope: Consumed 802us CPU time Nov 16 15:45:59.273733 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:45:59.273545 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:45:59.274525 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:45:59.273831 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:45:59.274525 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:45:59.273877 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:45:59.283088 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:45:59.284448 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:45:59.284419 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:45:59.288252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:45:59.288220 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:48:01.288178674 +0000 UTC m=+14086.224954296 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:45:59.288252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:45:59.288252 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:45:59.288815 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r23d257010b184ec7a74c9de1c8ae9462.scope: Succeeded. Nov 16 15:45:59.289360 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r23d257010b184ec7a74c9de1c8ae9462.scope: Consumed 991us CPU time Nov 16 15:48:01.300995 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:48:01.300927 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:48:01.301844 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:48:01.301223 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:48:01.301844 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:48:01.301273 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:48:01.311750 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:48:01.312855 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:48:01.312809 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:48:01.316533 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:48:01.316496 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:50:03.316469183 +0000 UTC m=+14208.253244805 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:48:01.316533 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:48:01.316533 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:48:01.317370 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r4dc25d53ad444cf7a8eec29964eea50a.scope: Succeeded. Nov 16 15:48:01.317866 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r4dc25d53ad444cf7a8eec29964eea50a.scope: Consumed 908us CPU time Nov 16 15:48:20.562306 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:48:20.562055163Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=ca9f66c4-369d-4dce-856c-97fe41761767 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:48:20.642969 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:48:20.562387166Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=ca9f66c4-369d-4dce-856c-97fe41761767 name=/runtime.v1.ImageService/ImageStatus Nov 16 15:50:03.406544 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:50:03.406471 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:50:03.407470 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:50:03.406889 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:50:03.407470 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:50:03.406915 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:50:03.416824 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:50:03.417933 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:50:03.417877 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:50:03.421587 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:50:03.421525 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:52:05.421487478 +0000 UTC m=+14330.358263200 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:50:03.421587 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:50:03.421587 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:50:03.422100 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r7c23b1bb4885479797306702db3b672d.scope: Succeeded. Nov 16 15:50:03.422678 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r7c23b1bb4885479797306702db3b672d.scope: Consumed 873us CPU time Nov 16 15:50:34.509459 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b-runc.FyRkq3.mount: Succeeded. Nov 16 15:52:05.462549 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:52:05.462413 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:52:05.463398 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:52:05.462766 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:52:05.463398 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:52:05.462811 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:52:05.473179 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:52:05.474294 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:52:05.474260 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:52:05.478030 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:52:05.477962 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:54:07.47793879 +0000 UTC m=+14452.414714512 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:52:05.478030 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:52:05.478030 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:52:05.478907 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ree73d430ebf6499f93574828fb6d5679.scope: Succeeded. Nov 16 15:52:05.479457 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ree73d430ebf6499f93574828fb6d5679.scope: Consumed 989us CPU time Nov 16 15:53:20.567816 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:53:20.567642120Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=527b1b80-f059-4e18-b6ef-d3108d9339fc name=/runtime.v1.ImageService/ImageStatus Nov 16 15:53:20.599250 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:53:20.567925823Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=527b1b80-f059-4e18-b6ef-d3108d9339fc name=/runtime.v1.ImageService/ImageStatus Nov 16 15:54:07.527488 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:54:07.527442 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:54:07.528298 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:54:07.527715 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:54:07.528298 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:54:07.527748 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:54:07.537976 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:54:07.539148 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:54:07.539103 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:54:07.542897 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:54:07.542827 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:56:09.542786797 +0000 UTC m=+14574.479562419 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:54:07.542897 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:54:07.542897 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:54:07.544108 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rcc5680eca5254096a0cbcc2cf6686819.scope: Succeeded. Nov 16 15:54:07.544636 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rcc5680eca5254096a0cbcc2cf6686819.scope: Consumed 969us CPU time Nov 16 15:54:09.446228 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b-runc.WEsMTH.mount: Succeeded. Nov 16 15:55:59.236913 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00312|connmgr|INFO|br0<->unix#718: 22 flow_mods in the last 0 s (22 deletes) Nov 16 15:55:59.259246 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00313|connmgr|INFO|br0<->unix#721: 2 flow_mods in the last 0 s (2 deletes) Nov 16 15:56:09.555812 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:56:09.555624 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:56:09.556512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:56:09.555921 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:56:09.556512 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:56:09.555970 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:56:09.566257 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:56:09.567266 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:56:09.567217 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:56:09.571200 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:56:09.571115 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 15:58:11.571093282 +0000 UTC m=+14696.507869004 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:56:09.571200 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:56:09.571200 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:56:09.571790 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r76052996348d4975a088440ee4e83657.scope: Succeeded. Nov 16 15:56:09.572317 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r76052996348d4975a088440ee4e83657.scope: Consumed 829us CPU time Nov 16 15:58:11.587893 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:58:11.587754 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 15:58:11.588736 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:58:11.588019 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 15:58:11.588736 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 15:58:11.588065 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 15:58:11.598247 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 15:58:11.599250 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 15:58:11.599225 1563 mount_linux.go:222] Detected OS with systemd Nov 16 15:58:11.603077 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 15:58:11.602948 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:00:13.602925687 +0000 UTC m=+14818.539701409 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 15:58:11.603077 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 15:58:11.603077 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 15:58:11.603941 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r3c48da1c1e2c49f9aa8ccda179a4e05c.scope: Succeeded. Nov 16 15:58:11.604639 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r3c48da1c1e2c49f9aa8ccda179a4e05c.scope: Consumed 807us CPU time Nov 16 15:58:20.573782 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:58:20.573531965Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=f9c697dc-7c94-4251-afc2-bc81c54fa08f name=/runtime.v1.ImageService/ImageStatus Nov 16 15:58:20.692575 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 15:58:20.573834068Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=f9c697dc-7c94-4251-afc2-bc81c54fa08f name=/runtime.v1.ImageService/ImageStatus Nov 16 16:00:13.639720 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:00:13.639592 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:00:13.640454 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:00:13.639894 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:00:13.640454 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:00:13.639924 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:00:13.650821 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:00:13.651858 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:00:13.651784 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:00:13.655528 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:00:13.655494 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:02:15.655468798 +0000 UTC m=+14940.592244420 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:00:13.655528 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:00:13.655528 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:00:13.656171 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r472d0b86203f40b0bb11b17deaa3814c.scope: Succeeded. Nov 16 16:00:13.656793 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r472d0b86203f40b0bb11b17deaa3814c.scope: Consumed 825us CPU time Nov 16 16:02:15.691651 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:02:15.691412 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:02:15.692537 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:02:15.691765 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:02:15.692537 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:02:15.691796 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:02:15.705208 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:02:15.706284 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:02:15.706221 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:02:15.710163 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:02:15.710064 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:04:17.71003816 +0000 UTC m=+15062.646813782 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:02:15.710163 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:02:15.710163 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:02:15.710828 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r9a752c707f5c47218d125f211d52e61d.scope: Succeeded. Nov 16 16:02:15.711425 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r9a752c707f5c47218d125f211d52e61d.scope: Consumed 881us CPU time Nov 16 16:03:20.580280 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:03:20.580083580Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=fc7f8942-4dc2-4560-8a46-7ceff7e12499 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:03:20.604170 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:03:20.580412183Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=fc7f8942-4dc2-4560-8a46-7ceff7e12499 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:04:17.807458 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:04:17.807388 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:04:17.808164 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:04:17.807710 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:04:17.808164 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:04:17.807740 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:04:17.817832 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:04:17.818956 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:04:17.818886 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:04:17.822426 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:04:17.822395 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:06:19.822368912 +0000 UTC m=+15184.759144634 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:04:17.822426 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:04:17.822426 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:04:17.823270 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r46949599c23f493e892057036b0f7128.scope: Succeeded. Nov 16 16:04:17.823892 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r46949599c23f493e892057036b0f7128.scope: Consumed 893us CPU time Nov 16 16:06:19.828793 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:06:19.828681 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:06:19.829521 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:06:19.828940 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:06:19.829521 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:06:19.828969 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:06:19.839075 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:06:19.840263 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:06:19.840222 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:06:19.844004 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:06:19.843940 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:08:21.843917326 +0000 UTC m=+15306.780693048 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:06:19.844004 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:06:19.844004 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:06:19.845077 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rf7711f4ce80d43479542cd5ecbb9ae56.scope: Succeeded. Nov 16 16:06:19.845863 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rf7711f4ce80d43479542cd5ecbb9ae56.scope: Consumed 896us CPU time Nov 16 16:08:20.586234 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:08:20.586111060Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=8746bf1a-04d0-43f0-88a6-8f0989530818 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:08:20.601758 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:08:20.586445162Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=8746bf1a-04d0-43f0-88a6-8f0989530818 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:08:21.876107 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:08:21.876050 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:08:21.876770 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:08:21.876442 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:08:21.876770 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:08:21.876474 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:08:21.886812 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:08:21.887918 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:08:21.887878 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:08:21.892770 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:08:21.892729 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:10:23.892703023 +0000 UTC m=+15428.829478645 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:08:21.892770 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:08:21.892770 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:08:21.892782 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r7dc3871c74b04888832af2fc11f25d87.scope: Succeeded. Nov 16 16:08:21.893509 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r7dc3871c74b04888832af2fc11f25d87.scope: Consumed 921us CPU time Nov 16 16:10:23.930584 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:10:23.930472 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:10:23.931444 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:10:23.930836 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:10:23.931444 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:10:23.930901 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:10:23.941066 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:10:23.942021 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:10:23.941989 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:10:23.945864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:10:23.945793 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:12:25.945769258 +0000 UTC m=+15550.882544880 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:10:23.945864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:10:23.945864 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:10:23.946645 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r1401567c04b64e1c856b96b42f87df26.scope: Succeeded. Nov 16 16:10:23.947249 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r1401567c04b64e1c856b96b42f87df26.scope: Consumed 739us CPU time Nov 16 16:12:25.977596 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:12:25.977540 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:12:25.978353 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:12:25.977921 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:12:25.978353 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:12:25.977956 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:12:25.987651 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:12:25.988775 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:12:25.988742 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:12:25.992498 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:12:25.992448 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:14:27.992427243 +0000 UTC m=+15672.929202965 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:12:25.992498 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:12:25.992498 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:12:25.993409 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r8bf3d6b439004b8b86882014637b5287.scope: Succeeded. Nov 16 16:12:25.993918 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r8bf3d6b439004b8b86882014637b5287.scope: Consumed 921us CPU time Nov 16 16:13:20.592434 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:13:20.592257942Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=ad392129-caf6-4817-aa4f-2eb93d1cf380 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:13:20.609715 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:13:20.592525544Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=ad392129-caf6-4817-aa4f-2eb93d1cf380 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:14:28.031063 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:14:28.030929 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:14:28.031665 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:14:28.031240 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:14:28.031665 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:14:28.031268 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:14:28.041692 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:14:28.042807 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:14:28.042719 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:14:28.046704 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:14:28.046633 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:16:30.046575101 +0000 UTC m=+15794.983350723 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:14:28.046704 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:14:28.046704 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:14:28.047618 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r507eda163cc247cc8720dfbff241b454.scope: Succeeded. Nov 16 16:14:28.048159 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r507eda163cc247cc8720dfbff241b454.scope: Consumed 906us CPU time Nov 16 16:16:30.052014 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:16:30.051962 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:16:30.052654 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:16:30.052259 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:16:30.052654 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:16:30.052291 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:16:30.063793 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:16:30.064850 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:16:30.064818 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:16:30.068656 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:16:30.068627 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:18:32.068598863 +0000 UTC m=+15917.005374485 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:16:30.068656 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:16:30.068656 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:16:30.069470 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r3d78d52abc9a456f971fcd5996c888b4.scope: Succeeded. Nov 16 16:16:30.069953 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r3d78d52abc9a456f971fcd5996c888b4.scope: Consumed 910us CPU time Nov 16 16:18:20.596815 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:18:20.596652960Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=9fe2c6ec-cc0a-437a-a3d6-09502297ec76 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:18:20.618728 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:18:20.596926062Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=9fe2c6ec-cc0a-437a-a3d6-09502297ec76 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:18:32.091870 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:18:32.091809 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:18:32.092663 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:18:32.092121 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:18:32.092663 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:18:32.092211 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:18:32.102894 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:18:32.103927 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:18:32.103894 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:18:32.107671 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:18:32.107596 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:20:34.10753512 +0000 UTC m=+16039.044310742 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:18:32.107671 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:18:32.107671 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:18:32.108315 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rb5989b72f9d14d4396c31443cfb0e291.scope: Succeeded. Nov 16 16:18:32.108805 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rb5989b72f9d14d4396c31443cfb0e291.scope: Consumed 836us CPU time Nov 16 16:18:57.715602 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:18:57.715461 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r2xl8_23402384-5d8d-47cb-9b23-e7128b9e0e74/dns/0.log" Nov 16 16:18:57.755467 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:18:57.755392 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r2xl8_23402384-5d8d-47cb-9b23-e7128b9e0e74/kube-rbac-proxy/0.log" Nov 16 16:18:57.836551 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:18:57.836511 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-27gwx_8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2/dns-node-resolver/0.log" Nov 16 16:19:01.055620 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:01.055550 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dg87v_aacb53e4-6716-4337-a1ae-4ac3a857cbb0/node-ca/0.log" Nov 16 16:19:02.236544 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:02.236460 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4zcm6_b2808f6f-6018-4fb1-8229-65770d542c6d/serve-healthcheck-canary/0.log" Nov 16 16:19:14.407035 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:14.406995 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_machine-config-daemon-q7lkj_29d0f8e3-e33b-4c9f-a290-193626492310/machine-config-daemon/0.log" Nov 16 16:19:14.435825 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:14.435787 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_machine-config-daemon-q7lkj_29d0f8e3-e33b-4c9f-a290-193626492310/oauth-proxy/0.log" Nov 16 16:19:19.065740 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:19.065649 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kxmx6_89cfb802-14c7-4c79-bc9c-015efafee773/node-exporter/0.log" Nov 16 16:19:19.090074 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:19.090038 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kxmx6_89cfb802-14c7-4c79-bc9c-015efafee773/kube-rbac-proxy/0.log" Nov 16 16:19:19.113993 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:19.113948 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kxmx6_89cfb802-14c7-4c79-bc9c-015efafee773/init-textfile/0.log" Nov 16 16:19:23.626183 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:23.626123 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/kube-multus-additional-cni-plugins/0.log" Nov 16 16:19:23.653702 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:23.653673 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/egress-router-binary-copy/0.log" Nov 16 16:19:23.675914 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:23.675872 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/cni-plugins/0.log" Nov 16 16:19:23.701596 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:23.701562 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/bond-cni-plugin/0.log" Nov 16 16:19:23.725308 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:23.725268 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/routeoverride-cni/0.log" Nov 16 16:19:23.755761 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:23.755710 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/whereabouts-cni-bincopy/0.log" Nov 16 16:19:23.782883 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:23.782837 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/whereabouts-cni/0.log" Nov 16 16:19:24.080619 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:24.080475 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vm4st_f2b609ff-b0ed-4929-b767-953432cca02b/kube-multus/0.log" Nov 16 16:19:24.327152 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:24.327107 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-q8qt5_bc424839-f613-4159-bb7f-0c1769bf6200/network-metrics-daemon/0.log" Nov 16 16:19:24.352572 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:24.352535 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-q8qt5_bc424839-f613-4159-bb7f-0c1769bf6200/kube-rbac-proxy/0.log" Nov 16 16:19:26.110268 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:26.110194 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-tqwqj_4dbb404b-812a-4297-b9ee-2cefe58d83ae/sdn/0.log" Nov 16 16:19:26.133230 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:26.133193 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-tqwqj_4dbb404b-812a-4297-b9ee-2cefe58d83ae/kube-rbac-proxy/0.log" Nov 16 16:19:26.160514 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:26.160474 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-tqwqj_4dbb404b-812a-4297-b9ee-2cefe58d83ae/drop-icmp/0.log" Nov 16 16:19:27.317180 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:27.317114 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-b5srf_a77123b5-b068-4e08-bbb0-243a4ef22749/network-check-target-container/0.log" Nov 16 16:19:29.138874 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:29.138815 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-cvdsv_279f55b2-5534-42ae-ba2d-588e2c3a3ffe/tuned/0.log" Nov 16 16:19:36.427168 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:36.427090 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-disk-csi-driver-node-p82p6_062fe84e-408b-4970-94b8-5df0ae82ba87/csi-driver/0.log" Nov 16 16:19:36.453876 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:36.453800 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-disk-csi-driver-node-p82p6_062fe84e-408b-4970-94b8-5df0ae82ba87/csi-node-driver-registrar/0.log" Nov 16 16:19:36.481602 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:36.481571 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-disk-csi-driver-node-p82p6_062fe84e-408b-4970-94b8-5df0ae82ba87/csi-liveness-probe/0.log" Nov 16 16:19:36.503963 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:36.503928 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-disk-csi-driver-node-p82p6_062fe84e-408b-4970-94b8-5df0ae82ba87/azure-inject-credentials/0.log" Nov 16 16:19:37.770995 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:37.770918 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-file-csi-driver-node-pzz7z_6037f731-36d1-49c8-a598-34b98d642a8e/csi-driver/0.log" Nov 16 16:19:37.793628 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:37.793595 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-file-csi-driver-node-pzz7z_6037f731-36d1-49c8-a598-34b98d642a8e/csi-node-driver-registrar/0.log" Nov 16 16:19:37.823092 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:37.823064 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-file-csi-driver-node-pzz7z_6037f731-36d1-49c8-a598-34b98d642a8e/csi-liveness-probe/0.log" Nov 16 16:19:37.849477 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:19:37.849447 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-file-csi-driver-node-pzz7z_6037f731-36d1-49c8-a598-34b98d642a8e/azure-inject-credentials/0.log" Nov 16 16:20:34.152350 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:20:34.152177 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:20:34.152990 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:20:34.152411 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:20:34.152990 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:20:34.152443 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:20:34.162602 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:20:34.163682 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:20:34.163617 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:20:34.167889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:20:34.167826 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:22:36.167800436 +0000 UTC m=+16161.104576058 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:20:34.167889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:20:34.167889 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:20:34.168227 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rd8950ca899234f67a184e65a1386b90d.scope: Succeeded. Nov 16 16:20:34.168732 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rd8950ca899234f67a184e65a1386b90d.scope: Consumed 882us CPU time Nov 16 16:22:36.193385 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:22:36.193333 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:22:36.194083 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:22:36.193564 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:22:36.194083 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:22:36.193594 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:22:36.203295 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:22:36.204239 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:22:36.204208 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:22:36.207980 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:22:36.207943 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:24:38.207917374 +0000 UTC m=+16283.144692996 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:22:36.207980 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:22:36.207980 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:22:36.208816 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r3236ac1ccbd64a538c9c1aae0fef4a83.scope: Succeeded. Nov 16 16:22:36.209338 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r3236ac1ccbd64a538c9c1aae0fef4a83.scope: Consumed 779us CPU time Nov 16 16:23:20.601848 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:23:20.601682680Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=92c7fb04-bd0a-477e-9f59-ac41e4de29e8 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:23:20.620723 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:23:20.601956582Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=92c7fb04-bd0a-477e-9f59-ac41e4de29e8 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:24:38.234218 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:24:38.234080 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:24:38.234923 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:24:38.234434 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:24:38.234923 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:24:38.234481 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:24:38.245648 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:24:38.246675 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:24:38.246616 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:24:38.250249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:24:38.250222 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:26:40.250182008 +0000 UTC m=+16405.186957630 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:24:38.250249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:24:38.250249 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:24:38.250992 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r7dd9ebd69bca43d88d66fbcfb49ade26.scope: Succeeded. Nov 16 16:24:38.251556 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r7dd9ebd69bca43d88d66fbcfb49ade26.scope: Consumed 816us CPU time Nov 16 16:26:40.275396 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:26:40.275244 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:26:40.276180 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:26:40.275557 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:26:40.276180 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:26:40.275606 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:26:40.285381 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:26:40.286723 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:26:40.286630 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:26:40.290585 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:26:40.290500 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:28:42.29046186 +0000 UTC m=+16527.227237582 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:26:40.290585 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:26:40.290585 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:26:40.291582 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rba33d7ad15df45878cecc6fc7bf7597e.scope: Succeeded. Nov 16 16:26:40.292112 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rba33d7ad15df45878cecc6fc7bf7597e.scope: Consumed 1ms CPU time Nov 16 16:28:20.606936 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:28:20.606751917Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=16b1641f-cd2f-452e-990b-6a2c341a1bad name=/runtime.v1.ImageService/ImageStatus Nov 16 16:28:20.638040 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:28:20.607017019Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=16b1641f-cd2f-452e-990b-6a2c341a1bad name=/runtime.v1.ImageService/ImageStatus Nov 16 16:28:42.334534 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:28:42.334437 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:28:42.335111 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:28:42.334673 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:28:42.335111 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:28:42.334704 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:28:42.344422 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:28:42.345469 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:28:42.345402 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:28:42.349351 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:28:42.349318 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:30:44.349290977 +0000 UTC m=+16649.286066599 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:28:42.349351 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:28:42.349351 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:28:42.349972 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r747b55eb9c87442b97c977baba83fc9e.scope: Succeeded. Nov 16 16:28:42.350481 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r747b55eb9c87442b97c977baba83fc9e.scope: Consumed 888us CPU time Nov 16 16:30:44.397027 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:30:44.396706 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:30:44.397743 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:30:44.397123 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:30:44.397743 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:30:44.397212 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:30:44.407933 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:30:44.409034 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:30:44.409001 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:30:44.413021 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:30:44.412947 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:32:46.412908017 +0000 UTC m=+16771.349683639 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:30:44.413021 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:30:44.413021 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:30:44.413793 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r7eb44b6ca119448abeb0e6cbeea751d5.scope: Succeeded. Nov 16 16:30:44.414369 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r7eb44b6ca119448abeb0e6cbeea751d5.scope: Consumed 894us CPU time Nov 16 16:30:44.444246 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b-runc.ZoQOiQ.mount: Succeeded. Nov 16 16:32:46.452527 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:32:46.452427 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:32:46.453123 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:32:46.452669 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:32:46.453123 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:32:46.452700 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:32:46.463044 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:32:46.464085 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:32:46.464038 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:32:46.467955 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:32:46.467905 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:34:48.467879235 +0000 UTC m=+16893.404654957 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:32:46.467955 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:32:46.467955 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:32:46.469015 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r44803499cf9640c3bc780dc66bcd7aba.scope: Succeeded. Nov 16 16:32:46.469520 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r44803499cf9640c3bc780dc66bcd7aba.scope: Consumed 795us CPU time Nov 16 16:33:20.611473 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:33:20.611322389Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=0ece818b-56f1-4134-9585-f0f2c28c2b1a name=/runtime.v1.ImageService/ImageStatus Nov 16 16:33:20.619568 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:33:20.611607391Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=0ece818b-56f1-4134-9585-f0f2c28c2b1a name=/runtime.v1.ImageService/ImageStatus Nov 16 16:34:24.448855 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b-runc.rZxMQb.mount: Succeeded. Nov 16 16:34:48.517671 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:34:48.517561 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:34:48.518436 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:34:48.517950 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:34:48.518436 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:34:48.517981 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:34:48.528983 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:34:48.530304 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:34:48.530273 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:34:48.533927 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:34:48.533895 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:36:50.533870885 +0000 UTC m=+17015.470646607 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:34:48.533927 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:34:48.533927 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:34:48.534930 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rd022ca60d656436e8cd73bc68e7ff291.scope: Succeeded. Nov 16 16:34:48.535417 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rd022ca60d656436e8cd73bc68e7ff291.scope: Consumed 1ms CPU time Nov 16 16:36:50.567163 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:36:50.567009 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:36:50.567700 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:36:50.567296 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:36:50.567700 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:36:50.567326 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:36:50.576854 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:36:50.577695 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:36:50.577631 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:36:50.581342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:36:50.581298 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:38:52.581273318 +0000 UTC m=+17137.518048940 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:36:50.581342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:36:50.581342 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:36:50.582331 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-re866e7cc681e4dda95cdd15d50bb7dc4.scope: Succeeded. Nov 16 16:36:50.582857 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-re866e7cc681e4dda95cdd15d50bb7dc4.scope: Consumed 678us CPU time Nov 16 16:38:20.617172 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:38:20.616970777Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=2d5bd17c-3ebc-4afe-9caa-7d334be0ddad name=/runtime.v1.ImageService/ImageStatus Nov 16 16:38:20.636070 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:38:20.617292479Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=2d5bd17c-3ebc-4afe-9caa-7d334be0ddad name=/runtime.v1.ImageService/ImageStatus Nov 16 16:38:52.601407 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:38:52.601359 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:38:52.602029 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:38:52.601606 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:38:52.602029 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:38:52.601635 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:38:52.611624 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:38:52.612787 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:38:52.612716 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:38:52.616340 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:38:52.616310 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:40:54.616286253 +0000 UTC m=+17259.553061975 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:38:52.616340 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:38:52.616340 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:38:52.617121 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r64c325aa1b6145edac2d5a139709d352.scope: Succeeded. Nov 16 16:38:52.617883 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r64c325aa1b6145edac2d5a139709d352.scope: Consumed 942us CPU time Nov 16 16:40:54.650440 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:40:54.650375 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:40:54.651204 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:40:54.650618 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:40:54.651204 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:40:54.650654 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:40:54.660145 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:40:54.661257 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:40:54.661225 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:40:54.665205 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:40:54.665097 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:42:56.665073056 +0000 UTC m=+17381.601848678 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:40:54.665205 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:40:54.665205 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:40:54.665938 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r40e56bf08e9748b0bfb0b824e6f7e129.scope: Succeeded. Nov 16 16:40:54.666528 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r40e56bf08e9748b0bfb0b824e6f7e129.scope: Consumed 896us CPU time Nov 16 16:40:59.446096 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-runc-d4a230741bf6d678b74af12a2ded9712dde11b1247e34d901a6303a78686e05b-runc.oSATr6.mount: Succeeded. Nov 16 16:42:56.686279 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:42:56.686040 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:42:56.686911 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:42:56.686480 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:42:56.686911 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:42:56.686510 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:42:56.696311 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:42:56.697444 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:42:56.697408 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:42:56.701242 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:42:56.701207 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:44:58.701179077 +0000 UTC m=+17503.637954799 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:42:56.701242 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:42:56.701242 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:42:56.701938 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r18c560ce8e7948dea05b01779ac6e9d0.scope: Succeeded. Nov 16 16:42:56.702542 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r18c560ce8e7948dea05b01779ac6e9d0.scope: Consumed 940us CPU time Nov 16 16:43:20.623062 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:43:20.622898312Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=75fa088c-6ed5-4fba-95e3-ea7a18001849 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:43:20.718088 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:43:20.623209615Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=75fa088c-6ed5-4fba-95e3-ea7a18001849 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:44:58.766691 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:44:58.766646 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:44:58.767465 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:44:58.766949 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:44:58.767465 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:44:58.766981 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:44:58.777482 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:44:58.778419 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:44:58.778367 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:44:58.782330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:44:58.782297 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:47:00.782267985 +0000 UTC m=+17625.719043607 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:44:58.782330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:44:58.782330 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:44:58.783369 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r2f63c76bba0f4af7b3c80075262cb3f6.scope: Succeeded. Nov 16 16:44:58.783988 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r2f63c76bba0f4af7b3c80075262cb3f6.scope: Consumed 782us CPU time Nov 16 16:47:00.833319 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:47:00.833242 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:47:00.834021 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:47:00.833600 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:47:00.834021 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:47:00.833631 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:47:00.844776 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:47:00.845910 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:47:00.845808 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:47:00.849842 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:47:00.849774 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:49:02.849749577 +0000 UTC m=+17747.786525199 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:47:00.849842 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:47:00.849842 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:47:00.850703 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-re88adce669f34b5a86ffc85747605287.scope: Succeeded. Nov 16 16:47:00.851303 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-re88adce669f34b5a86ffc85747605287.scope: Consumed 857us CPU time Nov 16 16:48:20.628776 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:48:20.628563168Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=988150e9-a8ef-4757-bbb9-7cb5098bcb5a name=/runtime.v1.ImageService/ImageStatus Nov 16 16:48:20.647057 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:48:20.628946171Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=988150e9-a8ef-4757-bbb9-7cb5098bcb5a name=/runtime.v1.ImageService/ImageStatus Nov 16 16:49:02.873741 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:49:02.873533 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:49:02.874539 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:49:02.873815 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:49:02.874539 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:49:02.873847 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:49:02.884365 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:49:02.885256 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:49:02.885226 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:49:02.889034 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:49:02.888931 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:51:04.888888669 +0000 UTC m=+17869.825664291 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:49:02.889034 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:49:02.889034 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:49:02.889903 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r9128296a1374418a99301cac72bd8098.scope: Succeeded. Nov 16 16:49:02.890570 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r9128296a1374418a99301cac72bd8098.scope: Consumed 762us CPU time Nov 16 16:51:04.921319 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:51:04.921256 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:51:04.922086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:51:04.921640 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:51:04.922086 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:51:04.921687 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:51:04.931938 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:51:04.933011 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:51:04.932928 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:51:04.936878 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:51:04.936811 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:53:06.936783852 +0000 UTC m=+17991.873559574 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:51:04.936878 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:51:04.936878 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:51:04.937479 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r855e3a3dbc5f43f9bad96b771bdeea19.scope: Succeeded. Nov 16 16:51:04.938011 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r855e3a3dbc5f43f9bad96b771bdeea19.scope: Consumed 850us CPU time Nov 16 16:53:06.987681 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:53:06.987629 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:53:06.988434 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:53:06.987929 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:53:06.988434 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:53:06.987977 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:53:06.997802 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:53:06.998961 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:53:06.998890 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:53:07.002830 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:53:07.002795 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:55:09.002767697 +0000 UTC m=+18113.939543319 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:53:07.002830 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:53:07.002830 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:53:07.003219 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r5408fc019ccd443698130f7dda34f461.scope: Succeeded. Nov 16 16:53:07.003719 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r5408fc019ccd443698130f7dda34f461.scope: Consumed 904us CPU time Nov 16 16:53:20.633859 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:53:20.633625004Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=7ada97b0-8a52-4ef4-baa7-8561886e0cdd name=/runtime.v1.ImageService/ImageStatus Nov 16 16:53:20.723325 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:53:20.634002807Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=7ada97b0-8a52-4ef4-baa7-8561886e0cdd name=/runtime.v1.ImageService/ImageStatus Nov 16 16:55:09.020784 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:55:09.020723 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:55:09.021481 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:55:09.021001 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:55:09.021481 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:55:09.021031 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:55:09.030829 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:55:09.031965 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:55:09.031897 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:55:09.035546 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:55:09.035452 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:57:11.035426741 +0000 UTC m=+18235.972202363 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:55:09.035546 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:55:09.035546 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:55:09.036250 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rc2a7265871eb4d07b067d7425dd33bd6.scope: Succeeded. Nov 16 16:55:09.037190 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rc2a7265871eb4d07b067d7425dd33bd6.scope: Consumed 922us CPU time Nov 16 16:55:59.283985 ropatil1611az-gs6zj-worker-westus-j4ngg ovs-vswitchd[1367]: ovs|00314|connmgr|INFO|br0<->unix#755: 2 flow_mods in the last 0 s (2 deletes) Nov 16 16:57:11.048152 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:57:11.048067 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:57:11.048781 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:57:11.048323 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:57:11.048781 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:57:11.048355 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:57:11.057918 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:57:11.058893 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:57:11.058857 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:57:11.062782 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:57:11.062713 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 16:59:13.062654218 +0000 UTC m=+18357.999429940 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:57:11.062782 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:57:11.062782 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:57:11.064655 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rdbb62abf02194ca5a58f0eaf13174cf6.scope: Succeeded. Nov 16 16:57:11.065174 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-rdbb62abf02194ca5a58f0eaf13174cf6.scope: Consumed 821us CPU time Nov 16 16:57:50.617917 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:57:50.617746 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r2xl8_23402384-5d8d-47cb-9b23-e7128b9e0e74/dns/0.log" Nov 16 16:57:50.656022 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:57:50.655993 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r2xl8_23402384-5d8d-47cb-9b23-e7128b9e0e74/kube-rbac-proxy/0.log" Nov 16 16:57:50.745318 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:57:50.745276 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-27gwx_8c7ca98a-3d4d-4a6f-b3ad-34b00a3427a2/dns-node-resolver/0.log" Nov 16 16:57:54.128465 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:57:54.128428 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dg87v_aacb53e4-6716-4337-a1ae-4ac3a857cbb0/node-ca/0.log" Nov 16 16:57:55.423226 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:57:55.423116 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4zcm6_b2808f6f-6018-4fb1-8229-65770d542c6d/serve-healthcheck-canary/0.log" Nov 16 16:58:08.022866 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:08.022790 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_machine-config-daemon-q7lkj_29d0f8e3-e33b-4c9f-a290-193626492310/machine-config-daemon/0.log" Nov 16 16:58:08.050759 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:08.050728 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_machine-config-daemon-q7lkj_29d0f8e3-e33b-4c9f-a290-193626492310/oauth-proxy/0.log" Nov 16 16:58:13.333240 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:13.333172 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kxmx6_89cfb802-14c7-4c79-bc9c-015efafee773/node-exporter/0.log" Nov 16 16:58:13.362360 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:13.362317 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kxmx6_89cfb802-14c7-4c79-bc9c-015efafee773/kube-rbac-proxy/0.log" Nov 16 16:58:13.389665 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:13.389633 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kxmx6_89cfb802-14c7-4c79-bc9c-015efafee773/init-textfile/0.log" Nov 16 16:58:18.518764 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:18.518721 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/kube-multus-additional-cni-plugins/0.log" Nov 16 16:58:18.547202 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:18.547167 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/egress-router-binary-copy/0.log" Nov 16 16:58:18.574586 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:18.574553 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/cni-plugins/0.log" Nov 16 16:58:18.602778 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:18.602751 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/bond-cni-plugin/0.log" Nov 16 16:58:18.628388 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:18.628362 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/routeoverride-cni/0.log" Nov 16 16:58:18.654308 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:18.654275 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/whereabouts-cni-bincopy/0.log" Nov 16 16:58:18.680786 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:18.680751 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2s6_57197822-83e2-43fe-a3d3-1a63580e3155/whereabouts-cni/0.log" Nov 16 16:58:18.985568 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:18.985439 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vm4st_f2b609ff-b0ed-4929-b767-953432cca02b/kube-multus/0.log" Nov 16 16:58:19.253820 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:19.253584 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-q8qt5_bc424839-f613-4159-bb7f-0c1769bf6200/network-metrics-daemon/0.log" Nov 16 16:58:19.277677 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:19.277649 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-q8qt5_bc424839-f613-4159-bb7f-0c1769bf6200/kube-rbac-proxy/0.log" Nov 16 16:58:20.638511 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:58:20.638346475Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=6a1cee88-4b74-4590-98ae-a44c9855cc37 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:58:20.675277 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 16:58:20.638611377Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=6a1cee88-4b74-4590-98ae-a44c9855cc37 name=/runtime.v1.ImageService/ImageStatus Nov 16 16:58:21.113807 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:21.113729 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-tqwqj_4dbb404b-812a-4297-b9ee-2cefe58d83ae/sdn/0.log" Nov 16 16:58:21.131584 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:21.131554 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-tqwqj_4dbb404b-812a-4297-b9ee-2cefe58d83ae/kube-rbac-proxy/0.log" Nov 16 16:58:21.156473 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:21.156403 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-sdn_sdn-tqwqj_4dbb404b-812a-4297-b9ee-2cefe58d83ae/drop-icmp/0.log" Nov 16 16:58:22.341229 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:22.341149 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-b5srf_a77123b5-b068-4e08-bbb0-243a4ef22749/network-check-target-container/0.log" Nov 16 16:58:24.203236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:24.203146 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-cvdsv_279f55b2-5534-42ae-ba2d-588e2c3a3ffe/tuned/0.log" Nov 16 16:58:31.921432 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:31.921264 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-disk-csi-driver-node-p82p6_062fe84e-408b-4970-94b8-5df0ae82ba87/csi-driver/0.log" Nov 16 16:58:31.949161 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:31.949117 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-disk-csi-driver-node-p82p6_062fe84e-408b-4970-94b8-5df0ae82ba87/csi-node-driver-registrar/0.log" Nov 16 16:58:31.976793 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:31.976753 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-disk-csi-driver-node-p82p6_062fe84e-408b-4970-94b8-5df0ae82ba87/csi-liveness-probe/0.log" Nov 16 16:58:32.006649 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:32.006611 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-disk-csi-driver-node-p82p6_062fe84e-408b-4970-94b8-5df0ae82ba87/azure-inject-credentials/0.log" Nov 16 16:58:33.299980 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:33.299894 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-file-csi-driver-node-pzz7z_6037f731-36d1-49c8-a598-34b98d642a8e/csi-driver/0.log" Nov 16 16:58:33.323876 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:33.323243 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-file-csi-driver-node-pzz7z_6037f731-36d1-49c8-a598-34b98d642a8e/csi-node-driver-registrar/0.log" Nov 16 16:58:33.348640 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:33.348549 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-file-csi-driver-node-pzz7z_6037f731-36d1-49c8-a598-34b98d642a8e/csi-liveness-probe/0.log" Nov 16 16:58:33.373271 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:58:33.373237 1563 logs.go:319] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_azure-file-csi-driver-node-pzz7z_6037f731-36d1-49c8-a598-34b98d642a8e/azure-inject-credentials/0.log" Nov 16 16:59:13.092296 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:59:13.092122 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 16:59:13.092880 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:59:13.092424 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 16:59:13.092880 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 16:59:13.092454 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 16:59:13.102714 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 16:59:13.103836 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 16:59:13.103759 1563 mount_linux.go:222] Detected OS with systemd Nov 16 16:59:13.107310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 16:59:13.107277 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 17:01:15.107252061 +0000 UTC m=+18480.044027783 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 16:59:13.107310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 16:59:13.107310 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 16:59:13.108070 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r713ff435d4424b25a5eefb935c346f53.scope: Succeeded. Nov 16 16:59:13.108648 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r713ff435d4424b25a5eefb935c346f53.scope: Consumed 887us CPU time Nov 16 17:01:15.132573 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 17:01:15.132515 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 17:01:15.133319 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 17:01:15.132820 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 17:01:15.133319 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 17:01:15.132852 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 17:01:15.142578 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 17:01:15.143599 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 17:01:15.143531 1563 mount_linux.go:222] Detected OS with systemd Nov 16 17:01:15.147222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 17:01:15.147189 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 17:03:17.147142597 +0000 UTC m=+18602.083918319 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 17:01:15.147222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 17:01:15.147222 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 17:01:15.147936 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r7246024884c14e10a5575b8f083e3b05.scope: Succeeded. Nov 16 17:01:15.148449 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r7246024884c14e10a5575b8f083e3b05.scope: Consumed 832us CPU time Nov 16 17:03:17.161995 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 17:03:17.161940 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 17:03:17.162810 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 17:03:17.162268 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 17:03:17.162810 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 17:03:17.162297 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 17:03:17.172520 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 17:03:17.173593 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 17:03:17.173504 1563 mount_linux.go:222] Detected OS with systemd Nov 16 17:03:17.177203 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 17:03:17.177112 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 17:05:19.177087831 +0000 UTC m=+18724.113863553 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 17:03:17.177203 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 17:03:17.177203 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 17:03:17.178249 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ra79b5d23f4d040dfa442d998165367cc.scope: Succeeded. Nov 16 17:03:17.178840 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-ra79b5d23f4d040dfa442d998165367cc.scope: Consumed 861us CPU time Nov 16 17:03:20.643673 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 17:03:20.643481975Z" level=info msg="Checking image status: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40" id=f0c80873-703c-4821-a883-f86e7ae678e9 name=/runtime.v1.ImageService/ImageStatus Nov 16 17:03:20.719738 ropatil1611az-gs6zj-worker-westus-j4ngg crio[1530]: time="2022-11-16 17:03:20.643781577Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9e44923d83df71e11fd4837b57daa843fc8a5c5f8e9eedba0e1a1eaf90f98ca3,RepoTags:[],RepoDigests:[quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c39aa024d407b037fc426c5e55a3613f5111ba787cfba1abccc44cf2db1e4a40],Size_:286791011,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=f0c80873-703c-4821-a883-f86e7ae678e9 name=/runtime.v1.ImageService/ImageStatus Nov 16 17:05:19.183402 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 17:05:19.183201 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 17:05:19.184200 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 17:05:19.183578 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 17:05:19.184200 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 17:05:19.183612 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 17:05:19.194149 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 17:05:19.195236 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 17:05:19.195203 1563 mount_linux.go:222] Detected OS with systemd Nov 16 17:05:19.199204 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 17:05:19.199112 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 17:07:21.19908572 +0000 UTC m=+18846.135861342 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 17:05:19.199204 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 17:05:19.199204 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 17:05:19.199781 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r0d0be171451c4a169126483c15707cbb.scope: Succeeded. Nov 16 17:05:19.201378 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r0d0be171451c4a169126483c15707cbb.scope: Consumed 870us CPU time Nov 16 17:07:21.207112 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 17:07:21.206989 1563 reconciler.go:192] "operationExecutor.UnmountVolume started for volume \"local\" (UniqueName: \"kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a\") pod \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\" (UID: \"f5df478d-fc7b-4fda-b15e-1a9a05fdf611\") " Nov 16 17:07:21.207734 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 17:07:21.207430 1563 volume_path_handler_linux.go:62] couldn't find loopback device which takes file descriptor lock. Skip detaching device. device path: "/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611" Nov 16 17:07:21.207734 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: W1116 17:07:21.207463 1563 volume_path_handler.go:217] Warning: Unmap skipped because symlink does not exist on the path: /var/lib/kubelet/pods/f5df478d-fc7b-4fda-b15e-1a9a05fdf611/volumeDevices/kubernetes.io~azure-disk/pvc-cf8f4da1-5211-406e-986b-55c19439ee1a Nov 16 17:07:21.217237 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: Started Kubernetes systemd probe. Nov 16 17:07:21.218165 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: I1116 17:07:21.218113 1563 mount_linux.go:222] Detected OS with systemd Nov 16 17:07:21.222197 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: E1116 17:07:21.222165 1563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a podName:f5df478d-fc7b-4fda-b15e-1a9a05fdf611 nodeName:}" failed. No retries permitted until 2022-11-16 17:09:23.222139399 +0000 UTC m=+18968.158915121 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "local" (UniqueName: "kubernetes.io/azure-disk//subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ropatil1611az-gs6zj-rg/providers/Microsoft.Compute/disks/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a") pod "f5df478d-fc7b-4fda-b15e-1a9a05fdf611" (UID: "f5df478d-fc7b-4fda-b15e-1a9a05fdf611") : blkUtil.DetachFileDevice failed. globalUnmapPath:/var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a, podUID: f5df478d-fc7b-4fda-b15e-1a9a05fdf611, bindMount: true: failed to unmount linkPath /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: unmount failed: exit status 32 Nov 16 17:07:21.222197 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Unmounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611 Nov 16 17:07:21.222197 ropatil1611az-gs6zj-worker-westus-j4ngg hyperkube[1563]: Output: umount: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/volumeDevices/ropatil1611az-gs6zj-dynamic-pvc-cf8f4da1-5211-406e-986b-55c19439ee1a/f5df478d-fc7b-4fda-b15e-1a9a05fdf611: target is busy. Nov 16 17:07:21.222706 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r847df5dece514578ab8b5e55c1834ef7.scope: Succeeded. Nov 16 17:07:21.223236 ropatil1611az-gs6zj-worker-westus-j4ngg systemd[1]: run-r847df5dece514578ab8b5e55c1834ef7.scope: Consumed 760us CPU time