Uploaded image for project: 'Data Foundation Bugs'
  1. Data Foundation Bugs
  2. DFBUGS-444

[2249318] Error RADOS object not found

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • odf-4.16.4
    • odf-4.13
    • ceph/Ceph-MGR/x86
    • None
    • None

      Description of problem (please be detailed as possible and provide log

      + Error RADOS object not found (Failed to operate read op for oid

      snippests):

      ceph -s
      cluster:
      id: 3e9184b4-47e0-49e3-892a-4d1a88459ece
      health: HEALTH_ERR
      Module 'devicehealth' has failed: [errno 2] RADOS object not found (Failed to operate read op for oid HGST_HUH721212AL4204_2AHLG9NY)

      services:
      mon: 3 daemons, quorum f,i,j (age 9d)
      mgr: a(active, since 5w)
      mds: 1/1 daemons up, 1 hot standby
      osd: 23 osds: 23 up (since 6d), 23 in (since 6d)
      rgw: 3 daemons active (3 hosts, 1 zones)

      data:
      volumes: 1/1 healthy
      pools: 24 pools, 689 pgs
      objects: 1.83M objects, 4.3 TiB
      usage: 13 TiB used, 137 TiB / 150 TiB avail
      pgs: 689 active+clean

      io:
      client: 1.1 MiB/s rd, 41 MiB/s wr, 23 op/s rd, 716 op/s wr
      cache: 1 op/s promote

      ceph df
      — RAW STORAGE —
      CLASS SIZE AVAIL USED RAW USED %RAW USED
      hdd 131 TiB 121 TiB 10 TiB 10 TiB 7.95
      ssd 19 TiB 17 TiB 2.6 TiB 2.6 TiB 13.51
      TOTAL 150 TiB 137 TiB 13 TiB 13 TiB 8.66

      — POOLS —
      POOL ID PGS STORED OBJECTS USED %USED MAX AVAIL
      ocs-storagecluster-cephblockpool 1 128 3.2 TiB 861.53k 9.6 TiB 8.75 33 TiB
      ocs-storagecluster-cephobjectstore.rgw.control 2 8 1.3 KiB 8 4.0 KiB 0 33 TiB
      ocs-storagecluster-cephfilesystem-metadata 3 32 15 GiB 43.04k 46 GiB 0.05 33 TiB
      ocs-storagecluster-cephfilesystem-data0 4 32 108 GiB 106.59k 327 GiB 0.32 33 TiB
      ocs-storagecluster-cephobjectstore.rgw.meta 5 8 22 KiB 61 2.2 MiB 0 33 TiB
      ocs-storagecluster-cephobjectstore.rgw.log 6 8 29 MiB 228 88 MiB 0 33 TiB
      ocs-storagecluster-cephobjectstore.rgw.buckets.index 7 8 25 MiB 418 76 MiB 0 33 TiB
      ocs-storagecluster-cephobjectstore.rgw.buckets.non-ec 8 8 3.1 MiB 39.99k 2.8 GiB 0 33 TiB
      .rgw.root 9 8 4.7 KiB 16 720 KiB 0 33 TiB
      ocs-storagecluster-cephobjectstore.rgw.buckets.data 10 32 679 GiB 388.63k 2.0 TiB 1.95 33 TiB
      .mgr 11 1 3.3 MiB 5 10 MiB 0 33 TiB
      ocs-storagecluster-cephblockpool-cache 12 32 290 GiB 80.19k 858 GiB 6.13 4.3 TiB
      ocs-storagecluster-cephobjectstore.rgw.control-cache 13 32 747 B 18 108 KiB 0 4.3 TiB
      ocs-storagecluster-cephfilesystem-metadata-cache 14 32 518 MiB 79.96k 1.5 GiB 0.01 4.3 TiB
      ocs-storagecluster-cephfilesystem-data0-cache 15 32 975 MiB 80.12k 2.9 GiB 0.02 4.3 TiB
      ocs-storagecluster-cephobjectstore.rgw.meta-cache 16 32 39 KiB 188 1.3 MiB 0 4.3 TiB
      ocs-storagecluster-cephobjectstore.rgw.log-cache 17 32 82 MiB 461 248 MiB 0 4.3 TiB
      ocs-storagecluster-cephobjectstore.rgw.buckets.index-cache 18 32 100 MiB 451 300 MiB 0 4.3 TiB
      ocs-storagecluster-cephobjectstore.rgw.buckets.non-ec-cache 19 32 1.0 MiB 70.48k 5.1 MiB 0 4.3 TiB
      .rgw.root-cache 20 32 6.1 KiB 38 372 KiB 0 4.3 TiB
      ocs-storagecluster-cephobjectstore.rgw.buckets.data-cache 21 32 48 GiB 80.04k 143 GiB 1.08 4.3 TiB
      device_health_metrics-cache 22 32 26 MiB 50 77 MiB 0 4.3 TiB
      hyperion-ssd-pool 23 32 1.6 KiB 5 17 KiB 0 4.3 TiB
      ocs-storagecluster-cephobjectstore.rgw.otp 24 32 0 B 0 0 B 0 33 TiB

      Version of all relevant components (if applicable):
      odf 4.12

      Regards,
      Manjunatha.

              pdonnell@redhat.com Patrick Donnelly
              rhn-support-mmanjuna Maragondanaguni Manjunatha
              Patrick Donnelly
              Elad Ben Aharon Elad Ben Aharon
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

                Created:
                Updated: