+++ This bug was initially created as a clone of Bug #2215262 +++

+++ This bug was initially created as a clone of Bug #2183041 +++

Description of problem:
The checkpoint doesn't work for crun, and got error like this "could not find symbol `criu_set_lsm_mount_context` in `libcriu.so`".

Version-Release number of selected component (if applicable):
[root@kvm-04-guest23 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.8 Beta (Ootpa)

[root@kvm-04-guest23 ~]# rpm -q crun criu criu-libs criu-devel kernel
crun-1.8.1-2.module+el8.8.0+18418+f0e540fe.x86_64
criu-3.15-3.module+el8.8.0+18060+3f21f2cc.x86_64
criu-libs-3.15-3.module+el8.8.0+18060+3f21f2cc.x86_64
criu-devel-3.15-3.module+el8.8.0+18060+3f21f2cc.x86_64
kernel-4.18.0-477.6.1.el8_8.x86_64

How reproducible:
always

Steps to Reproduce:
1. make sure the current system is RHEL 8.8 w/ cgroup v2 enabled
2. running tests/test_checkpoint_restore.py with downstream crun

Actual results:

[root@kvm-04-guest23 crun]# python3 tests/test_checkpoint_restore.py
1..3
2023-03-30T08:41:39.458084Z: could not find symbol `criu_set_lsm_mount_context` in `libcriu.so`
b''
Command '['/root/crun/crun', '--root', '/root/crun/.testsuite-run-28376/root', 'checkpoint', '--image-path=/root/crun/.testsuite-run-28376/checkpoint', 'test-tmpjaxqzz7d']' returned non-zero exit status 1.
not ok 1 - checkpoint-restore
ok 2 - checkpoint-restore-ext-ns #SKIP
ok 3 - checkpoint-restore-pre-dump #SKIP

Expected results:
the checkpoint should work for crun on RHEL 8.8

Additional info:
Giuseppe has committed a patch for this issue - https://github.com/containers/crun/pull/1183

— Additional comment from Tom Sweeney on 2023-03-30 19:50:29 UTC —

Possible ZeroDay BZ for 8.8 and 9.2. Assigning to @jnovy@redhat.com for any further BZ or packaging needs.

— Additional comment from Jindrich Novy on 2023-03-31 09:57:06 UTC —

Giuseppe, could you have a look at this one?

— Additional comment from Giuseppe Scrivano on 2023-03-31 09:59:35 UTC —

it is fixed upstream with: https://github.com/containers/crun/pull/1183

— Additional comment from Jindrich Novy on 2023-03-31 10:00:19 UTC —

It's actually part of crun-1.8.3.

— Additional comment from Giuseppe Scrivano on 2023-03-31 13:16:47 UTC —

(In reply to Jindrich Novy from comment #4)
> It's actually part of crun-1.8.3.

have you backported the patch? The fix didn't hit the release last week

— Additional comment from Tom Sweeney on 2023-04-04 00:25:43 UTC —

Setting up for 8.8 Zeroday

— Additional comment from RHEL Program Management Team on 2023-04-04 00:32:39 UTC —

This bug has been copied as 8.8.0 stream bug#2184221 and now must be resolved in the current update release, blocker flag set.

— Additional comment from Jindrich Novy on 2023-04-04 14:33:45 UTC —

Just added that PR now: https://gitlab.com/redhat/centos-stream/rpms/crun/-/merge_requests/57

— Additional comment from Tom Sweeney on 2023-04-04 21:13:08 UTC —

I'm not sure why bugzilla is setting some of these BZ's to blocker+ automagically, but this should only be a Zeroday fix, not a blocker+

— Additional comment from Jindrich Novy on 2023-04-05 07:47:17 UTC —

It is because rule engine sets blocker+ automatically for upcoming release to avoid regression, please don't touch it.

— Additional comment from Alex Jia on 2023-04-10 06:39:03 UTC —

The crun-1.8.3-1 is not available for 8.8 in brew system now.

— Additional comment from Alex Jia on 2023-04-11 01:43:09 UTC —

This bug has not been fixed by crun-1.8.3-2.module+el8.9.0+18566+0239040b yet.

[root@kvm-02-guest05 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.9 Beta (Ootpa)

[root@kvm-02-guest05 ~]# rpm -q crun criu criu-libs criu-devel kernel
crun-1.8.3-2.module+el8.9.0+18566+0239040b.x86_64
criu-3.15-3.module+el8.9.0+18566+0239040b.x86_64
criu-libs-3.15-3.module+el8.9.0+18566+0239040b.x86_64
criu-devel-3.15-3.module+el8.9.0+18566+0239040b.x86_64
kernel-4.18.0-484.el8.x86_64

[root@kvm-02-guest05 ~]# cd crun
[root@kvm-02-guest05 crun]# ll crun
lrwxrwxrwx. 1 root root 13 Apr 10 21:37 crun -> /usr/bin/crun

[root@kvm-02-guest05 crun]# python3 tests/test_checkpoint_restore.py
1..3
2023-04-11T01:38:45.420183Z: could not find symbol `criu_join_ns_add` in `libcriu.so`
b''
Command '['/root/crun/crun', '--root', '/root/crun/.testsuite-run-112404/root', 'checkpoint', '--image-path=/root/crun/.testsuite-run-112404/checkpoint', 'test-tmp497ag61j']' returned non-zero exit status 1.
not ok 1 - checkpoint-restore
ok 2 - checkpoint-restore-ext-ns #SKIP
ok 3 - checkpoint-restore-pre-dump #SKIP

— Additional comment from Giuseppe Scrivano on 2023-04-13 10:21:23 UTC —

We need the crun patch in https://github.com/containers/crun/pull/1183

— Additional comment from Tom Sweeney on 2023-04-17 20:34:20 UTC —

Assigning to Jindrich for any further packaging or BZ needs. Jindrich, please note Giuseppe's last comment.

— Additional comment from Jindrich Novy on 2023-04-18 07:49:26 UTC —

I added that patch on 4th Apr, just forgot to change the state.

— Additional comment from Alex Jia on 2023-04-18 07:55:35 UTC —

(In reply to Alex Jia from comment #12)
> [root@kvm-02-guest05 crun]# python3 tests/test_checkpoint_restore.py
> 1..3
> 2023-04-11T01:38:45.420183Z: could not find symbol `criu_join_ns_add` in
> `libcriu.so`
> b''
> Command '['/root/crun/crun', '--root',
> '/root/crun/.testsuite-run-112404/root', 'checkpoint',
> '--image-path=/root/crun/.testsuite-run-112404/checkpoint',
> 'test-tmp497ag61j']' returned non-zero exit status 1.
> not ok 1 - checkpoint-restore
> ok 2 - checkpoint-restore-ext-ns #SKIP
> ok 3 - checkpoint-restore-pre-dump #SKIP

The original bug has been fixed, this bug is different but similar one,
Giuseppe committed a patch for it: https://github.com/containers/crun/pull/1193

— Additional comment from Jindrich Novy on 2023-04-18 11:24:07 UTC —

The crun-1.8.4 already contains the patch: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=52003754

— Additional comment from Alex Jia on 2023-04-18 11:26:55 UTC —

This bug has been verified on crun-1.8.4-2.module+el8.8.0+18669+fa5aca5a (container-tools-rhel8-8080020230418075432.0f77c1b7).

[root@kvm-03-guest11 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.8 (Ootpa)

[root@kvm-03-guest11 ~]# rpm -q crun criu kernel
crun-1.8.4-2.module+el8.8.0+18669+fa5aca5a.x86_64
criu-3.15-3.module+el8.8.0+18060+3f21f2cc.x86_64
kernel-4.18.0-477.10.1.el8_8.x86_64

[root@kvm-03-guest11 crun]# ln -s /usr/bin/crun ./crun
[root@kvm-03-guest11 crun]# python3 tests/test_checkpoint_restore.py
1..3
ok 1 - checkpoint-restore
ok 2 - checkpoint-restore-ext-ns #SKIP
ok 3 - checkpoint-restore-pre-dump #SKIP

— Additional comment from errata-xmlrpc on 2023-05-02 09:11:49 UTC —

This bug has been added to advisory RHSA-2023:113680 by Jindrich Novy (jnovy@redhat.com)

— Additional comment from errata-xmlrpc on 2023-05-02 09:11:50 UTC —

Bug report changed to ON_QA status by Errata System.
A QE request has been submitted for advisory RHSA-2023:113680-01
https://errata.devel.redhat.com/advisory/113680

— Additional comment from errata-xmlrpc on 2023-05-02 09:12:12 UTC —

This bug has been added to advisory RHSA-2023:113680 by Jindrich Novy (jnovy@redhat.com)

— Additional comment from Alex Jia on 2023-05-04 10:04:52 UTC —

Moving this bug to verified state per Comment 18.

— Additional comment from Alex Jia on 2023-06-15 08:27:26 UTC —

[root@kvm-01-guest04 crun]# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.9 Beta (Ootpa)

[root@kvm-01-guest04 crun]# rpm -q crun criu kernel
crun-1.8.3-1.module+el8.9.0+19098+6e7a5e3f.x86_64
criu-3.15-3.module+el8.9.0+19098+6e7a5e3f.x86_64
kernel-4.18.0-497.el8.x86_64

[root@kvm-01-guest04 crun]# ln -s /usr/bin/crun ./crun
[root@kvm-01-guest04 crun]# python3 tests/test_checkpoint_restore.py
1..3
2023-06-15T08:21:33.370226Z: could not find symbol `criu_set_lsm_mount_context` in `libcriu.so`
b''
Command '['/root/crun/crun', '--root', '/root/crun/.testsuite-run-48402/root', 'checkpoint', '--image-path=/root/crun/.testsuite-run-48402/checkpoint', 'test-tmp6c6zaelx']' returned non-zero exit status 1.
not ok 1 - checkpoint-restore
ok 2 - checkpoint-restore-ext-ns #SKIP
ok 3 - checkpoint-restore-pre-dump #SKIP

— Additional comment from Alex Jia on 2023-06-15 11:07:50 UTC —

(In reply to Alex Jia from comment #1)
> [root@kvm-01-guest04 crun]# rpm -q crun criu kernel
> crun-1.8.3-1.module+el8.9.0+19098+6e7a5e3f.x86_64

The crun-1.8.3-1.module+el8.9.0+19098+6e7a5e3f belongs to container-tools-4.0-8090020230614164212.e7857ab1.

We need the crun 1.8.5 for 4.0 stream of the container-tools module on 8.9.

— Additional comment from Jindrich Novy on 2023-06-15 11:11:25 UTC —

Why we need to update crun to 1.8.5 in 4.0 stable stream Alex?

— Additional comment from Tom Sweeney on 2023-06-15 22:11:29 UTC —

I don't think we should backport this to the stable stream either. TY for reporting this Alex, but I'm going to close tis as won't fix. If you have a compelling reason to fix it, I'm willing to listen and we can open.

— Additional comment from Alex Jia on 2023-06-16 00:57:33 UTC —

(In reply to Jindrich Novy from comment #3)
> Why we need to update crun to 1.8.5 in 4.0 stable stream Alex?

Because this known bug was fixed by crun 1.8.5 ago, but it's for rhel8 stream not 4.0 stream,
anyway, we need to backpoint previous patch to buildah new crun verion, thanks!

— Additional comment from Alex Jia on 2023-06-16 01:00:35 UTC —

(In reply to Alex Jia from comment #5)
> Because this known bug was fixed by crun 1.8.5 ago, but it's for rhel8
> stream not 4.0 stream,
> anyway, we need to backpoint previous patch to buildah new crun verion,
> thanks!

Without this fixing, and then user can't use checkpoint function normally,
I also agree close this bug as WONTFIX if it's acceptable.

— Additional comment from Tom Sweeney on 2023-06-16 18:39:00 UTC —

Thanks for the background info @ajia@redhat.com . Let's stick with keeping this closed for now then. I'm concerned about messing something up in the stable stream. As this was a problem that was found by us, and a use case that I don't think many will run into, let's take a watch and see approach. If a customer reports it later, then we can reevaluate.

— Additional comment from Alex Jia on 2023-06-17 00:13:01 UTC —

(In reply to Tom Sweeney from comment #7)
> that I don't think many will run into, let's take a watch and see approach.
> If a customer reports it later, then we can reevaluate.

Got it, thank you Tom!