What were you trying to do that didn't work?

oscap can consume all the memory of the system, leading to system hang (because the system tries to reclaim memory) and/or oom-killer to kill various processes, including oscap (in the latter case, this prevents getting the result of the scan).
The issue doesn't occur while probing (which implements memory guard), but while dumping the XML result. The issue is hence more likely in the libxml2 library, please reassign if needed.

What is the impact of this issue to you?

1. not getting any result (oscap being killed)
2. system hanging trying to reclaim memory

Please provide the package NVR for which the bug is seen:

openscap-1.3.10-2.el9_3.x86_64

How reproducible is this bug?:

Always

Steps to reproduce

1. Create a VM with 8GB of memory but no swap
2. Populate /home with a lot of files not having a username and groupname but only uid/gid

   # uid=1001; mkdir /home/user$uid; for f in $(seq 1 10000); do touch /home/user$uid/$f; done; chown -R $uid:$uid /home/user$uid
   # for uid in $(seq 1002 1100); do cp -r /home/user1001 /home/user$uid && chown -R $uid:$uid /home/user$uid; done
   

3. Run oscap with CIS profile customized to execute xccdf_org.ssgproject.content_rule_file_permissions_ungroupowned and xccdf_org.ssgproject.content_rule_no_files_unowned_by_user rules (attached)

   # oscap xccdf eval --report oscap_report.html --profile xccdf_org.ssgproject.content_profile_cis_customized --tailoring-file /root/ssg-rhel9-ds-tailoring.xml  /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml

Expected results

oscap consuming reasonably the memory (~2GB max)

Actual results

oscap being killed after exhausting the system memory

Additional information

When scanning the system, oscap will not scan the whole content of /home because of the memory constraints.
However, this will anyway result in consuming all the memory of the system when writing the XML because of the consolidation of the 2 rules:

--- Starting Evaluation ---

Title   Ensure All Files Are Owned by a Group
Rule    xccdf_org.ssgproject.content_rule_file_permissions_ungroupowned
Ident   CCE-83906-8
W: oscap: Memory usage ratio limit reached! limit=0.330000, current=0.330001, used=2535 MB, free=3964 MB, total=7684 MB, count of items=478940
W: oscap: Memory usage ratio limit reached! limit=0.330000, current=0.330896, used=2542 MB, free=3956 MB, total=7684 MB, count of items=478940
Result  fail

Title   Ensure All Files Are Owned by a User
Rule    xccdf_org.ssgproject.content_rule_no_files_unowned_by_user
Ident   CCE-83896-1
W: oscap: Memory usage ratio limit reached! limit=0.330000, current=0.566082, used=4350 MB, free=2136 MB, total=7684 MB, count of items=1001
W: oscap: Memory usage ratio limit reached! limit=0.330000, current=0.566082, used=4350 MB, free=2136 MB, total=7684 MB, count of items=1001
W: oscap: Memory usage ratio limit reached! limit=0.330000, current=0.566082, used=4350 MB, free=2136 MB, total=7684 MB, count of items=1001
Result  fail

Killed