Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-14046

gcc with CFLAGS="-O2 -flto=auto" produces a binary with changed function

    • Icon: Story Story
    • Resolution: Not a Bug
    • Icon: Undefined Undefined
    • None
    • rhel-9.3.0
    • gcc
    • rhel-sst-pt-gcc
    • ssg_platform_tools
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • armv7hl
    • None

      Hello,

      when compiling aespipe from sources (https://loop-aes.sourceforge.net/aespipe/), GCC produces a wrong binary that generates the wrong cipher text when encrypting known text. This happens with these two compilation flags used TOGETHER:

      CFLAGS="-O2 -flto=auto"
      

      Interstingly, i686 and x86_64 arches are NOT AFFECTED. Only these arches are known to be affected:

      • armv7hl
      • aarch64
      • ppc64le
      • s390x

      The problem was first spotted when compiling aespipe package for Fedora-34:
      https://koji.fedoraproject.org/koji/taskinfo?taskID=62266480

      On Fedora-33, the package has compiled without problems.

      gcc v10 is known as the last working version
      gcc v11 is known as the first broken version

      The issue persists when compiling aespipe for Fedora rawhide with GCC v13.

      WORKAROUND is to use
      CFLAGS="-O1 -flto=auto"
      or
      CFLAGS="-O0 -flto=auto"

      or omit -flto=auto:
      CFLAGS="-O2"

      I have reproduced the issue on RHEL-9:

      • On RHEL-8, with GCC 8.5.0, aespipe compiles correctly with CFLAGS="-O2 -flto=auto"
      • On RHEL-9, with GCC 11.4.1, aespipe compiles wrongly with CFLAGS="-O2 -flto=auto"

      Reproducer:

      wget https://loop-aes.sourceforge.net/aespipe/aespipe-v2.4f.tar.bz2
      tar xvf aespipe-v2.4f.tar.bz2
      cd aespipe-v2.4f
      make clean; ./configure CFLAGS="-O2 -flto=auto"; make; make tests
      

      When binary is correct (expected output):

      *** Test results ok ***
      

      aespipe was compiled correctly and has generated the desired output when encrypting and decrypting messages.

      When the binary is wrong (reproducing the issue):

      make[1]: Entering directory '/root/A/aespipe-v2.4f'
      ./aespipe -v -p 3 -e AES128 -K ./gpgkey2.asc -G test-dir1 <test-file3 >test-file1 3<test-file4
      ./aespipe: C-language AES, 128 key bits, encrypting, multi-key-v2 mode, RAM not locked
      make test-part3
      make[2]: Entering directory '/root/A/aespipe-v2.4f'
      md5sum test-file1 >test-file2
      echo "f9825b79873f5c439ae9371c1a929a6c  test-file1" >test-file5
      make[2]: Leaving directory '/root/A/aespipe-v2.4f'
      cmp test-file2 test-file5
      test-file2 test-file5 differ: byte 1, line 1
      make[1]: *** [Makefile:120: test-part2] Error 1
      make[1]: Leaving directory '/root/A/aespipe-v2.4f'
      make: *** [Makefile:87: tests] Error 2
      

      It means that aespipe has NOT generated the expected output. The message was encrypted wrongly.

              mpolacek@redhat.com Marek Polacek
              jhladky1@redhat.com Jiri Hladky
              Marek Polacek Marek Polacek
              Vaclav Kadlcik Vaclav Kadlcik
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: