Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-8800

Coverity static analysis, suspicious bitwise logical expression, DigestUtil (Elytron)

XMLWordPrintable

      Coverity found suspicious logical operation https://scan7.coverity.com/reports.htm#v23632/p11778/fileInstanceId=9563899&defectInstanceId=2359232&mergedDefectId=1377462

      See detailed description of possible problem in [1]

      If I extend DigestUtilTest#testDecodeByteOrderedInteger with case from [1], test fails

              byte[] inputFF = CodePointIterator.ofString("000000FF").hexDecode().drain();
              assertEquals(0xFF, decodeByteOrderedInteger(inputFF, 0, 4));
      

      If I change decodeByteOrderedInteger implementation according to [1], all tests pass again.

      result |= (buf[offset + i] & 0xff);
      

      [1] http://findbugs.sourceforge.net/bugDescriptions.html#BIT_IOR_OF_SIGNED_BYTE

      Setting to high priority, because correct behavior of SASL Digest mechanism could be impacted.

            rhn-support-ivassile Ilia Vassilev
            mchoma@redhat.com Martin Choma
            Martin Choma Martin Choma
            Martin Choma Martin Choma
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: