Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-436

A-MQ 7 -- not enough synchronous disk writes to guarantee data integrity on openwire

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • A-MQ 7.0.0.ER14
    • A-MQ 7.0.0.ER12
    • None
    • None
    • Hide

      1. Configure A-MQ 7 to use NIO, and to sync to disk after every store update. E.g.:

      <journal-max-io>1</journal-max-io>
      <journal-type>NIO</journal-type>

      This is the default for A-MQ 6, so no particular changes need to be made.

      2. For each broker type, use a simple Java client (I can provide one) that creates a JMS Session (non-transacted), then posts, say, 100 messages to a destination.

      3. Capture system calls using "strace -fF [pid] >& somefile"

      4. Examine the file for calls to fsync() of fdatasync()

      Show
      1. Configure A-MQ 7 to use NIO, and to sync to disk after every store update. E.g.: <journal-max-io>1</journal-max-io> <journal-type>NIO</journal-type> This is the default for A-MQ 6, so no particular changes need to be made. 2. For each broker type, use a simple Java client (I can provide one) that creates a JMS Session (non-transacted), then posts, say, 100 messages to a destination. 3. Capture system calls using "strace -fF [pid] >& somefile" 4. Examine the file for calls to fsync() of fdatasync()

    Description

      When producing successive, persistent messages to a JMS destination in a single thread, the client it entitled to assume that if the send() operation returns without error, the message data has been made persistent by the broker – it is not just in memory.

      With NIO journalling, Artemis updates the journal by using (or, rather, asking the JVM to use) a write() followed by an fdatasync(). By default, ActiveMQ uses a write() followed by fsync() (although it can be configured to use fdatasync() instead).

      With ActiveMQ, I see almost exactly one sync update per message produced, as I would expected. With Artemis, I see a variable number, even when setting journal-max-io to 1.

      It seems that Artemis (with NIO) does not guarantee that message data has been flushed to disk before the client gets a successful acknowledgment. With a moderately fast disk, and a fast message producer, there seem to be far fewer sync writes than messages. This is consistent with a flushing behaviour that is time-based, not message-volume based.

      Attachments

        Activity

          People

            rh-ee-ataylor Andy Taylor
            rhn-support-kboone Kevin Boone
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: