Home > Java > Beware of ActiveMQ with JDBC persistence

Beware of ActiveMQ with JDBC persistence

ActiveMQ (and Red Hat’s AMQ derivative) is gaining ground. Many companies that need high availability already have highly available databases, so it makes sense to use ActiveMQ in a master/slave configuration with a JDBC message store. A bit slower than KahaDB or LevelDB, but simple to configure and very safe. Or is it? Well, it depends.

ActiveMQ really has two different message stores: one for normal messages and one for scheduled messages. For normal persistent messages it is possible to use KahaDB, LevelDB or JDBC. For scheduled messages there are only two alternatives: KahaDB and in-memory. So, if you have a highly available master/slave configuration with a JDBC backend for normal messages, all scheduled messages will be lost when the master dies, as they are stored in the local file system or in memory.

Scheduled messages are seldom used, so perhaps this is trivial? Unfortunately not. Most applications want to redeliver failed messages a few times before relegating them to a DLQ. ActiveMQ supports redelivery, but under the hoods it is implemented using scheduled messages. In other words all messages waiting for redelivery are lost at failover unless the job scheduler store is highly available, which means a highly available KahaDB store. Installations that use JDBC don’t do HA KahaDB.

In other words, take care when using ActiveMQ or AMQ with a JDBC backend! Normal messages are safe, but some are likely to be lost at failover. Is that acceptable?

A documentation bug has been filed for this.

Advertisements
Categories: Java
  1. Thomas
    2016-10-07 at 11:22

    Hi, we have suffered from this yesterday. Can you tell me which bug ID you filed?
    Thanks,
    Thomas

  2. Thomas
    2016-10-07 at 16:40

    I just checkd (created a login for that), there is no such issue created. I tried to find it, but it returned no result. The scheduler and the redelivery as a user of the scheduler are kind of the ugly child of ActiveMQ, the fresh introduced LevelDB has a hint on the HA page (http://activemq.apache.org/replicated-leveldb-store.html), that there are issues, if you try to use scheduled messages with the leveldb HA approach.

    • 2016-10-07 at 16:52

      Yep, there is no good solution. I haven’t checked how this is done in Artemis, though. Perhaps there is a better way. For now KahaDB seems to be the only option.

      By the way, there are other issues with scheduled messages in ActiveMQ, at least as of 5.11: custom JMS headers may be lost. Step lightly and take your time testing everything.

      • Thomas
        2016-10-07 at 17:24

        Actually we have been hit by the redelivery policy using scheduled messages that have suddenly taken all the space on the disk. That’s how we got aware of the issue.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: