Archive

Archive for the ‘Java’ Category

JBoss EAP 6 and CACHE_CONSUMER with Spring DMLC

The documentation for Spring’s DefaultMessageListenerContainer states:

Some Java EE servers only register their JMS resources with an ongoing XA transaction in case of a freshly obtained JMS Connection and Session.

Rumour has it that JBoss EAP is one of the guilty servers, but is it true? We couldn’t find any definitive answers for the recent versions. But yes, caching consumers with JBoss EAP 6 using XA transactions will fail. The application server only enlists with the active transaction when a connection is checked out from the pool. If the consumer is reused for other transactions that will seem to work (no error messages), but in fact the messages are consumed without transactions. If a rollback occurs messages are lost.

In short, be sure to use CACHE_NONE with JBoss EAP 6.

Categories: Java, Performance

Beware of interned strings with G1

Interned strings can always be problematic, as the size of the hash table is fixed. Perhaps the -XX:+UseStringDeduplication option can make things worse, as (according to native memory tracing) it seems to use the same pool. However, at present there is a more pressing problem with Java 8 and 9 using the G1 garbage collector: interned strings may not be garbage collected at all, resulting in a native memory leak. See JDK-8180048. What to do? Stay calm or switch to another garbage collector. But be sure to keep an eye on the native memory usage of your Java applications!

Categories: Java

Effective low-tech alternative to @formatter:off for Java

I love automatic formatting for source code. Back in 2004 I scrapped the detailed formatting guide for the project I was running and replaced it with a single sentence: all code must be formatted by the automatic formatter using the project’s conventions. Done. Sure, manual formatting can look better, but at what cost (and tedium)?

There are some places where the automatic formatter always fails, though, for example string concatenation for SQL commands. No problem, use @formatter:off and @formatter:on and there you are – until someone comes along who hasn’t enabled the tags. And disabling formatting completely can introduce tabs where there should be spaces, for example.

Fortunately there is an effective low-tech alternative that works everywhere (cross-IDE): comments! Use line comments after each line and the formatter can’t join them:


   "select something from some_table " //
   + " where some_column = ?"

It will still handle indentation.

Categories: Java

ActiveMQ redelivery plugin fails when client side redelivery is active

We recently had serious issues with the ActiveMQ redelivery plugin. Under high load messages would not be redelivered at all (according to the application logs), they went straight to the DLQ. In isolated tests redelivery worked, but not as intended. We would get six redeliveries in a few milliseconds and then nothing. Why?

It turned out to be a conflict between client-side redelivery and broker redelivery. The client side redelivery kicked in and quickly failed six times. Then when the broker got a chance the maximum redelivery count had already been reached, so the message was moved to the DLQ. See this post.

Following the advice in the post we added jms.redeliveryPolicy.maximumRedeliveries=0 to the broker url, and voila! It worked.

Categories: Java

Prevent hawtio from phoning home

Hawtio is bundled in several applications as a management console. In one project we are using it with JBoss EAP 6. However, when the application server starts hawtio attempts to update itself from Github:


Performing a pull in git repository .hawtio/config on remote URL:
https://github.com/hawtio/hawtio-config.git.
Subsequent pull attempts will use debug logging
Failed to pull from the remote git repo with credentials null due:
https://github.com/hawtio/hawtio-config.git:
407 Proxy Authentication Required. This exception is ignored.

I certainly don’t want applications in production to update themselves dynamically with unforseen effects and neither do I want them to phone home. What to do? Fortunately it is possible to control this, see the documentation. Simply add the following Java options:


-Dhawtio.offline=true
-Dhawtio.config.cloneOnStartup=false
-Dhawtio.config.pullOnStartup=false

Problem solved.

Categories: Java

Free toolset for JMS tests released

In 1996 I worked with IBM MQ Series using OS/2 and CICS. That was the first time I worked with performance tuning for messaging. Since then I have been involved in many projects with the same basic goal: make sure it works correctly (no lost messages under any circumstances) and make it fast. It has never been my full-time job, but as a consultant I have been there and done that a few times now. The more memorable have been webMethods, AQ, HornetQ, ActiveMQ/AMQ and (again) AQ. Every time I have ended up writing my own tools. Yes, every time. So, I’ll do it again, but this time I’ll release the code and make it work for as many JMS providers as possible. Or at least the ones I need.

You can find the project, now in 1.0, at GitHub. If you find it useful, please let me know. One major benefit of making the code open source is that the vendors always need to reproduce any issues on their own. With an open source tool that is easier. A widely used tool is even better, as it has some credibility from the start, so spread the word (and the code)!

If I have time during the summer I will write a series of posts on how to use the tools. Let me know if there is anything in particular I should cover!

Categories: Java, Performance

Debug XA transactions in JBoss EAP 6

How do you find out if XA transactions are actually working and if there are problems, how do you find them? Apart from good tests the answer is to read the logs. So, how do you enable detailed logs for XA transactions in JBoss EAP 6? Fire up the cli (jbossctl.sh cli) and run:


/subsystem=logging/logger=com.arjuna.ats.jta:add(level=TRACE)
/subsystem=logging/file-handler=ARJUNA:add(file=
 {"path"=>"arjuna.log",
  "relative-to"=>"jboss.server.log.dir"})
/subsystem=logging/file-handler=ARJUNA:read-resource
/subsystem=logging/logger=com.arjuna.ats.jta:assign-handler(name="ARJUNA")

This will log most of the relevant XA-related events to a new log file, arjuna.log. The noise ratio is very high, so the file will grow quickly. Be sure to disable logging after a while:


/subsystem=logging/logger=com.arjuna.ats.jta:write-attribute(
  name="level", value="WARN")

Sometimes it may be necessary to log even more. These are recommended by Red Hat:


/subsystem=logging/logger=org.jboss.jca:add(level=TRACE)
/subsystem=logging/logger=org.jboss.as.connector:add(level=TRACE)

Obviously that will make the log files grow even faster, so take care! Don’t try this on a busy system in production.

Categories: Java