Archive

Posts Tagged ‘EAP6’

Debug XA transactions in JBoss EAP 6

How do you find out if XA transactions are actually working and if there are problems, how do you find them? Apart from good tests the answer is to read the logs. So, how do you enable detailed logs for XA transactions in JBoss EAP 6? Fire up the cli (jbossctl.sh cli) and run:


/subsystem=logging/logger=com.arjuna.ats.jta:add(level=TRACE)
/subsystem=logging/file-handler=ARJUNA:add(file=
 {"path"=>"arjuna.log",
  "relative-to"=>"jboss.server.log.dir"})
/subsystem=logging/file-handler=ARJUNA:read-resource
/subsystem=logging/logger=com.arjuna.ats.jta:assign-handler(name="ARJUNA")

This will log most of the relevant XA-related events to a new log file, arjuna.log. The noise ratio is very high, so the file will grow quickly. Be sure to disable logging after a while:


/subsystem=logging/logger=com.arjuna.ats.jta:write-attribute(
  name="level", value="WARN")

Sometimes it may be necessary to log even more. These are recommended by Red Hat:


/subsystem=logging/logger=org.jboss.jca:add(level=TRACE)
/subsystem=logging/logger=org.jboss.as.connector:add(level=TRACE)

Obviously that will make the log files grow even faster, so take care! Don’t try this on a busy system in production.

Advertisements
Categories: Java

Oracle JDBC memory settings

Many applications use ancient versions of Oracle’s JDBC drivers, as they are downloaded manually and seldom upgraded. That is a pity as the newer drivers offer much better performance. However, some of the performance gains are bought with increased memory consumption and that can be a problem.

We ran into an issue with the connection pool in JBoss. It uses connections in round-robin, so as long as there is some load the pool tends to stay at peak size. Each connection normally keeps a buffer cache and it may cache other things as well. The size of the connections would grow over time, eventually consuming most of the heap.

A JBoss-specific workaround is to use the cli and close the idle connections (off-peak):


/subsystem=datasources/xa-data-source=DS/:flush-idle-connection-in-pool

DS is the name of the data source. A better approach may be to set the system property oracle.jdbc.useThreadLocalBufferCache to true. That moves the buffer cache from the connections to thread locals. Depending on how the application behaves that may be better as the memory is reclaimed when a thread dies. Plus a given thread may be more likely to issue the same SQL multiple times and can thus benefit more from the cache.

It may also be useful to limit the maximum buffer size with oracle.jdbc.maxCachedBufferSize. The implicit statement cache is normally off, but if it is enabled the size can be controlled using oracle.jdbc.implicitStatementCacheSize and oracle.jdbc.freeMemoryOnEnterImplicitCache can force buffers allocated for a statement to be released when it is put into the cache. In most cases it is best to leave those options alone.

See Oracle JDBC Memory Management for the whole story!

Categories: Java, Oracle, Performance

JBoss EAP6 JGroups MPING fails with invalid argument

What to do if JBoss EAP6 fails to discover other cluster members with the following error from JGroups?


[org.jgroups.protocols.MPING] failed sending discovery request:
  java.io.IOException: Invalid argument
  at java.net.PlainDatagramSocketImpl.send(Native Method)
  at java.net.DatagramSocket.send(DatagramSocket.java:693)
  at org.jgroups.protocols.MPING.sendMcastDiscoveryRequest(MPING.java:300)

In my case the solution was simple. Add -Djava.net.preferIPv4Stack=true to jbossctl.sh. Apparently the multicast code doesn’t work with IPv6 on my Linux version.

Categories: Java, Networking

Timeouts for Oracle XA datasources in JBoss EAP 6

The documentation for configuring datasources in JBoss EAP 6 is somewhat lacking when it comes to timeouts. Normally this is fine, but what if there are network issues? With the wrong timeout settings the application can hang until it is killed and restarted. With proper timeouts it can handle an outage and recover.

Here is an example:


<xa-datasource jndi-name="java:/AppDS" pool-name="AppDS">
  <xa-datasource-property name="URL">
    ${appdb.url}
  </xa-datasource-property>
  <xa-datasource-property name="nativeXA">true</xa-datasource-property>
  <xa-datasource-property name="ConnectionProperties">
    oracle.jdbc.ReadTimeout=330000
  </xa-datasource-property>
  <xa-datasource-class>
    oracle.jdbc.xa.client.OracleXADataSource
  </xa-datasource-class>
  <driver>oracle</driver>
  <security>
    <user-name>${appdb.user}</user-name>
    <password>${appdb.password}</password>
  </security>
  <xa-pool>
    <min-pool-size>${appdb.min.pool.size}</min-pool-size>
    <max-pool-size>${appdb.max.pool.size}</max-pool-size>
    <prefill>false</prefill>
    <use-strict-min>false</use-strict-min>
    <flush-strategy>FailingConnectionOnly</flush-strategy>
    <is-same-rm-override>false</is-same-rm-override>
    <no-tx-separate-pools/>
    <pad-xid>true</pad-xid>
    <wrap-xa-resource>true</wrap-xa-resource>
  </xa-pool>
  <validation>
    <valid-connection-checker class-name=
      "org.jboss.jca.adapters.jdbc.extensions.oracle.OracleValidConnectionChecker"/>
    <stale-connection-checker class-name=
      "org.jboss.jca.adapters.jdbc.extensions.oracle.OracleStaleConnectionChecker"/>
    <exception-sorter class-name=
      "org.jboss.jca.adapters.jdbc.extensions.oracle.OracleExceptionSorter"/>
  </validation>
  <timeout>
    <blocking-timeout-millis>60000</blocking-timeout-millis>
    <xa-resource-timeout>310</xa-resource-timeout>
    <query-timeout>300</query-timeout>
    <set-tx-query-timeout/>
  </timeout>
  <statement>
    <track-statements>false</track-statements>
  </statement>
  <recovery no-recovery="false">
    <recover-credential>
      <user-name>${appdb.user}</user-name>
      <password>${appdb.password}</password>
    </recover-credential>
  </recovery>
</xa-datasource>

The oracle.jdbc.ReadTimeout is essential. It sets the network timeout on the socket, making reads time out eventually in the face of a broken connection. The default TCP timeout for established connections is very long, so it is important to set a value. It should be larger than the transaction timeouts. The query timeout and tx-query-timeout both limit the time that individual statements can take. The query timeout is used when there is no transaction, otherwise the remaining time until transaction timeout is used.

Note that read timeout is in milliseconds, query timeout is in seconds.

Categories: Java, Networking, Oracle