Improve logging performance with Log4j

Log4j 1.2 is old, but still in very wide use. It works and for most applications performance is good enough. For log-intensive applications there are some tricks. The AsyncAppender can act as a buffer between the application and slower loggers (including FileAppender in some cases – be sure to use a large buffer though!), but that only helps for short bursts.

The FileAppender itself and its subclasses can work in buffered or immediate mode. Buffered mode is enabled with the BufferedIO parameter. When true the appender uses an 8k buffer and flushes the entire buffer to disk when it is full. This is much faster than the default operation, which flushes immediately for every log event. The disadvantage is that log entries are delayed and unless the appender is closed cleanly (a problem with most applications I’ve seen) many log events may be lost.

I have written a flush appender that can help. It can flush the file appenders when an event with high severity is logged (ensuring that errors are written to disk immediately). It can also flush and convert the file appenders to immediate mode when a specific message is logged, for example “Shutting down…”. That way no messages are lost during the shutdown process. The appender is available from Github and Maven Central (name.wramner.log4j:FlushAppender).

Categories: Java

Materialize hint required for complex SELECT in 12c

During migration from Oracle 11gR2 to 12c we encountered a weird error. We had a very complex select, that used WITH clauses, some with GROUP BY. It works in 11g, but with 12c it fails with:


ORA-00979: not a GROUP BY expression

Apparently something has changed. We have not yet found the root cause of the issue, but if we add /*+ materialize */ to the selects in the WITH clauses it works!

If the query is simplified a bit the hint is not required, even though it still has the same GROUP BY clauses. Smells like a bug.

Categories: Oracle

Beware of OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES

What can make a simple query that normally runs in split seconds using the primary key turn into a 19 minute full scan? Well, many things, but in this case non of the usual suspects. The primary key was usable and rebuilding it made no difference. We updated the already recent statistics to no avail. There was no SQL Profile defined and the query did not use hints. It turned out that at some point the parameter OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES had been set and as a result Oracle had happily collected baselines in order to prevent performance regressions, as the sales message goes. Deleting the baseline fixed the problem and the optimizer immediately switched to an index range scan on the primary key.

SQL baselines may serve a purpose, but if you don’t trust the optimizer to do the right thing, why would you trust it to collect baselines automatically in order to guide itself in the future? Beware of OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES and either create the baselines under controlled circumstances or let the optimizer do its job and correct it with profiles where necessary!

Categories: Oracle

Resolve fatal error LNK1123 with Visual Studio 2010 and .NET 4.5

Recently I had to fix a bug in an old C++ application developed with Visual Studio 2010. My computer is newer, so it has .NET 4.5 installed. Guess what? Without any changes the code failed to build with:

LINK: fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt

The problem should be fixed in SP1, but getting that installed was non-trivial without admin rights. Fortunately the issue can be traced to a single file: C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\cvtres.exe. The wrong version is 31,048 bytes, the correct version (from SP1) is 31,056 bytes. The wrong version has a dependency to msvcr100_clr0400.dll whereas the correct version depends on msvcr100.dll. Replace the file (for exampel from another installation with SP1 installed) and the problem is solved! Much faster than patching, not to mention trying to revert from .NET 4.5 to an older version.

Categories: Windows

Oracle Fusion Middleware with Docker

This is one of the best posts I’ve seen in a long time about Oracle Fusion Middleware (SOA Suite) and the Oracle 12c database. It shows how to use Chef and Puppet and Docker to build a container running Oracle Fusion Middleware 12 with Oracle 12c. Excellent and truly useful!

Categories: Oracle, SOA Suite

Where are Docker’s files?

I recently installed Docker on an Ubuntu server. It took me quite some time to find out where Docker stores all its (rather large) files. In my case it turned out to be in /var/lib/docker and I found it in this post. If you want to use a dedicated file system for Docker, perhaps using ZFS, it certainly helps to know where to put it!

Categories: Linux

Sharing EAR libraries with Maven and skinnyWars

This is not a new feature, but I have seen many applications with EAR-files that include the same dependencies in multiple modules, some times with different and possibly conflicting versions. The same library is found in WEB-INF/lib for multiple WAR-files, or both in WEB-INF/lib for a WAR and at the EAR level for EJB modules. This is inefficient and may cause subtle bugs. Ideally all shared libraries should be at the EAR level.

The solution with Maven is to use the skinnyWars option for the EAR:


<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-ear-plugin</artifactId>
      <version>2.9.1</version>
      <configuration>
        <defaultLibBundleDir>lib/</defaultLibBundleDir>
        <skinnyWars>true</skinnyWars>
        <archive>
          <manifestEntries>
            <Implementation-Version>${project.version}</Implementation-Version>
          </manifestEntries>
        </archive>
        <modules>
          <webModule>
            <groupId>com.example</groupId>
            <artifactId>Example-Web</artifactId>
            <bundleFileName>Example-Web.war</bundleFileName>
            <moduleId>Example-Web.war</moduleId>
          </webModule>
          <!-- other modules -->
        </modules>
      </configuration>
    </plugin>
  </plugins>
</build>

Next, the shared libraries must be referenced as dependencies from the EAR’s pom.xml. When there are few shared libraries it is probably best to include them as dependencies both in the EAR and in the other modules. If there are many dependencies they can be extracted to a separate Maven project and configured in a single place. The Maven project with shared dependencies should be defined to use packaging pom and should be referenced as a dependency with type pom from the EAR and the respective modules. That is really all there is to it.

Note! The skinnyWars support has been improved over the years. Other descriptions mention the need to filter out shared libraries from WAR files with includes/excludes rules. I don’t do that. The war files are complete and can be used as is. The Maven EAR plugin is smart enough to remove the shared libraries from the bundled WAR-files when it builds the EAR-file.

Categories: Java
Follow

Get every new post delivered to your Inbox.