Yesterday I struggled with a very frustrating issue. I was writing a NIO server using the excellent Netty framework and controlled it from jconsole using a custom MBean. To terminate the application I would call the MBean’s stop method and that would call the server’s stop method, which would shut everything down. No problems so far, but I needed to release several resources in a certain order and for some mysterious reason the JVM would terminate in the middle of the stop method, skipping several important steps!
I tracked the problem to Bootstrap.releaseExternalResources. A short time after that method was called the application would die. I moved the call around in the method and added sleep statements before and after and plenty of logging. Sure enough, whenever releaseExternalResources had been called the time was up.
Next I tried to find a call to System.exit in the Netty code. There was none. What could be causing this?
In hindsight it is obvious. The MBean server uses a daemon thread. When releaseExternalResources shuts down the Netty thread pools, the last non-daemon thread stops. At that point the JVM terminates.
I solved the problem by starting a new non-daemon thread from the MBean specifically for running the stop operation.