Archive for the ‘Networking’ Category

VirtualBox intermittent network outages

For about a year I have had terrible problems with short but very frequent network outages between Linux guests running in VirtualBox and the world at large at a customer site. At home it works, but there the connection is lost and then restored every few minutes. Very frustrating. Today I finally found a solution. It appears that it is a known bug that goes way back, see ticket 13839. Sure enough, changing the virtual network card to PCnet-PCI II solved the issue! No more outages.

Categories: Networking

JBoss EAP6 JGroups MPING fails with invalid argument

What to do if JBoss EAP6 fails to discover other cluster members with the following error from JGroups?

[org.jgroups.protocols.MPING] failed sending discovery request: Invalid argument
  at Method)
  at org.jgroups.protocols.MPING.sendMcastDiscoveryRequest(

In my case the solution was simple. Add to Apparently the multicast code doesn’t work with IPv6 on my Linux version.

Categories: Java, Networking

Timeouts for Oracle XA datasources in JBoss EAP 6

The documentation for configuring datasources in JBoss EAP 6 is somewhat lacking when it comes to timeouts. Normally this is fine, but what if there are network issues? With the wrong timeout settings the application can hang until it is killed and restarted. With proper timeouts it can handle an outage and recover.

Here is an example:

<xa-datasource jndi-name="java:/AppDS" pool-name="AppDS">
  <xa-datasource-property name="URL">
  <xa-datasource-property name="nativeXA">true</xa-datasource-property>
  <xa-datasource-property name="ConnectionProperties">
    <valid-connection-checker class-name=
    <stale-connection-checker class-name=
    <exception-sorter class-name=
  <recovery no-recovery="false">

The oracle.jdbc.ReadTimeout is essential. It sets the network timeout on the socket, making reads time out eventually in the face of a broken connection. The default TCP timeout for established connections is very long, so it is important to set a value. It should be larger than the transaction timeouts. The query timeout and tx-query-timeout both limit the time that individual statements can take. The query timeout is used when there is no transaction, otherwise the remaining time until transaction timeout is used.

Note that read timeout is in milliseconds, query timeout is in seconds.

Categories: Java, Networking, Oracle

Stupid network issues

Why do all network cards work with Windows, but not with Linux or BSD? Well, whatever. I spent a lot of time this summer trying to get two Internet connections from the same Linux box for high availability. No luck, worked for a while and then failed. Just found out that the built-in network interface from Sundance shuts down under moderate load on Linux. That explains why it didn’t work.

Now I want to use the same box for a DMZ. We have a smart TV and I don’t trust it at all. It needs Internet access, but I won’t allow it anywhere near anything important. I don’t trust my phone (unfortunately) and I certainly won’t trust a TV!

Fortunately the Linux box has one free PCI-e slot, so this should be easy. Just install a new network card and go! Well, not quite. I hate VIA on Linux. I bought two network cards as the first failed to deliver, but the final solution was to add some boot options to grub: pci=nomsi,noaer.

There are still warnings about interrupts that nobody cares about (irqpoll might help, but increases power consumption), but for now it seems to work.

Categories: Linux, Networking

Docker pitfalls for Internet-facing hosts

Planning to use Docker on an unprotected Internet-facing host? If so, don’t rush it. It works, but the default installation is probably not what you want.

By default Docker sets up iptables firewall rules for connections between the host and the containers. This is how it works on Ubuntu 14.04 and CentOS 7 and it is probably true for most distributions. The last thing I want on an Internet-facing host is something messing with the firewall!

What to do? A friend at Red Hat recommends overlay networking, for example with flannel as described here for Kubernetes with Fedora. It certainly seems like a much better (safer) option.

In summary, take care and make sure to test the firewall configuration not only when things are stable, but as containers are started and stoppped!

Categories: Linux, Networking

Routing for multiple uplinks with CentOS 7

How do you configure a Linux host to connect a local network to the Internet through multiple providers? Linux Advanced Routing and Traffic Control HOWTO is a good starting point. However, it only describes the commands, not where to put them.

With Red Hat and CentOS 7 the commands are split between several files. Following the HOWTO, assume that eth0 is connected to the LAN, eth1 to provider 1 and eth2 to provider 2.

Edit /etc/iproute2/rt_tables and add one line for T1 and one for T2. This creates two custom routing tables.

Optionally edit /etc/sysconfig/network and add NOZEROCONF=yes to get rid of the zero-config routing rules that are created by default.

In /etc/sysconfig/network-scripts/ there should be configuration files for all network cards: ifcfg-eth0, ifcfg-eth1 and ifcfg-eth2. Edit the files and remove GATEWAY, as we will add our gateways manually.

Create three new files named route-eth0, route-eth1 and route-eth2. They should contain the routing commands from the HOWTO. For example the route-eth1 could contain:

$P1_NET dev eth1 src $IP1 table T1
default via $P1 table T1
$P1_NET dev eth1 src $IP1

Optionally add a default as well (see the HOWTO for more advanced setups):

default via $P1

This covers the routes. Create rule-eth1 and rule-eth2 with the rules, for example rule-eth1:

from $IP1 table T1

Try it out and see how the routing tables change when a network interface is started or stopped.

This is just a starting point. My goal is to point out where the configuration in the HOWTO should go in Red Hat/CentOS 7, not to create a full-blown configuration. Good luck!

Categories: Linux, Networking

Dynamic ports in Windows

Recently we had some issues with a Windows 2008 server. The dynamic port range used by the server did not match the firewall rules. The dynamic port range is used when an application listens on port 0 in order to get an arbitrary free port. I turns out that it is quite easy to find and set the port range:

netsh int ipv4 show dynamicport tcp

The same syntax applies for IPv6 and UDP as well. To set the port range, use a similar command:

netsh int ipv4 set dynamicport tcp start=40000 num=1000

This sets the port range to 40000-41000. The smallest range of ports possible is 255 and the highest port number can’t exceed 65535.

Categories: Networking, Windows

How to disable link detection for Windows 7

Modern network cards and operating systems usually support media sense or link detection, where the system detects if a network cable is plugged in or not. Often this is beneficial, but there are drawbacks as well. As a developer I’m often working with server software on my laptop and it can be tricky to get it working without a network.

Fortunately the feature can be disabled. For Windows 7, open a command prompt as an administrator and run:

netsh interface ipv4 set global dhcpmediasense=disabled
netsh interface ipv6 set global dhcpmediasense=disabled

Reboot. The network will still be shown as disconnected by ipconfig, but it will be possible to ping the address (if a static or alternate IP address is used) and run server software that depends on it.

To restore media sense, run:

netsh interface ipv4 set global dhcpmediasense=enabled
netsh interface ipv6 set global dhcpmediasense=enabled

Again a reboot is required.

Categories: Networking, Windows

Proxy the proxy

Many corporations prevent direct Internet connections from the internal network, enforcing the use of a proxy server. That does not have to be an issue, but in Windows shops the proxy often requires Windows (NTLM) authentication. Few Java applications support that, there are even many native Windows applications that fail.

Want to upgrade Eclipse or jDeveloper or install a cool plugin? Not very convenient, everything must be downloaded and installed manually. Want to use Maven? Again, all dependencies must be downloaded and installed manually.

Enter Cntlm, a small and efficient proxy server that supports NTLM. Install it locally and use it as a go-between. Applications such as Eclipse can connect to Cntlm without authentication and Cntlm talks to the official proxy.

There is only one snag. Cntlm needs a user id and password, or at least a password hash. Be sure to stop or update Cntlm before changing your password, or you may be locked out!

Categories: Networking

Persistent static routes

I wanted to direct traffic to the network 192.168.1 via rather than via the default gateway and this is how I did it. It is kind of basic, but it varies between platforms.

In Windows it is very easy, simply run:

route -p add mask metric 10

The route is added immediately and will survive restarts.

For Red Hat Linux, CentOS and their relatives, create a file in /etc/sysconfig/network-scripts named route-ethX where X is the interface number. In my case I created the file /etc/sysconfig/network-scripts/route-eth0. The file should contain:

where N is the route number. In my example:


Finally for Ubuntu and other Debian-related Linux versions, modify the file /etc/network/interfaces and add an up command. In my case I added:

up route add -net netmask gw eth0

Including the other lines for eth0 this resulted in:

iface eth0 inet static
up route add -net netmask gw eth0
auto eth0
Categories: Linux, Networking, Windows