Archive

Archive for the ‘Linux’ Category

Fighting with biosboot and efi boot in CentOS 7

In two recent Linux installations I have struggled with biosboot and efi boot partitions. At first I didn’t know why they were needed, I have never seen them before, perhaps because I generally use old hardware for my Linux boxes.

The rule is apparently fairly simple. If the system booted in efi mode, the disks must use gpt and an efi boot partition is required. If on the other hand the system booted in legacy mode there are two options. If the disk is larger than 2T then gpt is required and a biosboot partition is needed. For smaller disks it is possible to use msdos without a biosboot or efi boot partition, using only the mbr for the boot loader.

You can tell if the system booted in efi by checking for /sys/firmware/efi. If it exists the system booted in efi, otherwise it booted in legacy mode. The boot method is controlled by bios settings, not by Linux.

I tried to install CentOS 7. For a simple one-disk installation it is a breeze, but what do you do if you want to use software raid? The installer can create boot and root in raid1, but there is no support for creating efi boot or biosboot partitions on multiple drives.

For biosboot I solved it by escaping to a command prompt, (Ctrl-Alt-F2), creating a gpt disk with gdisk with a single 1M partition created in advance for all disks. That worked, but of course the boot loader is only installed on one of the disks, so after a reboot I had to run grub2-install for all disks.

For efi boot I failed. The installer didn’t like my M2 disk and kept complaining. I tried without it, using two other identical disks. That worked better, but I still couldn’t create the partition from the installer and when I used gdisk the installer would either complain that /boot/efi had to exist on a valid gpt disk or that there was an invalid software raid setup on the newly formatted disks. After several attempts, each needing a reboot cycle, I gave up and booted the system in legacy mode, using msdos disks with classic mbr. Everything worked, including the M2 disk.

This should not be rocket science. The end result is well defined, the problem is when you have to work against the installer rather than with it. I realize that most of Red Hats corporate customers use hardware RAID, but there are many servers out there using software raid as well. Better support for mirrored disks in the installer would be great!

Advertisements
Categories: Linux

Compression ratio and time for common tools

One application I’m working with produces horrendous amounts of log files. They are so large that the process of compressing them and moving them to another server actually has a performance impact on the system in spite of nice and ionice. I decided to compare the most common compression tools. At home I’m using bzip2, as I want the best possible compression ratio. Would that be a good idea?

Compression ratio and time

As the table shows bzip2 is frightfully expensive, but it does produce small files. In addition it makes a big difference if gzip is told to optimize for speed (1) or compression (9), or if it runs at the default level (6). Compress is basically an outdated tool and was included as it is sometimes quoted as being faster than gzip. Here that was not the case.

Categories: Linux

Stupid network issues

Why do all network cards work with Windows, but not with Linux or BSD? Well, whatever. I spent a lot of time this summer trying to get two Internet connections from the same Linux box for high availability. No luck, worked for a while and then failed. Just found out that the built-in network interface from Sundance shuts down under moderate load on Linux. That explains why it didn’t work.

Now I want to use the same box for a DMZ. We have a smart TV and I don’t trust it at all. It needs Internet access, but I won’t allow it anywhere near anything important. I don’t trust my phone (unfortunately) and I certainly won’t trust a TV!

Fortunately the Linux box has one free PCI-e slot, so this should be easy. Just install a new network card and go! Well, not quite. I hate VIA on Linux. I bought two network cards as the first failed to deliver, but the final solution was to add some boot options to grub: pci=nomsi,noaer.

There are still warnings about interrupts that nobody cares about (irqpoll might help, but increases power consumption), but for now it seems to work.

Categories: Linux, Networking

Take care with BTRFS

I have long been an avid fan of ZFS, but though I have been running it on Linux for quite some time I’m not ready to trust it there. The upgrades can be painfull. No lost data so far, but several temporary outages with down time. Perhaps BTRFS is ready for prime time? In short no. It seems very promising, but Red Hat still has reservations. I have tested it with non-essential data. The basics are there and there are many cool features, but when I started to use some of them the file system suddenly turned read only. Why? Tried again, several times. Finally found it – old and fixed bug, but the fix is in the 4.x kernels. Way ahead of Red Hat/CentOS 7.
Of course, a read-only file system is much better than a corrupted one, so that was good. Even so I’ll wait a few more years before I trust anything vital to BTRFS, just in case. Having said that the long-term promise is there, so in five years I bet my servers will be running on BTRFS (unless the ZFS on Linux story changes)!

Categories: Linux

Sorting and rotating photos

I have long used my own Java program for sorting and adjusting photos, but now I have found a better solution: jhead for Linux. Combining it with Dropbox and a cron job I can get my uploaded images, move them out of Dropbox (to save space online, I don’t pay and I like to keep my data close), rotate if needed and sort into folders by year, month and day. The jhead command is simple:


jhead -n%Y/%m/%d/%f -autorot -exonly *.*

This sorts photos by the date taken and rotates them as needed. Images without creation date are left in the main folder for manual sorting.

Categories: Linux

Windows-like keyboard shortcuts in Linux Eclipse

I have worked with Windows as my main development platform since Windows 3.1 and the keyboard shortcuts are hardwired by now. Unfortunately Microsoft has failed utterly. In my opinion Windows 7 is the pinnacle from a usability standpoint, Windows 8 was a disaster and Windows 10 is not that much of an improvement. It is time to move on, in particular as Microsoft seems determined to spy on customers.

Java is cross-platform, so I can use Linux. It works well and bash is great, but the keyboard shortcuts are plain wrong. Fortunately there is a solution. For example, to expand a treeview in Eclipse StackOverflow recommends this for GTK 2:


binding "gtk-binding-tree-view" {
    bind "j"        { "move-cursor" (display-lines, 1) }
    bind "k"        { "move-cursor" (display-lines, -1) }
    bind "h"        { "expand-collapse-cursor-row" (1,0,0) }
    bind "l"        { "expand-collapse-cursor-row" (1,1,0) }
    bind "o"        { "move-cursor" (pages, 1) }
    bind "u"        { "move-cursor" (pages, -1) }
    bind "g"        { "move-cursor" (buffer-ends, -1) }
    bind "y"        { "move-cursor" (buffer-ends, 1) }
    bind "p"        { "select-cursor-parent" () }
    bind "Left"     { "expand-collapse-cursor-row" (0,0,0) }
    bind "Right"    { "expand-collapse-cursor-row" (0,1,0) }
    bind "semicolon" { "expand-collapse-cursor-row" (0,1,1) }
    bind "slash"    { "start-interactive-search" () }
}
class "GtkTreeView" binding "gtk-binding-tree-view"

And this for GTK 3:


@binding-set MyTreeViewBinding {
    bind "Left"     { "select-cursor-parent" ()
                      "expand-collapse-cursor-row" (0,0,0) };
    bind "Right"    { "expand-collapse-cursor-row" (0,1,0) };
}
GtkTreeView {
    gtk-key-bindings: MyTreeViewBinding;
}

With Ubuntu the files to edit are found below /usr/share/themes.

Categories: Java, Linux, Windows

Docker pitfalls for Internet-facing hosts

Planning to use Docker on an unprotected Internet-facing host? If so, don’t rush it. It works, but the default installation is probably not what you want.

By default Docker sets up iptables firewall rules for connections between the host and the containers. This is how it works on Ubuntu 14.04 and CentOS 7 and it is probably true for most distributions. The last thing I want on an Internet-facing host is something messing with the firewall!

What to do? A friend at Red Hat recommends overlay networking, for example with flannel as described here for Kubernetes with Fedora. It certainly seems like a much better (safer) option.

In summary, take care and make sure to test the firewall configuration not only when things are stable, but as containers are started and stoppped!

Categories: Linux, Networking