Billix: a Sysadmin's Swiss Army Knife

Does anyone remember Linuxcare? Founded in 1998, Linuxcare was a company that provided support services for Linux users in corporate environments. I remember seeing Linuxcare at the first ever LinuxWorld conference in San Jose, and the thing I took away from that LinuxWorld was the Linuxcare Bootable Business Card (BBC). The BBC was a 50MB cut-down Linux distribution that fit on a business-card-size compact disc. I used that distribution to recover and repair quite a few machines, until the advent of Knoppix. I always loved the portability of that little CD though, and I missed it greatly until I stumbled across Damn Small Linux (DSL) one day.

After reading through the DSL Web site, I discovered that it was possible to run DSL off of a bootable USB key, and that old love for the Bootable Business Card was rekindled in a new way. It wasn't until I had a conversation with fellow sysadmin Kyle Rankin about the PXE boot environment he'd implemented, that I realized it might be possible to set up a USB key to do more than merely boot a recovery environment. Before long, I had added the CentOS and Ubuntu netinstalls to my little USB key. Not long after that, I was mentioning this in my favorite IRC channel, and one of the fellows in there suggested I put the code on SourceForge and call it Billix. I'd had a couple beers by then and thought it sounded like a great idea. In that instant, Billix was born.

Billix is an aggregation of many different tools that can be useful to system administrators, all compressed down to fit within a 256MB bootable USB thumbdrive. The 256MB size is not an arbitrary number; rather, it was chosen because USB thumbdrives are very inexpensive at that size (many companies now give them away as advertising gimmicks). This allows me to have many Billix keys lying around, just waiting to be used. Because the keys are cheap or free, I don't feel bad about leaving one in a server for a day or two. If your USB drive is larger than 256MB, you still can use it for its designed purpose—storing files. Billix doesn't hamper normal use of the USB drive in any way. There also is an ISO distribution of Billix if you want to burn a CD of it, but I feel it's not nearly as convenient as having it on a USB key.

The current Billix distribution (0.21 at the time of this writing) includes the following tools:

So, with one USB key, a system administrator can recover or repair a machine, install one of eight different Linux distributions, test the memory in a system, get into a Windows machine with a lost password or wipe the disks of a machine before repurpose or disposal. In order to install any of the netinstall-based Linux distributions, a working Internet connection with DHCP is required, as the netinstall downloads the installation bits for each distribution on the fly from Internet-based mirrors.

Hopefully, you're excited to check out Billix. You simply can download the ISO version and burn it to a CD to get started, but the full utility of Billix really shines when you install it on a USB disk. Before you install it on a USB disk, you need to meet the following prerequisites:

To install the USB-based version of Billix, first check your drive. If that drive has the U3 Windows software on it, you may want to remove it to unlock all of the drive's capacity (see the Resources section for U3 removal utilities, which are typically Windows-based). Next, if your USB drive has data on it, back up the data. I cannot stress this enough. You will be making adjustments to the partition table of the USB drive, so backing up any data that already is on the key is critical. Download the latest version of Billix from the Sourceforge.net project page to your computer. Once the download is complete, untar the contents of the tarball to the root directory of your USB drive.

Now that the contents of the tarball are on your USB drive, you need to install a Master Boot Record (MBR) on the drive and set a bootsector on the drive. The Master Boot Record needs to be set up on the USB drive first. Issue a install-mbr -p1 <device> (where <device> is your USB drive, such as /dev/sdb). Warning: make sure that you get the device of the USB drive correct, or you run the risk of messing up the MBR on your system's boot device. The -p1 option tells install-mbr to set the first partition as active (that's the one that will contain the bootsector).

Next, the bootsector needs to be installed within the first partition. Run syslinux -s <device/partition> (where <device/partition> is the device and partition of the USB drive, such as /dev/sdb1). Warning: much like installing the MBR, installing the bootsector can be a dangerous operation if you run it on the wrong device, so take care and double-check your command line before pressing the Enter key.

Figure 1. The Billix Boot Menu

At this point, your USB drive can be unmounted safely, and you can test it out by booting from the USB drive. Once your system successfully boots from the USB drive, you should see a menu similar to the one shown in Figure 1. Simply choose the number for what you want to boot, run or install, and that distribution will spring into action. If you don't select a number, Damn Small Linux will boot automatically after 30 seconds.

Damn Small Linux is a miniature version of Knoppix (it actually has much of the automatic hardware-detection routines of Knoppix in it). As such, it makes an excellent rescue environment, or it can be used as a quick “trusted desktop” in the event you need to “borrow” a friend's computer to do something. I have used DSL in the past to commandeer a system temporarily at a cybercafé, so I could log in to work and fix a sick server. I've even used DSL to boot and mount a corrupted Windows filesystem, and I was able to save some of the data. DSL is fairly full-featured for its size, and it comes with two window managers (JWM or Fluxbox). It can be configured to save its data back to the USB disk in a persistent fashion, so you always can be sure you have your critical files with you and that it's easily accessible.

All the Linux distribution installations have one thing in common: they are all network-based installs. Although this is a good thing for Billix, as they take up very little space (around 10MB for each distro), it can be a bad thing during installation as the installation time will vary with the speed of your Internet connection. There is one other upside to a network-based installation. In many cases, there is no need to update the newly installed operating system after installation, because the OS bits that are downloaded are typically up to date. Note that when using the Red Hat-based installers (CentOS 4.6, CentOS 5.1 and Fedora 8), the system may appear to hang during the download of a file called minstg2.img. The system probably isn't hanging; it's just downloading that file, which is fairly large (around 40MB), so it can take a while depending on the speed of the mirror and the speed of your connection. Take care not to specify the USB disk accidentally at the install target for the distribution you are attempting to install.

The memtest86 utility has been around for quite a few years, yet it's a key tool for a sysadmin when faced with a flaky computer. It does only one thing, but it does it very well: it tests the RAM of a system very thoroughly. Simply boot off the USB drive, select memtest from the menu, and press Enter, and memtest86 will load and begin testing the RAM of the system immediately. At this point, you can remove the USB drive from the computer. It's no longer needed as memtest86 is very small and loads completely into memory on startup.

The ntpwd Windows password “cracking” tool can be a controversial tool, but it is included in the Billix distribution because as a system administrator, I've been asked countless times to get into Windows systems (or accounts on Windows systems) where the password has been lost or forgotten. The ntpwd utility can be a bit daunting, as the UI is text-based and nearly nonexistent, but it does a good job of mounting FAT32- or NTFS-based partitions, editing the SAM account database and saving those changes. Be sure to read all the messages that ntpwd displays, and take care to select the proper disk partition to edit. Also, take the program's advice and nullify a password rather than trying to change it from within the interface—zeroing the password works much more reliably.

DBAN (otherwise known as Darik's Boot and Nuke) is a very good “nuke it from orbit” hard disk wiper. It provides various levels of wipe, from a basic “overwrite the disk with zeros” to a full DoD-certified, multipass wipe. Like memtest86, DBAN is small and loads completely into memory, so you can boot the utility, remove the USB drive, start a wipe and move on to another system. I've used to this to wipe clean disks on systems before handing them over to a recycler or before selling a system.

In closing, Billix may not make you coffee in the morning or eradicate Windows from the face of the earth, but having a USB key in your pocket that offers you the functionality to do all of those tasks quickly and easily can make the life of a system administrator (or any Linux-oriented person) much easier.