Hack and /

It used to be that the true sign you were dealing with a Linux geek was the pile of computers lying around that person's house. How else could you experiment with networked servers without a mass of computers and networking equipment? If you work as a sysadmin for a large company, sometimes one of the job perks is that you get first dibs on decommissioned equipment. Through the years, I was able to amass quite a home network by combining some things I bought myself with some equipment that was too old for production. A major point of pride in my own home network was the 24U server cabinet in the garage. It had a gigabit top-of-rack managed switch, a 2U UPS at the bottom, and in the middle was a 1U HP DL-series server with a 1U eSATA disk array attached to it. Above that was a slide-out LCD and keyboard in case I ever needed to work on the server directly.

The 1U server acted as my primary server for just about everything. It was the gateway router, local mail relay and secondary MX for my personal domains, DNS server, DHCP server, and with the eSATA array, it became our home NAS (Network Attached Storage) array that we used for general file storage and backups. Everything generally worked well, and if you ignored the power bill and the space it took up, it was quite the impressive setup in its day.

The key phrase here is “in its day”, because these days, a combination of virtualization and cloud computing means you are more likely to see a Linux geek with a laptop than a pile of servers. As computers have become faster, smaller and cheaper, my 1U server was starting to show its age. Given that all of my eggs were in this basket, I started wondering about what I'd do if one of the expensive components on the server failed. Although the server had been stable up to this point, I realized it wouldn't last forever, and if it did break, I could buy modern hardware for the cost of replacing, for instance, one of its fancy serial-attached SCSI drives. I ended up researching a lot of different options, and in this article, I describe how I ended up replacing that 24U cabinet and all the hardware in it with something that's smaller than a shoe box, much lower-powered and relatively cheap.

At the beginning of my search, I started down a more traditional route with a cheap 1U server and a modern motherboard, but I quickly started narrowing down the motherboards to small, lower-power solutions given this machine was going to run all day. As I started considering some of the micro ATX solutions out there, it got me thinking: could I use a Raspberry Pi? After all, the latest iteration of the Raspberry Pi has a reasonably fast processor, a decent amount of RAM, and it's cheap, so even if one by itself wasn't enough to manage all my services, two or three might do the trick and not only be cheaper than a standard motherboard but lower power as well.

If you have been reading my column through the years, you'll know I'm no stranger to solving problems with Raspberry Pis—whether it's controlling the temperature of a beer fridge, creating a gaming media center, flashing coreboot onto an X200 or controlling my 3D printer. Unfortunately when you are talking about a home server, in particular a NAS, even recent Raspberry Pis have some limitations.

The first limitation with a Raspberry Pi isn't the CPU or the RAM, but the network card. A 10/100 network card is fast enough for some services around the house, but if you are setting up a NAS these days, those big media files demand a gigabit network, and the USB2 port on a Raspberry Pi isn't fast enough to drive a USB gigabit NIC.

The second limitation is disk I/O. Even if a Raspberry Pi had a gigabit NIC, your storage options are limited to what you can fit on a microSD card or a hard drive hanging off one of the USB2 ports, and USB2 is just too slow for a modern NAS. If a Raspberry Pi had USB3, you could bypass the network limitations with a USB3 gigabit NIC to it, but as it stands, the I/O is just too slow to replace even an old 1U server that has eSATA disks on a gigabit network.

So a Raspberry Pi was out of the race, but that got me thinking—I knew there were other cheap ARM single-board computers out there that had different hardware options. If I could find one with decent network and storage I/O, maybe it could be a contender.

One of the first places I ended up when searching for a Raspberry Pi with faster I/O was the Banana Pi. Despite the similar name, this project is completely unrelated to Raspberry Pi, even though the board is a similar size and price ($35), and it has a dual-core 1GHz ARM Cortex A-7 processor in it with 1GB RAM, so even its specs were similar to some of the older Raspberry Pi revisions.

The big difference between a Banana Pi and Raspberry Pi though was the fact that it touted both a gigabit network and a SATA2 port! That meant I could hang one of the large hard drives from my old NAS off of a Banana Pi. This got me thinking. My current solution had a number of large hard drives in a software RAID5. While any individual drive wouldn't be large enough for my files, I could buy one of the newer larger drives out there, and that still would be cheaper than some of the traditional solutions.

Of course, a single Banana Pi with a single drive wouldn't solve my fault-tolerance problems. If the drive failed, I would be sunk. Given how cheap the Banana Pis were, I started applying some cloud computing approaches to my home network. Instead of worrying about an individual, potentially unstable server, what if I split the load and fault tolerance across multiple Banana Pis? In a prior article, I walked through creating a GlusterFS cluster on Raspberry Pis, and I realized that Banana Pis would work even better in that case.

I still wasn't sure how well it would work, but I was sure enough that I first bought one Banana Pi to put it through its paces, and when I was reasonably satisfied with its performance, I bought a second one and set them up in a two-node GlusterFS cluster. I've used GlusterFS for a number of years, and what I've learned during those years is that setting up GlusterFS is easy; it's the maintenance—particularly when dealing with faults on an unreliable network—that's hard.

I started to realize that if I really wanted a chance at this cluster being reliable, I would need to add a third Banana Pi to the cluster. Among other things, a third node would help combat split brain—a scenario when each member of a two-node cluster thinks it's the master—and it would help distribute the load. Although a Banana Pi has enough power to run a 2.5" hard drive off the motherboard, the 3.5" hard drives I wanted to use required a separate power supply. As I started adding up all of the Banana Pis, this arrangement was beginning to look kind of clunky, and I started worrying about the overall network load when a new large file was uploaded to the server and it had to be sharded and shared. I started wondering if I was making this a bit too complicated.

It was around this point that I started researching other small ARM computers that might offer more ports or more resources, and I ran across the ODROID XU4. I had been somewhat familiar with the ODROID line of computers in the past, but up to this point, I hadn't really had a need for one.

The ODROID XU4 in particular caught my attention, because although it was between two to three times as much as a Banana Pi (the cheapest I found was $75), it had double the RAM (2GB) and an eight core 2GHz ARM processor. Now this is some pretty respectable hardware that had a better chance of handling all of the load from my previous four-core AMD 1U server. It also had a gigabit NIC, and although it didn't have any SATA ports, it did have some USB3 ports, which offer similar bandwidth both to the SATA2 port on the Banana Pi and the eSATA port I was using on my old 1U server.

Since the ODROID XU4 used USB3 instead of SATA, I started pricing out standalone USB3 enclosures for my existing hard drive. While I was looking up 3D-printed cases for the ODROID XU4, I noticed one particular project on Thingiverse where someone had come up with a design for a case that mounted your ODROID XU4 on an existing USB3 drive enclosure. This case was designed for the Mediasonic ProBox HF2-SU3S2, which was a four-drive USB3 and eSATA enclosure that ran about $100—around the amount of money I was going to spend on a few standalone USB3 enclosures for my 3.5" SATA drives.

More important with this enclosure than the price was that I realized since the drives would be presented to the computer individually, even though they were connected through a singular USB3 port, I could maintain the existing software RAID I had in place! No multiple levels of backups and restore or Linux software RAID voodoo to go through—I could just plug in and go like with my 1U eSATA array. That ease of migration pushed me over the edge, so even though I had a couple Banana Pis in the house, I decided to order an ODROID XU4 and the Mediasonic enclosure. I ended up printing out the special Mediasonic ODROID XU4 case while I was waiting on the hardware to arrive.

The good thing about most ARM boards these days is they all tend to provide at least standard Debian images, so once my board arrived, it was simple to set up a Debian server similar to my existing one and port over all of my configuration files. The day I set up the big server move was actually pretty uneventful. Because my RAID configuration file already was copied over to the new server, it was just a matter of moving the drives to the new array and mounting it. The rest of my services worked out of the box after I copied the configuration files over.

The eight-core processor so far has been more than enough for all of the standard tasks I put my home server through, and this machine serves as a DNS server, secondary MX, home NAS and a number of other services without skipping a beat. I'm sure if I did a lot of media transcoding or something I might notice some slowdown with ARM versus a classic Intel processor, but after a few months with this new set up for all of the things I do, it seems more than adequate.

Really the main thing I miss on this new setup is virtualization. I have a particular image gallery I use to share pictures with my family, and it hasn't kept up with the times, so I've found myself having to run it in a virtual machine on an old version of Ubuntu Server. Also, I really wanted to set up a separate backup server instead of running my backups on the same main machine. Of course, I had those Banana Pis just lying around, so I was able to use one for my classic image gallery and attach a SATA drive to the other one and turn it into a nice standalone backup server for my important files.

Figure 1. My New Server Rack

The real payback on this solution though is in power savings. These ARM processors sip power compared to my old server, and I figure I'll be able to pay for the whole upgrade in power savings alone during the year. Plus, I ended up selling my old server cabinet and freed up a lot of space in my garage. This new solution fits on a shelf or two off in the corner. All in all, it's been nice to get with the times and use new, small, low-power hardware. Plus, I know if I have a hardware problem now, replacement hardware is low-cost and easy to come by.