Last week I taught an Advanced Threat Tactics course at the Lonestar Application Security conference. I like to provide ample hands-on opportunities in my courses. The students retain much more this way. I decided to use the class proceeds to build a killer virtual machine server for my students to hack on.
Requirements
My requirements were as follows:
- Run a lot of virtual machines at once (this is not a very specific requirement)
- Headless. I live in a Washington, DC apartment. I do not want to waste room providing a keyboard, monitor, and mouse to administer it. I do not want to travel with these items either.
- Travel friendly. I used this server to teach a class. It needed to travel with me.
- MacOS X friendly. I am not a Windows user.
Hardware
I initially built my server to run VMWare ESXi. I built my server using parts from newegg.com. I read that newegg.com has a terrible record with dead on arrival hard drives, so I bought these from staples.com.
Here’s the part list:
- Case: Shuttle SH67H3 [ $240 ]
- Network Card: Intel EXPI9301CTBLK 10/100/1000Mbps PCI-Express Network Adapter [ $30 ]
- RAM: CORSAIR XMS3 16GB (4 x 4GB) [ $100 ]
- Processor: Intel Core i7-2600 Sandy Bridge 3.4Ghz (3.7GhZ Turbo Boost) Quad-Core [ $300 ]
- Hard drive: Two 1TB Seagate Barracuda disks, 7200 RPM with 32MB cache [ $140 ] *
Total cost? $810
- I picked up two hard drives to allow a RAID-1 configuration
Software
The operating systems problem is an easy one. I own an MSDN subscription. This gives me access to all of Microsoft’s operating systems going back to Windows 3.x and DOS. Missing is Windows 95, 98, and 2000. I believe this is because of a court ruling related to their crippled Java many years ago. I’ve had MSDN since June and I am extremely happy with it.
VMWare ESXi
The virtual machine software was not such an easy story. I tried VMWare ESXi first. It doesn’t need a host operating system, meaning more system resources go to powering my virtual machines. I own VMWare Fusion for MacOS X and I love it. VMWare ESXi seems like the natural choice.
I installed VMWare ESXi, a process that was not without its trials. At some point it no longer recognized my USB keyboard. I found plugging my keyboard into a port in the back of the system allowed the install process to execute smoothly. I quickly learned that the best way to manage ESXi is through the vSphere client. vSphere is only available for Windows. This immediately went against my MacOS X friendly requirement.
Also, I did not know what functionality was missing from VMWare ESXi/vSphere without their paid package. I quickly learned that the missing functionality included the ability to quickly clone virtual machines. For someone setting up a penetration testing lab, this feature is a must. Many people on Twitter came to the rescue with an assortment of hacks to get around this limitation. I don’t like hacks for simple things like cloning a virtual machine.
Also, using vSphere in a Windows virtual machine was painfully slow for me. VMWare ESXi is out for me.
VMWare Workstation
Because I’m already familiar with VMWare Fusion and Workstation for Windows, I opted to try VMWare Workstation. I installed Ubuntu 10.04LTS. I tried export the user interface using X windows and I tried interacting with it through VNC. Even on my local network, both options were too slow. VMWare Workstation quickly failed the test for me.
VirtualBox
The last option I tried was Oracle’s VirtualBox. I should have tried it first. I installed VirtualBox on Ubuntu 10.04LTS server. I didn’t install X windows at all. The entire install was done while logged in remotely from Terminal.app on MacOS X. I then set up phpVirtualBox to administer the system. phpVirtualBox is fantastic. I can easily configure, make linked clones of, snapshot/restore, and start/stop my virtual machines.
Through phpVirtualBox it’s also trivial to create multiple “host-only” networks and assign them to your virtual machines. This is great for creating isolated enclaves bridged only by a dual-homed virtual machine I set up.
VirtualBox also has an RDP server built-in. phpVirtualBox exposes a flash RDP client to allow me to interact with the virtual machines in my browser. The VirtualBox Guest Additions also fixed wonky mouse issues for me.
In the end I went with VirtualBox and phpVirtualBox as my headless virtual machine solution of choice.
One tip: when running VirtualBox, make sure the kvm-intel and kvm-amd kernel modules are not loaded. These modules conflict with VirtualBox and your VMs will not start. Just because they don’t exist on one boot doesn’t mean you won’t see them later. I was bit by this. Save yourself from my pain.
Conclusion
So, with my hardware in place I set up a penetration testing lab. I had a Windows 2008 domain controller, several Windows 2003 servers, several Windows XP and Windows 7 workstations, and a garden variety of Linux boxes for various purposes. I’ve had up to 15 virtual machines running simultaneously and can go higher. This lab passed my “run lots of virtual machines test”.
Using my browser and phpVirtualBox I was able to administer every aspect of the virtual machine creation and configuration process. This system passed the headless requirement.
This box fit into my carry on luggage and TSA let me through the checkpoint with it no problem. At BWI two or three personnel looked at it quizzically, but they let it through. I was quite happy as I didn’t want to check it for fear of what that might do to the system. So this system passed the travel requirement too.
And, thanks to the headless admin through a browser, this system is definitely MacOS X user friendly as well.
Setting up a penetration testing lab is a good way to hone your system administration skills. It’s also a lot of fun and makes a great place to experiment with different attack techniques at a greater scale.
If you have any questions, leave them in the comments.