Ben's blog

So here we are, 2012 and physical media are going away really fast. We won’t even talk about CDs which have been relegated to the role of plastic dust collectors; hard drives even are being abstracted by a myriad of cloud based solutions. Their purpose is shifting towards a container for the OS and nothing else. Filesystems & their hierarchies become hidden in a bid to remove any need to organize files, rather, you are supposed to throw it all up in the cloud and search on metadata.

While moving away from physical media is convenient and inevitable, I like the hierarchical organization that directories provide. What’s more intuitive than a labeled container with stuff in it?

How can we detach our hard drives from their physical shells, move them around in an omnipresent cloud and keep them secure?

By creating a file, attaching it to loopback & creating an encrypted partition in it!

Here’s how to do it

Create a file that will be your soft hard drive with:

dd if=/dev/zero of=/tmp/ffs bs=1024 count=524288

This will create a 512MB file (524288/1024).

Make sure that the loopback device #0 is free:

losetup /dev/loop0

You should see something telling you that there is “No such device or address”.

Attach the soft hard drive to the loopback device:

sudo losetup /dev/loop0 /tmp/ffs

And then make sure it was indeed attached by re-running:

losetup /dev/loop0

Create an encrypted partition on your attached soft hard drive:

sudo cryptsetup --verify-passphrase luksFormat /dev/loop0 -c aes -s 256 -h sha256

Open your encrypted partition:

sudo cryptsetup luksOpen /dev/loop0 ffs

Create a filesystem in it:

sudo mkfs.ext3 -m 1 /dev/mapper/ffs

And mount it like a regular disk:

sudo mount /dev/mapper/ffs /mnt

When you are done using your encrypted soft hard drive you will want to umount it:

sudo umount /mnt

Close it:

sudo cryptsetup luksClose ffs

Detach it from loopback:

losetup -d /dev/loop0

These steps can be automated of course. As a quick reminder, using the drive goes “loopback attach -> crypt open -> mount” and when you’re done it’s “umount -> crypt close -> loopback detach”.

That’s it! media-less & secure storage.

Tested on: Ubuntu 12.04 64b

Download

crackzor_1.1.c.gz

Previous versions:

crackzor_1.0.c.gz

Quick start

Download & extract with “tar zxvf crackzor_1.0.tar.gz”

Make sure you have the right packages in place

sudo apt-get install build-essential libopenmpi-dev openmpi-bin libssl-dev

Compile with

mpicc -O3 crackzor.c -o crackzor -lm -lssl -lcrypto

Create a file called “machines” containing a newline separated list of every machine that are in your cluster, for example:
```
machine00.domain.com
machine01.domain.com
machine02.domain.com
machine03.domain.com
machine04.domain.com
```
Open MPI uses SSH for communication between nodes, as such, you need to make sure that the node you will be launching crakzor from is able to do SSH key based authentication to all the other nodes in the cluster. For my example above, if machine00 is where you will be working from, you will want to
```
ssh-copy-id machine0X.domain.com
```
where X E [0,4] (yes, machine00 needs to be able to SSH to itself).
You now need to disseminate your executable across all the machines that will be running it:
```
for machine in `cat machines`; do scp crackzor $machine:~; done
```
Pro-tip: having network storage attached to all the machines makes this step unnecessary.
Run with:
```
mpirun -npernode Y -machinefile machines crackzor fbade9e36a3f36d3d676c1b808451dd7 abcdefghijklmnopqrstuvwxzy 1 1
```
where Y is the number of cores each machine in your cluster has. If you are running this on machines with 2 CPUs with 8 cores each, Y = 8 * 2 = 16.

Tested on Ubuntu 10.04 64b / Ubuntu 12.04 64b / Ubuntu 14.04 64b

Screenshots

mpirun -npernode 16 -machinefile machines ./crackzor 7ca4793dcdff46ecda38e48d65b6c913 abcdefghijklmnopqrstuvwxzyABCDEFGHIJKLMNOPQRSTUVWXYZ 1 7

This is what “htop” looks like with a bunch of processes spawned & hammering every core:

Statistics

For the purpose of testing crackzor, we give it the md5 hash of an 8 character word and tell it to bruteforce it up to 7 characters. This insures that we will compute every permutation up to 7 character longs. The characters I asked it to permute are “abcdefghijklmnopqrstuvwxzyABCDEFGHIJKLMNOPQRSTUVWXYZ”, our sample space size is thus 52^7 + 52^6 + 52^5 + 52^4 + 52^3 + 52^2 + 52^1 = 1,048,229,971,204.

Here is the raw data, and here it is graphed:

I wish it would show the linear progression more but 3 things got in the way:

approaching the machine’s actual number of cores on the Dell blades leaves little room for linear expansion
which is emphasized in a multiuser environment where other users run other computation
the EC2 bar flattens the graph a bit but I still wanted to show how it compares

Ideally I would run through a few iterations of EC2 to observe its progression but hey, it’s expensive :).

Limitations

Right now, the only hashing algorithm supported by crackzor is MD5. It can very easily be expanded upon.
I also may not be using the fastest MD5 method with the fastest call, distribution is what I’m interested in.
Distributing password cracking among multiple machines is throwing linear resources to an exponential problem!

Posts

Loopback & crypt: a filesystem, within an encrypted partition, within a file