Choose color scheme

Category Archives: unix / linux

  • Resilient SSH Tunnel

    # Use netcat to connect to tunnel entrance port. If its exit code is not zero, the tunnel needs to be brought up
    nc -w1 localhost $tunnel_entrance_port > /dev/null
    if [[ $? -ne 0 ]]; then
            echo "creating new tunnel for MySQL"
            /usr/bin/ssh -f -N -L $tunnel_entrance_port:$destination:$destination_port $tunnel_end
            if [[ $? -eq 0 ]]; then
                    echo "  tunnel to $destination through $tunnel_end created successfully"
                    echo "  an error occurred creating a tunnel to $destination through $tunnel_end"
  • Script to reboot a Comtrend AR-5381u Modem when connectivity is lost

    # default username for this model
    export MODEMUSERNAME=admin
    # default password for model
    export MODEMPASSWORD=user12345
    export COOKIE=/tmp/cookie_jar
    export CURL=/usr/bin/curl
    export PING=/bin/ping
    export GREP=/bin/grep
    export CUT=/usr/bin/cut
    $PING -c 5 > /dev/null 2>&1
    if [ $? -ne 0 ]
            rm $COOKIE > /dev/null 2>&1
            export OUTPUT=`$CURL -v -c $COOKIE 'http://$MODEMUSERNAME:$MODEMPASSWORD@' 2>> /tmp/output`
            export SESSIONKEY=`echo "$OUTPUT" | $GREP var | $GREP sessionKey | $CUT -d"=" -f2 | $CUT -d"'" -f2`
            echo "kicking modem with session key $SESSIONKEY"
            export OUTPUT=`$CURL -v -c $COOKIE "http://$MODEMUSERNAME:$MODEMPASSWORD@$SESSIONKEY" 2>> /tmp/output`
  • Gnuplot one-liner from Hell

    Here’s a convenient one liner to chronologically plot data on the command line


    Screen Shot 2014-04-02 at 11.18.44 AM


    export width=`stty size | cut -d " " -f2`; export height=`stty size | cut -d " " -f1`-10; cat /tmp/data | sed "s/ /T/" | gnuplot -e "set terminal dumb $width $height; set autoscale; set xdata time; set timefmt \"%Y-%m-%dT%H:%M:%S\"; set xlabel \"time\"; set ylabel \"counter\"; plot '-' using 1:2 with lines"


    /tmp/data contains the following:

    2000-01-01 00:00:00 1
    2000-01-01 01:00:00 2
    2000-01-01 02:00:00 3
    2000-01-01 03:00:00 2
    2000-01-01 04:00:00 3
    2000-01-01 05:00:00 4
    2000-01-01 06:00:00 5
    2000-01-01 07:00:00 4
    2000-01-01 08:00:00 3
    2000-01-01 09:00:00 4
    2000-01-01 10:00:00 4
    2000-01-01 11:00:00 5
    2000-01-01 12:00:00 4
    2000-01-01 13:00:00 4
    2000-01-01 14:00:00 3
    2000-01-01 15:00:00 2
    2000-01-01 16:00:00 3
    2000-01-01 17:00:00 4
    2000-01-01 18:00:00 4
    2000-01-01 19:00:00 5
    2000-01-01 20:00:00 6
    2000-01-01 21:00:00 6
    2000-01-01 22:00:00 7
    2000-01-01 23:00:00 8
    2000-01-02 00:00:00 7
    2000-01-02 01:00:00 7
    2000-01-02 02:00:00 6
    2000-01-02 03:00:00 8
    2000-01-02 04:00:00 9
    2000-01-02 05:00:00 9
    2000-01-02 06:00:00 9
    2000-01-02 07:00:00 8
    2000-01-02 08:00:00 7
    2000-01-02 09:00:00 5
    2000-01-02 10:00:00 4
    2000-01-02 11:00:00 4
    2000-01-02 12:00:00 4
    2000-01-02 13:00:00 3
    2000-01-02 14:00:00 2
    2000-01-02 15:00:00 2
    2000-01-02 16:00:00 1
  • FreeBSD manual multipath script

    I recently ran into an issue installing FreeBSD on a system that already had some disks & zpools. Because the disks were partitioned previously, automatic multipath was not an option as the last sector of all hard drives isn’t available to store an ID. The remaining option is to do manual multipath, and it needs to be done every time the system boots.

    Here’s an rc script that will run early in the sequence and create a multipath “link” between drives based on their serial number.


    # PROVIDES: manual_multipath
    # REQUIRE: sysctl
    # BEFORE: hostid
    . /etc/rc.subr
            echo "> manual_multipath script started"
            echo "> linking drives with the same serial number with gmultipath"
            devices=`/usr/bin/find /dev -maxdepth 1 -regex '.*da[0-9]*' | /usr/bin/cut -d '/' -f 3`
            for device in $devices
                    echo $device
                    serial=`camcontrol inquiry $device -S`
                    substring=`echo "$serials" | /usr/bin/sed -n "s/|$serial|.*//p" | /usr/bin/wc -c`
                    if [ $substring -eq 0 ]
                            for newdevice in $devices
                                    newserial=`camcontrol inquiry $newdevice -S`
                                    if [ "$device" != "$newdevice" -a "$serial" == "$newserial" ]
                                            echo "  same as $newdevice!"
                                            counter=`expr $counter + 1`
                                            arg2=$arg2" $newdevice"
                            if [ $found_multi -eq 1 ]
                                    gmultipath create $arg1 $arg2
            echo "> manual_multipath script finished, found $counter matches"
    load_rc_config $name
    run_rc_command "$1"

    Don’t forget to “chmod 555 /etc/rc.d/manual_multipath”.

    Lastly, when importing a zpool from the drives you just multipathed, make sure to specify where to look for devices or you might end up importing a mix of multipath and regular devices. Make sure to “zpool import -d /dev/multipath”.

    I’m delving pretty deep into FreeBSD, time to grow an epic beard.

  • ZFS send/receive accross different transport mechanisms

    Sending ZFS snapshots across the wires can be done via multiple mechanisms. Here are examples of how you can go about it and what the strengths and weaknesses are for each approach.


    strengths: encryption / 1 command on the sender

    weaknesses: slowest


    zfs send tank/volume@snapshot | ssh zfs receive tank/new_volume


    strengths: pretty fast

    weaknesses: no encryption / 2 commands on each side that need to happen in sync


    on the receiver

    netcat -w 30 -l -p 1337 | zfs receive tank/new_volume

    on the sender

    zfs send tank/volume@snapshot | nc 1337

    (make sure that port 1337 is open)


    strengths: fastest

    weaknesses: no encryption / 2 commands on each side that need to happen in sync


    on the receiver

    mbuffer -s 128k -m 1G-I 1337 | zfs receive tank/new_volume

    on the sender

    zfs send tank/volume@snapshot | mbuffer -s 128k -m 1G -O

    (make sure that port 1337 is open)

    SSH + Mbuffer

    strengths: 1 command / encryption

    weaknesses: seems CPU bound by SSH encryption, may be a viable option in the future?


    zfs send tank/volume@snapshot | mbuffer -q -v 0 -s 128k -m 1G | ssh 'mbuffer -s 128k -m 1G | zfs receive tank/new_volume'

    Finally, here is a pretty graph of the relative time each approach takes:

    SSH + MBuffer would seem like the best of both worlds (speed & encryption), unfortunately it seems as though CPU becomes a bottleneck when doing SSH encryption.

  • MDNS/Bonjour printer discovery script

    Here’s a script I wrote whose purpose is to discover the printers that are currently being advertised by Bonjour on the network. The reason I wrote it was for a Nagios check that would in term verify that our printers were present. Writing it took me through the meanders of MDNS in Python & on Linux with multiple vlans. Let’s just say non-trivial.



    Sample output

  • The impairing lack of light pollution

    When we lived in the city, ambient light pollution was such that I could set my CCTV cams to a certain brightness/contrast and the limited auto adjustments they did were enough to cope with day & night. In the middle of the forest, the night gets full on #000000 dark. The poor cams can’t adjust and I need to pick whether I want to record at night and get white frames during the day, or at daytime and get black frames during the night.

    I wrote the following script which computes the average brightness of a cam’s current frame and issues more drastic adjustments if needed. It is obviously tailored for my FI8918Ws but the same idea can be used for others.

    $img = @imagecreatefromjpeg( '<username>&pwd=<password>' ) ;
    if( $img===false ) {
        die( "Unable to open image" ) ;
    $w = imagesx( $img ) ;
    $h = imagesy( $img ) ;
    $total_r = 0 ;
    $total_g = 0 ;
    $total_b = 0 ;
    for( $i=0 ; $i<$w ; $i++ ) {
        for( $j=0 ; $j<$h ; $j++ ) {
            $rgb = imagecolorat( $img, $i, $j ) ;
            $total_r += ($rgb >> 16) & 0xFF;
            $total_g += ($rgb >> 8) & 0xFF;
            $total_b += $rgb & 0xFF;
    $average_brightness = round( ( $total_r / ($w*$h) + $total_g / ($w*$h) + $total_b / ($w*$h) ) / 3 ) ;
    echo $average_brightness, "n" ;
    if( $average_brightness<30 ) {
        echo "night time!n" ;
        echo "moden" ;
        $result = file_get_contents( '<username>&pwd=<password>' ) ;
        sleep( 10 ) ;
        echo "contrastn" ;
        $result = file_get_contents( '<username>&pwd=<password>' ) ;
        sleep( 10 ) ;
        echo "brightnessn" ;
        $result = file_get_contents( '<username>&pwd=<password>' ) ;
    } else if( $average_brightness>170 ) {
        echo "day time!n" ;
        echo "moden" ;
        $result = file_get_contents( '<username>&pwd=<password>' ) ;
        sleep( 10 ) ;
        echo "contrastn" ;
        $result = file_get_contents( '<username>&pwd=<password>' ) ;
        sleep( 10 ) ;
        echo "brightnessn" ;
        $result = file_get_contents( '<username>&pwd=password>' ) ;
  • Verizon’s 4620L, a great device for the technically inclined

    My family recently moved to a fairly remote area, the question of internet access has been a major one for the couple of months leading to the move. Besides satellite & dial-up, our only option was Verizon’s MiFi (3G or 4g if you’re lucky) in the form of a hotspot device: the 4620L.

    I was afraid that the 4620L would try to be too smart and not let you tinker with it very much, very few decent reviews are available online and the official documentation is seriously lacking. Fortunately this couldn’t be further from the truth, it is a great little device that performs well and lets you turn all its knobs.

    When using “USB tethered mode” I was afraid I’d need specific drivers and a software suite running but lo and behold, it actually just pretends to be an ethernet device over USB. Absolutely perfect to put a Linux router in front of it!

    One thing that did not get properly QA’d is the “Enable DCHP Server” checkbox which simply doesn’t work. But guess what, I want to do my own routing and I’d like to avoid NATing from the 4620L to the Linux router. One way to circumvent this is to use the “Config File Download” and “Config File Upload” options which are meant as a way to backup & restore configuration but since the file is all intuitively labeled XML it’s easy to disable the DHCP server from there.

    While you’re in there, you can also override the maximum number of “Available Wi-fi Connections” (5 when using 3G). They probably have this restriction so regular Joe user doesn’t hook a gazillion device and complain about speed over 3G. Reaching this limit is very easy nowadays.

    A new mission

    Verizon’s plan is pretty pricy and very metered… All we get is 5GB per month, each additional 1GB will cost us $10. Ouch… I need to configure the network to consume as few bytes as possible. Netflix is out, AdBlock is in, automatic updates of various types are out. Above all, my home server will now be doing some serious routing, the goal of which is to allow devices to be on the home intranet while minimizing their use of the internet.

    No inbound connection

    That’s right, the IP you get from Verizon is in the private range (RFC 1918), this means they are doing some NATing of their own. You can forward ports all you want on your 4620L this will have no effect. Your only option is some cumbersome hole punching.

    We’ll be talking routing in a next post, I would have liked to find this information about the device & Verizon’s setup so I wanted to put it out there sooner rather than later.

  • Change default home Unity lens

    Because we don’t necessarily want the home lens to be the default one in Unity, and unlike other lenses it is hardcoded left & right. Here’s a little trick that will let you pick a different lens as the default for when you click on Dash.

    edit the file: /usr/share/unity-2d/shell/dash/Dash.qml

    replace line 79 “onDashActivateHome: activateHome()” by “onDashActivateHome: activateLens(X)” where X is the index of the lens you want to load (count from left to right starting from 0).

    You’ll want to restart Unity for this to take effect.


  • Loopback & crypt: a filesystem, within an encrypted partition, within a file

    So here we are, 2012 and physical media are going away really fast. We won’t even talk about CDs which have been relegated to the role of plastic dust collectors; hard drives even are being abstracted by a myriad of cloud based solutions. Their purpose is shifting towards a container for the OS and nothing else. Filesystems & their hierarchies become hidden in a bid to remove any need to organize files, rather, you are supposed to throw it all up in the cloud and search on metadata.

    While moving away from physical media is convenient and inevitable, I like the hierarchical organization that directories provide. What’s more intuitive than a labeled container with stuff in it?

    How can we detach our hard drives from their physical shells, move them around in an omnipresent cloud and keep them secure?

    By creating a file, attaching it to loopback & creating an encrypted partition in it!

    Here’s how to do it
    • Create a file that will be your soft hard drive with:
    dd if=/dev/zero of=/tmp/ffs bs=1024 count=524288

    This will create a 512MB file (524288/1024).

    • Make sure that the loopback device #0 is free:
    losetup /dev/loop0

    You should see something telling you that there is “No such device or address”.

    • Attach the soft hard drive to the loopback device:
    sudo losetup /dev/loop0 /tmp/ffs
    • And then make sure it was indeed attached by re-running:
    losetup /dev/loop0
    • Create an encrypted partition on your attached soft hard drive:
    sudo cryptsetup --verify-passphrase luksFormat /dev/loop0 -c aes -s 256 -h sha256
    • Open your encrypted partition:
    sudo cryptsetup luksOpen /dev/loop0 ffs
    • Create a filesystem in it:
    sudo mkfs.ext3 -m 1 /dev/mapper/ffs
    • And mount it like a regular disk:
    sudo mount /dev/mapper/ffs /mnt
    • When you are done using your encrypted soft hard drive you will want to umount it:
    sudo umount /mnt
    • Close it:
    sudo cryptsetup luksClose ffs
    • Detach it from loopback:
    losetup -d /dev/loop0

    These steps can be automated of course. As a quick reminder, using the drive goes “loopback attach -> crypt open -> mount” and when you’re done it’s “umount -> crypt close -> loopback detach”.

    That’s it! media-less & secure storage.

    Tested on: Ubuntu 12.04 64b