Choose color scheme

Category Archives: unix / linux

  • process file descriptor count

    I’ve recently had to deal with a process leaking file descriptors. The following command came in handy as a quick way to count how many file descriptors a process is using.

    Let’s say that we want to count them for the process(es) called firefox:

    ps -ae | grep firefox | perl -lane 'print $F[0]' | while read filename; do ls /proc/$filename/fd; done | wc -w

    TADAA!

  • VM stuck on a task

    I’ve recently lost control of a VM that was stuck at 95% of a task. I waited and tried to regain control of the VM, nothing helped. This is how I got around it:

    SSH into the ESX on which the VM is instantiated
    cat /proc/vmware/vm/*/names | grep <vm_name>
    note the vmid
    /proc/vmware/vm/<vmid>/cpu/status
    note the group vmid
    /usr/lib/vmware/bin/vmkload_app -k 9 <group_vmid>
    

    That’s it!

  • ssh tunnel to circumvent a firewall

    My work place like many others has a pretty restrictive firewall that doesn’t let me ssh into my own machine. To get in the network, one has to use VPN which means that a furious battle will rage getting this to work in linux; but above all you won’t get to ssh from your phone.

    So if you have a home server, run the following command on your work machine and it will create a tunnel from your work machine to your home server:

    ssh -f -N -R 1337:localhost:22 root@home_server

    Now login to your home server and when you

    ssh localhost -p 1337

    You will in fact be sshing to your work machine via magic fairies & such.

    It adds a level of indirection which sucks major balls, so you can copy some keys and get all that automated but I don’t want to go into these details. Figure it out.

    You can go away now.

  • Count how many file descriptors are being used by every process of a certain name

    here’s a neat little command:

    ps -ae | grep <process_name> | perl -lane ‘print $F[0]’ | while read filename; do ls /proc/$filename/fd; done | wc -w

    just replace process_name by httpd for example and it’ll tell you how many file descriptions are in use by all the processes with http in them.

  • Postfix & Spamassassin integration allowing for custom processing

    This assumes that you have postfix installed and running as your SMTP server

    First, make sure that you’re root

    whoami

    I probably shouldn’t have to explain that if you’re reading this but just in case; if that last command returned something else than ‘root’ issue the following command

    sudo su

    and enter your password

    step 1: Let’s install the packages we’re gonna need

    apt-get update
    apt-get install spamassassin spamc

    step 2: Now we configure spamassasin

    cat /etc/default/spamassassin | sed -e 's#ENABLED=0#ENABLED=1#g' > /etc/default/spamassassin
    cat /etc/default/spamassassin | sed -e 's#CRON=0#CRON=1#g' > /etc/default/spamassassin
    cat /etc/spamassassin/local.cf | set -e 's## rewrite_header Subject *****SPAM*****#rewrite_header Subject [*****SPAM*****] > /etc/spamassassin/local.cf

    and we start/restart it

    /etc/init.d/spamassassin restart

    step 3: We create a little script that will take desired action upon spamassassin flagging

    create a user called spamassassin (or whatever you want as long as you keep it consistent)

    useradd -m spamassassin

    then edit the script file /home/spamassassin/spamcheck and throw the following in it

     # variables
     
    SENDMAIL="/usr/sbin/sendmail -i"
     
    EGREP=/bin/egrep
     
    SPAMLIMIT=10
     
    # exit codes from <sysexits.h>
     
    EX_UNAVAILABLE=69
     
    # clean up when done or when aborting.
     
    trap "rm -f /tmp/out.$$" 0 1 2 3 15
     
    # pipe message to spamc
     
    cat | /usr/bin/spamc -u spamd > /tmp/out.$$
     
    # are there more than $SPAMLIMIT stars in X-Spam-Level header? :
     
    if $EGREP -q "^X-Spam-Level: *{$SPAMLIMIT,}" < /tmp/out.$$
     
    then
     
    # option 1: move spam messages to sideline dir so a human can look at them later:
     
    mv /tmp/out.$$ /home/spamassassin/`date +%Y-%m-%d_%R`-$$
     
    # option 2: divert spam message to an alternate e-mail address:
     
    #$SENDMAIL xyz@xxxx.xx < /tmp/out.$$
     
    # option 3: just delete the spam message
     
    # rm -f /tmp/out.$$
     
    # option 4: still relay the email to the recipient with the subject of the email now containing [*****SPAM*****]
     
    # $SENDMAIL "$@" < /tmp/out.$$
     
    else
     
    $SENDMAIL "$@" < /tmp/out.$$
     
    fi
     
    # Postfix returns the exit status of the Postfix sendmail command.
     
    exit $?

    make sure that you

    chown spamassassin:spamassassin /home/spamassassin/spamcheck
    chmod 750 /home/spamassassin/spamcheck

    step 4: Ok, so we got spamassassin going and a little script that will take an email and throw it in /home/spamassassin if it’s spam (if you chose option1) now we just need to tell postfix to pass all messages to that script

    edit /etc/postfix/master.cf and replace

    smtp      inet  n       -       -       -       -       smtpd

    with

    smtp      inet  n       -       n       -       -       smtpd -o content_filter=spamcheck:dummy

    also add the following 2 lines at the bottom of the file (the indentation is important)

    spamcheck   unix  -       n       n       -       10      pipe
    
    flags=Rq user=spamassassin argv=/bin/spamcheck -f ${sender} -- ${recipient}

    We’re almost there, just restart postfix and you’re good to go!

    /etc/init.d/postfix restart

    If you wanna test that out, watch the log while you send emails to your servers

    tail -f /var/log/syslog

    send a clean mail, make sure that it reaches destination, then send something you know will get flagged as spam and make sure it ends up in /home/spamassasin instead of the intended recipient.

    The reason we choose option 1 here is because there’s no point in still relaying a flagged email as it will still clog the recipient’s mailbox. On the other hand we don’t want to just delete it if spamassassin makes a mistake we want to play it safe and keep every emails should something arise, we quarantine the bad ones in /home/spamassassin

    Lastly, as long as you have postfix just feeding the emails to a script like we just did, it’s easy to become fancier and do all kinds of processing to the email, on my server I actually call a php script that throws emails in a DB.

  • Silly Apache warning

    If the following happens to you:

    apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName

    just issue a:

    echo Servername `cat /etc/hostname` >> /etc/apache2/apache2.conf

    Tested on: Ubuntu server 9.04 32b / Ubuntu 12.04 64b

  • recursive name based delete

    Here’s a neat little command that will let you delete specified files/directories recursively and based on their names.

    Let’s do a dry run first to make sure that the command will go through the right files. Run:

    <code class="plain plain">find <directory_to_start_the_recursion_in> -name <file_name</code>>

    Keep in mind that if you’re gonna have asterisks (*) in the <file_name> you need to escape them like so:

    find /var/www -name *.jpg

    make sure that the result only lists the files/directories that you indeed want to obliterate. Then improve that last command by adding:

    find <directory_to_start_the_recursion_in> -name <file_name> -exec rm -rf {} ;

    Since this is a pretty dangerous command even after a dry run, you can use -ok instead of -exec which will prompt you for approval everytime the command it executed.

    find <directory_to_start_the_recursion_in> -name <file_name> -ok rm -rf {} ;

    This is of course not limited to rm :)

  • recursive type based chmod

    Here’s a cool little script that will recursively chmod, giving a permission based on whether it’s dealing with a file or a directory. This is very convenient when you want to add that +x to directories but not files.

    find $1 -type f -exec chmod $2 {} ;
    find $1 -type d -exec chmod $3 {} ;

    Go ahead and edit /usr/bin/chmod_script, copy paste these 2 lines in there, then issue a chmod 755 /usr/bin/chmod_script as root, that’s it!

    Usage syntax is as follows:

    chmod_script <directory_to_start_the_recursion_in> <permissions_for_files> <permissions_for+directories>

    so if I want to use it on /var/www do:

    chmod_script /var/www 644 755

    Enjoy!