Xen, XFS and the barrier error

Probably nobody is going to hit this but since I’ve lost half a morning puzzling my mind about this error, I think it would be useful to blog about it.
We have some old Xen machine running Debian Lenny with Xen 3.2 and while upgrading a VM using a XFS root partition to Debian Wheezy (it was a Squeeze), I’ve got this nice error:

[ 5.024330] blkfront: xvda1: barrier or flush: disabled
[ 5.024338] end_request: I/O error, dev xvda1, sector 4196916
[ 5.024343] end_request: I/O error, dev xvda1, sector 4196916
[ 5.024360] XFS (xvda1): metadata I/O error: block 0x400a34 (“xlog_iodone”) error 5 buf count 3072
[ 5.024369] XFS (xvda1): xfs_do_force_shutdown(0x2) called from line 1007 of file /build/linux-s5x2oE/linux-3.2.46/fs/xfs/xfs_log.c. Return address = 0xffffffffa009fed5
[ 5.024394] XFS (xvda1): Log I/O Error Detected. Shutting down filesystem
[ 5.024401] XFS (xvda1): Please umount the filesystem and rectify the problem(s)
[ 5.024411] XFS (xvda1): xfs_log_force: error 5 returned.
[ 5.024419] XFS (xvda1): xfs_do_force_shutdown(0x1) called from line 1033 of file /build/linux-s5x2oE/linux-3.2.46/fs/xfs/xfs_buf.c. Return address = 0xffffffffa005f8a7

this strange error simply means that the underlying device doesn’t provide barrier support! The simple, quick fix (once you know it) is to specify in the fstab nobarrier option. I think that I found this problem because in Linux 3.2 the barrier option is enabled by default, while it was off in previous kernel versions.

How to remove a port from a port-channel in a Dell PowerConnect switch

This is a “note to self” type post, and basically because Google seems unable to find a direct answer to this simple question.
So it’s simple as this


# configure
(config)# interface ethernet NN
(config-if)# no channel-group


et voilà, the ethernet port doesn’t belong anymore to the port channel. It should work with PowerConnect 5324, PowerConnect 5424, PowerConnect 5448, PowerConnect 5548 etc.

/etc/hosts and the thousand-characters-long line

This is a self-note in the case I encounter another strange behaviour like this. We were experiencing a strange problem with MySQL and DNS. I was trying to do this:

$ mysql -h server.mysql
Unknown MySQL server host 'server.mysql' (-1)

but both dig and a normal ping (which in turns uses libc and nsswitch to do the name resolving) were working:

$ dig +short server.mysql
192.168.10.1

$ ping server.mysql
PING server.mysql (192.168.10.1) 56(84) bytes of data.
64 bytes from server.mysql (192.168.10.1): icmp_req=1 ttl=64 time=0.399 ms


and obviously connecting using the MySQL client and the IP address worked. So, what was happening? The smarter amongst you maybe already know the problem: a very very large line in /etc/hosts was driving the mysql client crazy (but not ping). Removing the “files” database fron the hosts entry in /etc/nsswitch.conf showed where the problem lied, and fixing the bad-ass line fixed the problem

Apache2: seg fault or similar nasty error detected in the parent process

If you happen to see a message like this

seg fault or similar nasty error detected in the parent process

when reloading Apache2, and if you’re using PHP5 through mod_php5, then it may be related to having an extension loaded via php.ini and not really present on the system. It was my case with a redis extension (redis.so) and I banged my head a day before finding it.

Customize the console prompt in VMWare ESXi 4.0

The default console prompt of VMWare ESXi 4.0 really sucks, it’s black&white, it gives no info about the host you are connected to and if you have more than one host this is becomes quickly an headache.
So, how do you change it? Pretty easy:


echo 'export PS1="\[33[01;32m\]\u@\h\[33[00m\]:\[33[01;34m\]\w\[33[00m\]\$ "' > $HOME/.profile

then exit from the shell (ssh or local) and enter again and you will have a pretty nice colored console prompt :)

EDIT: ok, it seems that I cannot post “backslash zero” with WordPress. so please put before any “33” in this string “backslash zero” (the symbol and the number, not the two words). Thanks to Daniel for pointing this out. If you know a way to solve this, please share it :)

TCP sockets in your shell

Usually I always used netcat to send TCP data but there’s another way, builtin in bash. You can send the string “Hello world” to 192.168.1.1 on port 15123 using:

echo "Hello world" > /dev/tcp/192.168.1.1/15123

Simple, quick and effective :)

Note: thanks to Waldner for pointing out that it’s built-in in bash and there’s no need for an “mknode”

A handful of bash tips – part 1

During last weeks I’ve been doing lot of scripting (a user management system for our internal infrastructure which is formed by lots of services) and it ended being something like 3000 lines of bash code. Not too much but probably this is my greatest experience til now with bash :)

So, I’ve learnt a couple of things with this project and I’d like to share some tips and lessons learnt during the process. I’m not a bash guru, and if you find something could be improved, feel free to leave a comment.

Stand on the giant’s shoulders

You can find lot of docs on bash, like the Advanced Bash Scripting Guide, but these are better in my opinion

• http://mywiki.wooledge.org/BashGuide/
• http://mywiki.wooledge.org/BashPitfalls/

and the #bash IRC channel on irc.freenode.net. Just lurking the conversations will teach you lot of things

Use the latest and greatest bash version

This means right now to use Bash4. I mean, if you are using a shell to script things, use the latest version and take advantage of all its features! Really, I don’t care about portability, I script for my systems where I have full control. I’ve upgraded a Debian box from Lenny to Squeeze just to get bash4, go figure :) Obviously if you cannot update easily or you need to be portable, don’t fall in the bashisms trap, try to be as POSIX as possible and discard my tip(s).

Quote everything. I mean, really everything

If you are used to the simple $VARIABLE form, drop it right now and use always “${VARIABLE}”. It’s cleaner, it’s safer, it supports white spaces in the variable content etc.

Only the first quote matters

I’ve seen very often (and I was using this too) escaped quote chars when creating queries or strings to be passed somewhere. For example

QUERY="SELECT * FROM foo WHERE name=\"${VAR}\";"

because you fear that ${VAR} won’t be expanded if single quoted. But what really matters in this case is the first double quote. bash will interpret every other single quote before the closing double quote as a normal char, and pass it to the next hop. So this is perfectly right:

QUERY="SELECT * FROM foo WHERE name='${VAR}';"

${VAR} will be expanded as expected. This is quite useful if calling another script with parameters via ssh, and leave a cleaner syntax (I hate escaping chars)

ssh user@host "/path/to/script 'foobar goes first' 'second parameter'"

Use shift when receiving parameters

Probably you are already doing this, anyway it’s a lesson learnt in these days. If you’re are passing parameters to functions/other scripts instead of

PARAM1="${1}"
PARAM2="${2}"
# etc

use the power of shift. If you are going to change your mind about those parameters, you won’t need to rename everything.

PARAM1="${1}"
shift 1
PARAM2="${1}"
shift 1
# etc

Use input redirection instead of pipes when possible

A classic example

cat /path/to/mylst|while read foo
do
# do somtething with foo
done

should be

while read foo
# do something with foo
done < /path/to/mylist

why? because input redirection it is meant to do that!

Convert pwdLastSet to a human readable date

Here it is a simple (and a bit hacky, I know) one-liner for bash shell (even under Windows if you are using Cygwin) to convert the cryptic pwdLastSet timestamp of Active Directory (which represent when a user has changed the last time his/her AD password)


D=128457325992343750; date -d "01/01/1601 UTC $(let D=D/10000000; echo $D) seconds"


where the very large number after the first D= it’s your pwdLastSet value. This strange timestamp it’s a 1/100 of a nanosecond (so, it’s 1/10^7 seconds) and the ticks are counted from January 1st, 1601. Don’t ask me why, probably they didn’t like the Epoch time :)

Disable directory listing in Apache with Debian

If you find one of your servers with the ugly directory listing enabled, there’s a quick way to disable it in Debian

# echo autoindex | a2dismod
# /etc/init.d/apache2 restart

For other Apache installations in other distro, you can simple find the Autoindex option in your config file and delete it manually, then restart Apache

EDIT: a cleaner and more elegant way to achieve the same is, as the comments section says

# a2dismod autoindex

thanks :)


Note to self: accessing LVM volumes from live CD

If you have a Linux machine with LVM volumes on it, and you need to boot it from a live CD (for example SysRescueCD) and you wonder where your volumes are, just issue (in an LVM enabled system, of course)


# vgchange -a y


and your volumes will be available under /dev/$VOLUME-GROUP-NAME or under /dev/mapper