nginx and the too many open files limit

So, nginx is fast, nginx is light, nginx is great but… nginx can be nasty too, with undocumented unexpected behaviors. What happened today? We were putting in production a new reverse proxy based on nginx 1.4.1 and one of the obvious thing you do when putting it in production is to raise the nofile limit from the standard 1024 (at least on Debian). So, you expect that nginx will inherit those pam_limit little numbers but… no!! If you check out /proc/$nginx_worker_PID/limits you will see that nofile is still 1024. So, obviously someone is cheating. Looking at the nginx documentation about nofile you will see that the interesting option worker_rlimit_nofile has no default value, so one would think that this value would be inherited from the system conf but, as you have already figured out, it’s not that way. You have to explicitly set, for example:

worker_rlimit_nofile = 100000;

in your main part of nginx.conf to have it as you wish. BTW this overrides limits.conf even if you put a lower value in limits.conf, so if you’re using nginx >= 1.4 just tune this configuration option to solve the “too many open files” problem.

Column names with trailing spaces in tables and views

Just found out a bizarre feature of MySQL: you simply cannot create a table with a trailing space in a column name. Something like this, doesn’t work:

create table a (`a ` int);
ERROR 1166 (42000): Incorrect column name 'a '


There’s a thread in MySQL’s lists explaining it

Ok, it seems there’s a technical limitation but this solution seems like a lazy workaround to me.
The problem is with CREATE VIEW. In recent MySQL builds you cannot either create a view with a trailing space in a column name. But recent means, at least, greater than 5.0.51a, cause this version DOES let you create a view with a trailing space.

# mysql --version
mysql Ver 14.12 Distrib 5.0.51a, for debian-linux-gnu (x86_64) using readline 5.2

mysql> create view b as select a `a ` from a;
Query OK, 0 rows affected (0.00 sec)


There’s a big problem here: 5.0.51a is the default and supported version in Debian Lenny! So, if you have Lenny as a master of a more recent MySQL installation, be careful cause a CREATE VIEW with a trailing space in a column name will break your replication. Grrrr. MySQL, I hate you, really.

HUDlite-server is crap

HUDlite-server, part of HULite which is a tool to monitor and manage queues and agents in a Asterisk based callcenter, is pure crap. It’s a perl-thing but closed source 8with perl2exe) which simply doesn’t work if not installed by default in a Trixbox environment. There is only an RPM for CentOS available to download (after you manage to pass the broken link they have on their website), which doesn’t even correct RPMs dependencies in it!! It requires for example perl-POE but it doesn’t complain at install time, it only throws you an error the first time you try to execute it manually (because the init.d script has a faboulos 2> /dev/null…. are you idiot or what?). Then, after you have fullfilled all the dependency, it silently dies after a few seconds, and all you can find out from a strace is

rt_sigaction(SIGHUP, {SIG_DFL}, NULL, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL}, NULL, 8) = 0
rt_sigaction(SIGQUIT, {SIG_DFL}, NULL, 8) = 0
stat64("/usr/lib/libtb.so", 0x813b428) = -1 ENOENT (No such file or directory)
open("/usr/lib/libtb.so", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
stat64("/var/adm/bin/astwatch.pl", 0x813b428) = -1 ENOENT (No such file or directory)
pipe([3, 4]) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x5f8708) = 3555
close(4) = 0
rt_sigaction(SIGINT, {SIG_IGN}, {SIG_DFL}, 8) = 0
rt_sigaction(SIGQUIT, {SIG_IGN}, {SIG_DFL}, 8) = 0
waitpid(3555, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0) = 3555
--- SIGCHLD (Child exited) @ 0 (0) ---
rt_sigaction(SIGINT, {SIG_DFL}, NULL, 8) = 0
rt_sigaction(SIGQUIT, {SIG_DFL}, NULL, 8) = 0
read(3, "", 4) = 0
close(3) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x5f8708) = 3557
unlink("/tmp/p2xtmp-3549/Util.so") = 0


where the only useful piece of information is this libtb.so and astwatch.pl missing, but I have really no clue (after 1+ hour spent googling) where to find them and if they will solve my problem. So…

rpm -e hudlite-server


and I don’t want to see anymore this crap in my life. Fonality: you sucks, you have no clue about what this open-source thing is and how to have a growing community of happy users. All you want to do is suck milk from your customer with minimal effort.

WTF

WTF… today (sunday, just to let you know… ) our ISP left us again, twice in a week, with a power blackout (and we are in colocation, so wonder what happened). This is plain ridicolous and I hope we’re going to change (once again) provider. The other sysadmin writing on this blog spent 4 hours of his life just for this, and it was sunday! 

WTF…