After a lengthier post mortem it turns out it wasn't mod_userdir after all. They just installed an app which could exploit mod_userdir, but clearly another initial exploit got them to that stage.

Turns out it's some insecure PHP script - probably PHPMyAdmin - which let the hacker crew pull remote files direct to /tmp. As Ensim uses chroot jails, it probably isn't any of the regular sites' apps, so PHPMyAdmin is the only suspect. Anyway, the files they wrote included:

• w00t - I believe this can be used to exploit a mod_userdir bug
• '...' - a folder contain tons of goodies, trojan versions of real shell tools, etc.
• dc.txt - a clever Perl script which opens a socket to a remote machine, execs /bin/sh and points stdin/stdout/stderr to the socket.

In '...' I found a ton of files, these being the most interesting:

• poster.pl - posts details of the exploit to www.zone-h.org and www.delta5.com.br (clearly its a numbers game)
• upd.pl - a Perl script designed to do a UDP flood on a specified target
• ssh.php and ssh2.php - invalid PHP files seemingly designed to run commands on a Windows server??
• mass.txt - Perl app called 'IndexOver 0.2' which looks for all index.html files and overwrites them with a "we owned you" type message .. this appears to have only worked on ONE index.html file on the whole machine
• krad - some sort of 'crash this machine' app?

Inside '...' was another folder called shv5, which is a rootkit. It appears this was not run, however. I went through its setup routine and it's really thorough. It updates MD5 hashes, changes libraries, tricks tripwire, and can deal with Slackware, Debian, SuSE, RedHat..

After checking out www.zone-h.org (who I think are morons for letting hackers compete in this silly game of 'who can hack the most servers'), it seems Brazil is giving and taking tons of hacks right now, some kinda hacker turf war, I guess. So, South America is banned. I'll unban them in a few months when I've upgraded my server.