Server not hacked *sighs in relief* :)

Timendus · Post by **Timendus** » Mon 16 Jan, 2006 10:14 am

I woke up this morning to the alarming sound of my server rebooting. It never reboots by itself, it's a Linux machine, not some crappy Windows server. I hoped it would be a power failure or something, those are quite common in my house, but my clock radio wasn't reset.

So I got worried, got out of bed and opened my laptop up. Browsing to random websites that I host got me even more worried as it only returned 404's. Forcing myself to stay calm I turned on the monitor of my server, and logged in as root. The moment I pressed Enter the screen went black and my server started rebooting itself again

"F@*k, a root kit" went through my head, and obviously I pulled the LAN plug at this point, and booted my server again. This time I could log in without a reset, and I started to bring up some services. Samba worked properly, and I was able to make a backup of a few websites, including the API site. One website that I've been working on (and that I get payed for) is due for release this week. I managed to rescue all the scripts, but I'm not sure what the status of my SQL database is...

So now I'm staring at my server, restarting Apache only got me more 404's, so there's clearly something wrong with my configuration, and I really hope that it was a coincidence that the machine rebooted at the exact moment I entered my root password

I'll ask some of my flatmates if they've seen any power failures, but I fear for the worst.

If any of you have advice for me on how to check for root kits/hacking activity, how to get rid of them, or have other ideas of what could be going on, please let me know!

Timendus · Post by **Timendus** » Mon 16 Jan, 2006 10:44 am

Ah, thank God one of my flatmates confirmed a power failure... that explains a few things, but it's still quite strange that Apache isn't working properly...

Post by **benryves** » Mon 16 Jan, 2006 11:33 am

Timendus wrote:...but it's still quite strange that Apache isn't working properly...

Why is that surprising in any way?

I've never used the Linux version, to be honest, but the Windows version has a habit of doing weird and not-very-wonderful things, especially when you throw PHP into the bargain

Have you tried running a disk checking utility? Maybe the config files were "broken" when the machine suddenly lost power.

Timendus · Post by **Timendus** » Mon 16 Jan, 2006 11:53 am

benryves wrote:Have you tried running a disk checking utility? Maybe the config files were "broken" when the machine suddenly lost power.

Not yet. Can you advise any?

And Apache ran perfectly for almost a full year untill this morning...

Post by **benryves** » Mon 16 Jan, 2006 12:08 pm

Timendus wrote:
benryves wrote:Have you tried running a disk checking utility? Maybe the config files were "broken" when the machine suddenly lost power.
Not yet. Can you advise any?

Only chkdsk.exe - I've only ever dealt with Windows. I don't know (doubt) it'll run off a WinME/DOS boot floppy, though. Maybe the Windows recovery console can do something, but I'm sure there are Linux equivalents.

CoBB · Post by **CoBB** » Mon 16 Jan, 2006 12:11 pm

He's most probably using some journaling filesystem (i. e. anything but ext2) that doesn't require such utilities.

Timendus · Post by **Timendus** » Mon 16 Jan, 2006 12:35 pm

The drive with software and config files is xfs, the drive for data storage and hosted websites is reiserfs.

Anyway, I got rid of an error, and I can get Apache to host a file in my documentroot, though it sends out the wrong headers so it gets shown as txt instead of interpreted as html by my browser...

Also, I had put something in my config that would allow different users to put files in their ~/website directory that would automatically get hosted, but I can't seem to find those lines in my config, nor does it work...

In other words, I have to read up on Apache configuration, because I totally forgot everything about it. It's been a year since I last looked at them, and I have no idea which file exactely has been corrupted, nor in what way...

Post by **benryves** » Mon 16 Jan, 2006 12:40 pm

It should be httpd.conf, though where that is on your system I'm not sure

(Apache should come with httpd.default.conf with a basic set up as well).
The user directory config is under the "UserDir" section.
Have you checked for any rogue/missing .htaccess files?

Timendus · Post by **Timendus** » Mon 16 Jan, 2006 12:51 pm

Yes, I had found that file, and the UserDir section, but Apache doesn't seem to listen to what I put there... It's a bit of a mess with different config files with different names all referencing and including each other, or sometimes including entire directories of config files

And nope there aren't any .htaccess files in the directories that I'm testing with...

Andy_J · Post by **Andy_J** » Mon 16 Jan, 2006 3:02 pm

Here's a thought. Does the box automatically update Apache, or have you manually updated it, since the last time it rebooted? I'm thinking an update snuck in there that changed where it looks for config files. Gentoo did that a couple months ago; without knowing what distro you have I couldn't tell you.

As for disk checking... `telinit S` to drop to single user mode, then (if it didn't automatically do it) `mount -o remount,ro` each mount point to mount read-only, followed by a fsck on each. `telinit 3` should take you back to normal operation (though you can `telinit 6` to reboot and make sure everything goes back to normal

).

Timendus · Post by **Timendus** » Mon 16 Jan, 2006 5:09 pm

Yes, it's probably something like that. I do use Gentoo, but that update that caused some upgrade problems was last September. You'd think that I must have rebooted my server at least once since then

But maybe my uptime is better than I thought...

Anyway, I've got parts of it working again, still working on a few aspects (it f@*ks up the headers; html is sent out as plaintext, so firefox displays the source instead of rendering the page, images are forced downloads instead of images, and it doesn't parse scripts yet... just great

).

Edit by tr1p1ea - Language please ...

lloydkirk1989 · Post by **lloydkirk1989** » Mon 16 Jan, 2006 5:48 pm

Maybe a sytem restore would help...?

Timendus · Post by **Timendus** » Mon 16 Jan, 2006 10:23 pm

I don't do system wide backups

Timendus wrote:Edit by tr1p1ea - Language please ...

Why can't we say fu[slaps himself]?

I mean; it's a very natural process that happens all around the globe every day...

Timendus · Post by **Timendus** » Tue 17 Jan, 2006 1:17 am

Hooray! I more or less rewrote my configuration from scratch, moved every Apache related file on my harddrive, and got an incredible headache, but I managed to get Apache 2.0 to run again, with all the neccessary modules. There's only one directory with scripts that doesn't want to allow execution for unknown reasons

Anyway, the API site is back up

MaxCoderz

Server not hacked sighs in relief :)

Server not hacked sighs in relief :)