Any experts here on intrusion postmortem?
Kendric Beachey
a very good one, I think, or at least whatever he exploited wasn't as
dangerous as it could have been.
I do not think he got root...he just found a testing account that had
a password that was too easy to crack. He didn't (or couldn't) make
much of an attempt to cover his tracks, as the tools I used to find
and disable his stuff were top, ps, ls, crontab, kill and
killall...usually the more dangerous crackers override those, from
what I understand.
My guess on how he got in: I have AT&T U-Verse, including their
residential gateway. I set its firewall up to let port 80 and port 22
point to my linux box, so I could ssh home from work, and so my wife
could do some html/css experiments on it (she hasn't done that yet
though). Sometimes the gateway seems to get a bur under its saddle
and says "there is a router behind this router, do you want me to
resolve the problem?" If you say yes, it puts one machine into
"DMZplus" mode, which as far as I can tell sends ALL ports to that
machine. Nice. My linux machine was DMZplus-ified when I went in to
shut off ports 80 and 22...no telling how long it had been that way.
So that doesn't narrow down what he could have exploited, but maybe
points to how the doors all got unlocked.
As near as I can tell the main activity he was doing was portscanning,
but what he might have done with the results, I don't know. After
killing his account I shut the machine off. We had been having
internet trouble across the board since the thunderstorm this weekend,
and I'm wondering whether maybe this cracker had something to do with
the trouble. It's still not 100% great though and we have a tech
coming to our house this evening.
I'm thinking about burning the test account's home dir to a CD for
inspection. I haven't deleted any of his files, only his crontab
entry that spawned a new job every minute if it got killed. Would
anyone on the list want a look at the files?
Beyond that...to get the machine back to a usable state...does anyone
have any suggestions? It seems like it would be foolish to assume
just killing that account would take care of the problem for good, but
would that be sufficient to hobble along for a week or so until I can
cp my own home dir elsewhere and reinstall? Or would it be safer to
keep the machine off the network until I can get that done?
Kendric Beachey
Jeffrey Watts
Don'ts:
1) Don't use a firewall that does stupid stuff like that. Buy a $30 Linksys.
2) Use the system in a meaningful way until it's been reinstalled.
Dos:
1) If you need to remotely SSH into the box, use the AllowUsers feature of SSH and TCPwrappers.
2) Reinstall any other systems on your network if they aren't current on patches or share common passwords with the compromised system.
Jeffrey.
"He that would make his own liberty secure must guard even his enemy from oppression; for if he violates this duty he establishes a precedent that will reach to himself." -- Thomas Paine
Joseph Kearns
Be sure to look for hidden folders: `ls -ap` or some other method. Crackers tend to hide their utilities.
You might also find your log files helpful (unless the cracker was clever enough to modify them).
Personally, I would follow Jeffery's advice and do a backup/reinstall.
--Joe
Adrian Griffis
shouldn't trust that machine again, until you reinstall.
You really won't have a choice about using that Residential Gateway.
To get service, at all, I think you'll have to leave that machine in
place. But don't trust it as your firewall. Put another firewall
between you and it, and just consider everything outside your firewall
own as untrusted.
If you are going to put services out there where they are reachable on
the internet, you need to plan on keeping them patched. You should
either set it up to update automatically (does anyone here have
experience with that) or set up some way to remind you to do it. You
really can't afford to let yourself get significantly behind on
patches for a service that is internet facing.
Adrian
Billy Crook
access by keys. Then you don't have to worry about weak passwords.
On Thu, Jun 18, 2009 at 16:05, Billy Crook<billy...@gmail.com> wrote:
> I also use fail2ban for ssh. When someone tries to guess an account's
...
Adrian Griffis
option on the 'ls' command. There are various kernel modules which
can be included in rootkits and used to hide files and/or processes.
Once these kernel modules are loaded, they can be used to do things
like make an executable file read as though it were a different file
when opened for reading, so that the checksum can appear to match the
distributed version of the executable even though the binary has
change.
Seriously, folks, it is really more trouble than you want to invest to
make sure your system is clean without reinstalling it. Don't try to
clean it up. Just backup the files you really need, and reinstall.
Adrian
Billy Crook
connectivity from their address for one year in iptables.
If you're interested in analysing the attack, boot to some livecd, and
make two copies of the entire drive to image files on other drives
with dd. Then compare both images' hashes and the hash of the drive.
Then restore your last backup after reformatting the current drive, or
on a new drive. Then loopback mount the image, and use gnu find to
look for all files changed during the period of suspected compromise.
They could have changed the times of files they owned though, so the
better way would be to compare its state at time of imagine to its
state at the time of your last trusted backup.
Jeffrey Watts
You really won't have a choice about using that Residential Gateway.
To get service, at all, I think you'll have to leave that machine in
place. But don't trust it as your firewall. Put another firewall
between you and it, and just consider everything outside your firewall
own as untrusted.
Thanks for clarifying. I meant for him to put the new firewall in between his AT&T box and the rest of the network. :)
If you are going to put services out there where they are reachable on
the internet, you need to plan on keeping them patched. You should
either set it up to update automatically (does anyone here have
experience with that) or set up some way to remind you to do it. You
really can't afford to let yourself get significantly behind on
patches for a service that is internet facing.
Most Linux distros have an update script that should run automatically. If it's not, it usually just needs to be chkconfiged on.
J.
Rezty Felty
Rezty
Billy Crook
makes me more secure.... Surely history has shown that anything
standard is insecure... I also randomize the pocket where I keep my
wallet, and car keys, Occasionally drive on the opposite side of the
road, etc...</tongue>
Christofer C. Bell
My two cents worth, in addition to the plethora of good suggestions you have already gotten, I always have my external ssh on a non-standard port. No sense in making it an easier for hackers than you have to. ;)
--
Chris
Dustin Decker
"If only there were evil people somewhere insidiously committing evil deeds, and it were necessary to separate them from the rest of us and destroy them. But the line dividing good and evil cuts through the heart of every human being. And who is willing to destroy a piece of his own heart?"
~ Alexander Solzhenitsyn
Kendric Beachey
postmortem...just offering up the evidence in case anyone likes
looking at these things as a hobby sort of thing.
Nick Mucci took me up on the offer; I burned the bad dude's home dir
to a CD and passed it to him.
The actual machine has been reinstalled from scratch with jaunty, and
all updates applied (and set to do security updates automatically).
I've been unable to get the nvidia glx drivers working since the
reinstall, so I'm on the old "nv" driver, but as a consolation, that
means my screen actually shows what I'm doing when I'm logging in.
With the "real" nvidia drivers it apparently used some weird mode that
my monitor couldn't show, so I had to login blindly.
Kendric Beachey