[ale] Diff the whole file system?

663 views
Skip to first unread message

James Sumners

unread,
Mar 23, 2012, 9:10:20 AM3/23/12
to Atlanta Linux Enthusiasts - Yes! We run Linux!
I have a situation where I'm being forced to allow a remote installer
to have root level sudo access to install their company's product
(don't like it, but it's out of my hands). Technically, I have the
system setup such that they should not need such access, but I can't
change the monkey's script. Anyway, I'd like to be able to sort of
"snapshot" my file system before I let them in so that I can go back
and look at a before and after difference. Do any of you know of such
a tool? Could this be done with rsync?

I've read that LVM supports snapshots at the block level. Seeing as
they are block level snapshots I don't see how that will help me
figure out what the installer changed. I'd be able to revert the
changes, but not study them.

--
James Sumners
http://james.roomfullofmirrors.com/

"All governments suffer a recurring problem: Power attracts
pathological personalities. It is not that power corrupts but that it
is magnetic to the corruptible. Such people have a tendency to become
drunk on violence, a condition to which they are quickly addicted."

Missionaria Protectiva, Text QIV (decto)
CH:D 59
_______________________________________________
Ale mailing list
A...@ale.org
http://mail.ale.org/mailman/listinfo/ale
See JOBS, ANNOUNCE and SCHOOLS lists at
http://mail.ale.org/mailman/listinfo

Jim Kinney

unread,
Mar 23, 2012, 9:21:46 AM3/23/12
to Atlanta Linux Enthusiasts

Look for a tool called rootshell. It's like a full session recorder history tool with remote archival ability. Set it up as roots shell on sudo and logout on closing rootshell.

Brian Stanaland

unread,
Mar 23, 2012, 9:25:15 AM3/23/12
to Atlanta Linux Enthusiasts
Have you checked out Tripwire? It's typically used to check config files for changes so I don't know how well it'd handle the whole file system but it's worth a look.

Brian S.
--
The more laws and order are made prominent,
The more thieves and robbers there will be.
Lao-tzuThe Way of Lao-tzu
Chinese philosopher (604 BC - 531 BC)

James Sumners

unread,
Mar 23, 2012, 9:38:19 AM3/23/12
to Atlanta Linux Enthusiasts
I'm not having any luck finding this. Got a link?

On Fri, Mar 23, 2012 at 09:21, Jim Kinney <jim.k...@gmail.com> wrote:
> Look for a tool called rootshell. It's like a full session recorder history
> tool with remote archival ability. Set it up as roots shell on sudo and
> logout on closing rootshell.

--

James Sumners

unread,
Mar 23, 2012, 9:38:41 AM3/23/12
to Atlanta Linux Enthusiasts
Hmm, I forgot about tripwire. I'll check into it.

On Fri, Mar 23, 2012 at 09:25, Brian Stanaland <br...@stanaland.org> wrote:
> Have you checked out Tripwire? It's typically used to check config files for
> changes so I don't know how well it'd handle the whole file system but it's
> worth a look.
>
> Brian S.

--

James Sumners

unread,
Mar 23, 2012, 10:01:28 AM3/23/12
to Atlanta Linux Enthusiasts
*sigh* Of course Tripwire isn't in the RHEL5 repositories. But
searching for it did lead me to "radmind"[1] in the rpmforge repo I
have added. This sounds exactly like what I need. It also sounds
completely awesome.

[1] -- http://rsug.itd.umich.edu/software/radmind/

Ed Cashin

unread,
Mar 23, 2012, 10:05:18 AM3/23/12
to Atlanta Linux Enthusiasts
I wrote something a lot more simple than tripwire. tripwire was fine
but took forever to fully understand and didn't seem convenient
enough. My integrit software was designed to be "done", not active,
so it hasn't been updated in a long time, but it ought to work.

http://sourceforge.net/projects/integrit/

But anyway that will just tell you what has changed, not how, as diff
would for text files.

To get a diff, you could use rsync to make a remote backup like this
at leisure beforehand (untested):

set -xe
for d in / /var; do
nice -n 20 rsync -axc "$d" fruity:/backups/A"$d"
done

... then later on host "fruity", create hard links (a fast operation
taking up very little extra space),

cp -al /backups/A /backups/B

... then after the changes have been made on the host you're worried
about, rsync again but do it to "B", not "A", transmitting only things
that have changed. That way on fruity, you can do,

cd /backups
diff -urN A B

--
  Ed Cashin <eca...@noserose.net>
  http://noserose.net/e/
  http://www.coraid.com/

Lightner, Jeff

unread,
Mar 23, 2012, 10:17:00 AM3/23/12
to Atlanta Linux Enthusiasts
I once had to give sudo access to DBAs to run Oracle's root.sh script for installations. Since it would be way to easy to modify root.sh to exploit root I setup a script that would mail the contents of root.sh before executing it. That way I could examine the script they actually ran to verify it hadn't been monkeyed with.

#!/bin/bash
#
# Script to allow DBAs to run root.sh script during installation.
# Initial write 12-Mar-2008 jlightne
#

ROOT_SH=$1
PID=$$

# First see if this was called directly via sudo. Error out if not.
# (User must run as themselves NOT as Oracle/Application admin user.)
#
if [ ! $SUDO_USER ]
then echo ERROR: This command must be run by sudo.
exit 1
fi

# Set a temporary working directory then get environment information
# that was passed into this invocation for later email/review.
#
SUDO_TMP=/usr/local/bin/sudo_tmp
SETFILE=${SUDO_TMP}/root_sh_env.$PID
echo "'who am i' reports: `who am i`" >$SETFILE
set >>$SETFILE


# Insure /bin and /usr/bin are in PATH
#
PATH=$PATH:/bin:/usr/bin

# To do the attachments we need uuencode. Error out if it doesn't exist.
# FYI: sharutils is the package that installs uuencode.
#
UUENCODE=/usr/bin/uuencode
if [ ! -x $UUENCODE ]
then echo "ERROR: uuencode command not found. Contact UNIX Admins."
exit 1
#else echo uuencode exists
fi

# Get basename of the script requested to be run. Error out if not "root.sh"
#
BASEROOTSH=`basename $ROOT_SH`
if [ ! ${BASEROOTSH} = root.sh ]
then echo "ERROR: This command only works on root.sh scripts provided by Oracle"
exit 1
else echo "Command is root.sh"
fi

# Allow user to run ./root.sh or /full/pathto/root.sh - determine top level
# directory either based on user input of full path or with pwd command if "./".
#
if echo $ROOT_SH |grep '\./root.sh' >/dev/null 2>&1
then PARENTDIR=`pwd |awk -F/ '{print $2}'`
DIRNAME=`pwd`
else PARENTDIR=`echo $ROOT_SH |awk -F/ '{print $2}'`
DIRNAME=`dirname $ROOT_SH`
fi

# If the tol level directory determine above is not /oracle or /appl_top
# error out and tell user it must be one of those.
#
echo Top level directory is /$PARENTDIR
if ! [ ${PARENTDIR} = oracle -o ${PARENTDIR} = appl_top ]
then echo -e "ERROR: This command only works on root.sh under /oracle or /appl_top \n top level directories."
exit 1
fi

# Specifically prevent use of ".." within full path if input to prevent
# attempts to bypass top level directory test (
# e.g. /oracle/../root/root.sh not allowed because it same as /root/root.sh
#
echo Full path of command is ${DIRNAME}/${BASEROOTSH}.
if echo $ROOT_SH |grep '\.\.' >/dev/null 2>&1
then echo ERROR: You may not include .. in path of root.sh. Input full path.
exit 1
fi

# Now we need to convert the root.sh to dos ascii so it will be readable as
# attachment when emailed.
#
unix2dos -q -n $ROOT_SH ${SUDO_TMP}/root.sh.$PID
unix2dos -q $SETFILE

# Now do the email to UNIX Admins.
# Sent the text of the root.sh as an attachment and also send the enviornment
# file created earlier as an attachment.
# NOTE: Exchange spam filter blows up on .sh and other suffixes which is why
# .txt is appended to the file names sent.
#
(echo -e "${DIRNAME}/${BASEROOTSH} run by ${SUDO_USER} at `date`.\n\nSee attachments for contents of ${BASEROOTSH} as well as the environment at the time it was run.";uuencode ${SUDO_TMP}/root.sh.$PID root.sh.txt;uuencode $SETFILE env.txt) |mailx -s "Oracle root.sh run by ${SUDO_USER}" us...@example.com us...@example.com

# Wait 10 seconds before actually running the script to give the email time
# to be sent in case this is a hack attempt.
#
echo $ROOT_SH will run as root in 10 seconds.
sleep 10
echo Running now.
$ROOT_SH

# Remove the temporary file created earlier
#
rm ${SUDO_TMP}/root.sh.$PID $SETFILE

http://sourceforge.net/projects/integrit/

cp -al /backups/A /backups/B


Athena®, Created for the Cause(tm)
Making a Difference in the Fight Against Breast Cancer

---------------------------------
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential information and is for the sole use of the intended recipient(s). If you are not the intended recipient, any disclosure, copying, distribution, or use of the contents of this information is prohibited and may be unlawful. If you have received this electronic transmission in error, please reply immediately to the sender that you have received the message in error, and delete it. Thank you.
----------------------------------

al...@alanlee.org

unread,
Mar 23, 2012, 10:24:02 AM3/23/12
to Atlanta Linux Enthusiasts, James Sumners

 

Use rsync.  Before the install, rsync the file system to a backup directory.  Then after the install use rsync to compare.  Example command lines for the backup:

 

rsync -rvax / /backup_mounted_fs/

 

OR

 

rsync -rvax / joe@othermachine:/pre_install_backup/


Example command lines for the compare:


rsync -rvaxcn / /backup_mounted_fs/

rsync -rvaxcn / joe@othermachine:/pre_install_backup/


Note the trailing slashes, they are needed.  Also note the addition of 'c' and 'n' options on the compare.  They cause rsync to compare using checksums instead of file metadata and to perform a test run without actually changing data respectively - with the 'v' flag outputting what it would have synced.  'x' instructs rsync to not descend into other file systems (eg. /proc, etc).

 

It's a very useful tool for things like this.

 

-Alan

James Sumners

unread,
Mar 23, 2012, 10:31:57 AM3/23/12
to Atlanta Linux Enthusiasts
Perfect! Thank you.

On Fri, Mar 23, 2012 at 10:05, Ed Cashin <eca...@noserose.net> wrote:
> I wrote something a lot more simple than tripwire.  tripwire was fine
> but took forever to fully understand and didn't seem convenient
> enough.  My integrit software was designed to be "done", not active,
> so it hasn't been updated in a long time, but it ought to work.
>
>  http://sourceforge.net/projects/integrit/

--

James Sumners

unread,
Mar 23, 2012, 10:34:28 AM3/23/12
to al...@alanlee.org, Atlanta Linux Enthusiasts
I think I'm going to go with Ed Cashin's tool Integrit, but this is
handy information to have. I might just do both.

al...@alanlee.org

unread,
Mar 23, 2012, 10:52:56 AM3/23/12
to James Sumners, Atlanta Linux Enthusiasts

 

One more thing I forgot.  You can also supply a --delete argument to the compare line to have the rsync test run output what is no longer present from source to destination.

 

Glad I could help.

 

-Alan H.

Jim Kinney

unread,
Mar 23, 2012, 12:28:27 PM3/23/12
to Atlanta Linux Enthusiasts

Doh! It's rootsh.
http://sourceforge.net/projects/rootsh/

On Mar 23, 2012 9:41 AM, "James Sumners" <james....@gmail.com> wrote:

Watson, Keith

unread,
Mar 23, 2012, 1:27:42 PM3/23/12
to Atlanta Linux Enthusiasts
Here are some tools that might help:

CheckInstall
http://asic-linux.com.mx/~izto/checkinstall/


Installwatch
http://asic-linux.com.mx/~izto/checkinstall/installwatch.html


instmon
http://freecode.com/projects/instmon


sinstall
http://sourceforge.net/projects/sinstall/


slacktrack
http://freecode.com/projects/slacktrack


strace Analyzer
http://en.community.dell.com/techcenter/high-performance-computing/w/wiki/2264.aspx
http://preview.tinyurl.com/7c7hf79


keith


--

Keith R. Watson Georgia Institute of Technology
IT Support professional Lead College of Computing
keith....@cc.gatech.edu 801 Atlantic Drive NW
(404) 385-7401 Atlanta, GA 30332-0280

Brian Mathis

unread,
Mar 23, 2012, 2:43:54 PM3/23/12
to Atlanta Linux Enthusiasts
On Fri, Mar 23, 2012 at 9:10 AM, James Sumners <james....@gmail.com> wrote:
> I have a situation where I'm being forced to allow a remote installer
> to have root level sudo access to install their company's product
> (don't like it, but it's out of my hands). Technically, I have the
> system setup such that they should not need such access, but I can't
> change the monkey's script. Anyway, I'd like to be able to sort of
> "snapshot" my file system before I let them in so that I can go back
> and look at a before and after difference. Do any of you know of such
> a tool? Could this be done with rsync?
>
> I've read that LVM supports snapshots at the block level. Seeing as
> they are block level snapshots I don't see how that will help me
> figure out what the installer changed. I'd be able to revert the
> changes, but not study them.
>
> James Sumners


The standard RHEL replacement for Tripwire is AIDE. It's good enough
for detecting what files have changed, but doesn't have diffs.

You mention LVM snapshots, and that's really the best way to go. You
can take the LVM snapshot and mount it somewhere, then you can do all
your diffs between that and the live filesystem. LVM snapshots would
be quite useless if there was no ability to access the snapshot in
some way, and they don't require double the disk space like other
methods.

The only issue with LVM snapshots is that you need enough free PEs in
the VG to hold the changes made to the filesystem. Most OS installers
do not keep any free PEs. You can view free PEs using 'vgdisplay'.
If you don't have any free, you could shrink an LV to free up some
space, or add an additional disk to the VG.


❧ Brian Mathis

Aleksey Tsalolikhin

unread,
Mar 31, 2012, 10:02:19 PM3/31/12
to Atlanta Linux Enthusiasts
Hi, James. You might have handled this already, but just for your interest,
CFEngine has a file integrity monitoring feature, like Tripwire. It can track
file checksums and attributes (ownership, permissions, etc.).

There is an example of using this "detect_all_change" feature at
http://www.linuxjournal.com/article/10924Just replace "/etc" with "/".
Test on a non-production system first, this is pretty cpu and disk intensive.

Best,
-at

James Sumners

unread,
Mar 31, 2012, 11:45:13 PM3/31/12
to Atlanta Linux Enthusiasts
Thank you. Integrit was the perfect solution for me. It's very simple
to setup and use. I also did an rsync of the whole file system just to
be sure.
Reply all
Reply to author
Forward
0 new messages