There's the folks like me who spend 90% of the day in the GUI, then
type _very carefully_--mostly from others' recipes--in the command
line; then there are the Corys and Dannys and Bens of the world who
seem perfectly happy to do everything in an ssh tunnel, only
occasionally popping their heads up to perform some task in a GUI app,
then back down they go.
In most cases, I imagine the two approaches reflect where people
started out (UNIX vs. System 7 or what have you). But, I really do get
the feeling that I'm _missing out_ by not spending a *lot* more time
learning stuff like bash and regular expressions inside out. At the
same time, I don't want to make my own ketchup or line my own brakes,
you know? There must be a reasonable line for each person's skillset.
So I guess my next question for the floor: when you have the option to
do something in either a terminal or in a GUI and the results will be
identical, how do you choose which you'll use? Are there tasks that us
schlubs are doing above ground that can be accomplished much more
efficiently (and more _easily_) in the command line? Examples? Nominate
2-3 commands we should learn and use instead of the GUI equivalent, por
favor.
[1]: http://del.icio.us/tag/osxcli (Great links for us terminal
n00bs)
I'm an admin. In my world, every thing important is a text file, and
Unix gives me 1000 tools to work with text files. Example: Yesterday,
I needed a list of all of the windows machines in the department. Our
computer inventory is (obviously) a text file.
# cat ~/work/inventory | grep -i window | awk '{print $4}' | sort |
mail -s "Windows Hosts"
For the uninitiated, this opened the file which contains the list of
computers, pulled out the lines which included window or WINDOW,
extracted the host name, alphabetized them, and composed an email
containing our findings.
Learning regular expressions is like learning another language, but
it's a language I use every day.
Numero 2:
I'll second John S J Anderson: file manipulation. Working with files,
especially multiple files at once, is an order of magnitude easier in
the shell than with any Finder (or replacement).
Typically, actually, any task that should be doable in just a few steps
is faster in a CLI (especially when scripted) than 5-10 mouse clicks at
disparate points on the screen.
http://pix.merlinmann.com/PathFinderBlowout.jpg
So, just in terms of workflow, if you find you need the CLI for a quick
task but don't feel like lugging out a new terminal window, you can
just pop a terminal drawer for whatever directory you're in.
Surprisingly handy and very much of the "two worlds," I think.
There's some other stuff there that's pretty nice too that I'll go into
in the article: text preview, running apps drawer, a drop pile, and a
series of "favorites" drawers. It's pretty great and well worth the
modest registration price (US$34).
[1]: http://www.cocoatech.com/pf.php (Path Finder is a very
powerful Finder replacement with a built-in terminal drawer, per-window)
As for ways to make the CLI work for you and how to approach I would
suggest learning about Bash aliases as a first step.
One thing everyone should do is customize their CLI environment. In
the CLI world many programs are customized through their various .
(dot) files. Bash itself is set up by .profile and .bashrc. ViM is
governed by its .vimrc. All of these are located in your home folder
(directory).
Now, how does one get started customizing all these varied dot files?
I started by cribbing from this site
http://www.dotfiles.com/
This is all just turning into some sort of massive mish-mash how-to.
But, I just highly suggest that once you start learning to make the CLI
a place you like, that looks the way you want and behaves as you want
that you'll find it far less intimidating. Once you've configured your
own prompt there's something of a feeling of satisfied conquest. Sort
of a "look ma, I did that" effect that just makes doing more, easier.
And, lastly don't be afraid to screw up. You will. The most that will
happen is you'll get an error message. And don't be afriad to ask some
Linux Guru. They're not the evil ogres they're made out to be and a
lot of them use or like OS X too.
Sorry to drag on like this but there's so much to learn about the CLI.
And, none of it should be frightening.
When I run Linux, I instinctively use the shell. I just don't get it.
There's something in the UI that's forcing me this way, and I don't
know what it is.
Looks incredibly flexible, but quite a learning curve. I'll pick up the
O'Reilly book I guess, and start whacking my way around.
Here goes nothing. :)
[1]: http://www.mindlube.com/products/emacs/ (OSX build of GNU
emacs)
Probably the most important thing to understand about the way that Unix
works, in general, is that it's a 'tool-based' system. Instead of doing
the Microsoftian design concept, wherein a single tool does everything
under the sun, there are a bunch of small tools that can be hooked
together to do something complex. There are some other examples in this
group of parsing a file and printing out a certian column of the data.
Some of my examples given, there are certianly Apple-specific ways to
do these things. But, these are things that *I* would do in the
commandline instead of manually.All of these things are also
'portable', in other words, they should work across all Unix-like
operating systems.
For a pre-flight checklist item, learn the 'man' command. It takes a
bit of getting used to to get the hang of how manpages are written.
But, once you do, you'll cry when you have to use systems that don't
have them. (e.g. windows) Being a good System Administrator doesn't
mean that you know everything. It means that you can quickly and
effectivly find answers to your problems.
I'm assming at this point, that you can do very basic shell commands,
such as:
mv - move a file (You can also use this to 'rename' a file)
cp - copy a file
rm - remove (delete) a file
ls - list contents of a directory
cd - change directory to something
After the basics, learn the 'find' command. Find out how powerful it
is. Say you have to do some housecleaning on your home directory
becuase you're out of disk space. So, try:
find ~/ -size +1024 -atime +30 -print
So, what does this do? ~/ is telling it to search your home directory.
~/ is short for your home directory on virtually all unix-like systems.
-size is finding anything above 1024 characters. You'll probably have
to mess with this one a bit to truly find large files
-atime is finding all the files that haven't been accessed in the last
30 days.
-print will print the results.
Unless you're amazingly tidy, unlike me, you'll probably end up with,
say, images in random places in your home directory. In a fit of
organization, you decide to make an images directory. Keep in mind,
this is somewhat dangerous, because you could move things you didn't
intend to. But, let's do a:
find ~/ -type f -name "*.jpg" -exec cp {} ~/images \;
Swahili, you say! Not quite. As a point of practice, do 'man find' find
specifically what this does. In short, it's going to find every 'file'
in your home directory that looks like *.jpg and copy it to ~/images.
One more example that I do use from time to time. You've downloaded
some sort of tarball from somewhere containing something. Some yo-yo
before you set permissions that makes it quite impossible for you to
reasonably use the contents. It's a bunch of flat text files/html
files/non-executable files.
find /path/to/dir -type f -exec chown 644 {} \;
We're going to go and find all of the files, and change the
permissions. Likewise, changing -type f to -type d to find the
directories. Change chown to 755 here, though. Read 'man chown' to get
an idea of what that does.
Find is a powerful tool, but it takes some getting used to. I've
certianly shot myself in the foot a number of times with it.
Another concept that I've brought up through the find examples above is
that of 'regular expressions.' If you've been around a commandline of
any sort, you've probably come across the * character, which stands for
'glob'. Basically, this means match anything.
Want to show all of the word documents in a given directory? Use ls
*.doc. Over the weekend I was scanning a bunch of images, doing minor
modifications to them in Photoshop, saving them as .psd, then using
photoshop to make a low-res .jpg version of them to put on my website.
I created a simple Photoshop action to help me with this, but it dumped
all of the images into the same directory. I find it kind of a pain to
sort through .psd and .jpg, because sometimes my itchy mouse finger
picks the wrong one when uploading images.
I always keep a 'temporary' folder in my home directory. It's ~/tmp,
and I use it in times like this. I can easily dump files in there, and
I know that it's safe to nuke them. So, I can do the following:
cd ~/Documents/images/tickets
mv *.jpg ~/tmp
Then I can go through them when I do the upload.
I haven't really gotten into chaining tools together, or using other
tools to do things. But, this is a good start. Good luck!
1) Add Terminal's ability to create a filepath out of a drag-and-drop
to perl's ability to, well, do just about anything, and you have the
makings of a custom, background file copy over the WAN that won't force
you to stay logged in while it chugs away (see #2). For example, I
have a script that accepts input (in the form of a dragged-and-dropped
file or folder), and copies (using psync) files between the two.
There's also an option to syncronize, ie delete at the target end. I
have other, longer scripts that run over rsync/ssh for when you're not
on the LAN/WAN, but this one I use daily and often. To wit:
#!/usr/bin/perl -w
print "Drag your source to the Terminal Window and hit the Return
key\n";
$FILESOURCE = (<STDIN>);
chomp($FILESOURCE);
print "\nDrag your destination to the Terminal Window and hit the
Return key\n";
$FILEDEST = (<STDIN>);
chomp($FILEDEST);
print "\nDo you want to delete files on the Destination that are not on
the Source? [y/N]:";
$DELETE = (<STDIN>);
chomp($DELETE);
## THIS IS WHERE THE PSYNC STARTS ##
if($DELETE =~ "y") {
print "you've chosen to delete...\n";
system("time psync -dv $FILESOURCE $FILEDEST
2>~/Desktop/logged_errors");
}else{
print "you've chosen not to delete...\n";
system("time psync -v $FILESOURCE $FILEDEST
2>~/Desktop/logged_errors");
}
2) Run the above in a custom screen (type "man screen" for more
information on this nifty feature), and I can log off my work computer,
take a leisurely walk home, have dinner, watch Law & Order, log back
into my work computer via ssh, type screen -R, and see the progress of
the file copy.
psync can be found at http://www.dan.co.jp/cases/macosx/psync.html
-J
bash-2.05b$ man screen
No manual entry for screen
bash-2.05b$ screen --help
bash: screen: command not found
I think I'll go sulk for a while.
As I previously mentioned, Unix is a group of tools that are designed
to be chained together. I'm going to give some examples that people
would find useful, as well as examples of more complex things that can
be easily done from the commandline. When trying new things out, it's
important that you don't do it as root, and it's always nice to have a
'sandbox' in my home directory to play with. Usually I use ~/tmp for
this. Simply COPY files you'd like to muck with to there, and have fun!
Probably two of the most common things you'll see in the Unix CLI is
that of a pipe, or |, and 'grep'. Piping commands can include
redirecting output somewhere, as we'll see. 'grep' is short for 'Global
Regular Expression Parser', which is short for 'find'. Although, 'grep'
is very powerful, more powerful than many 'find' implementations I've
seen.
The command 'ps' shows the processes that are running. The -aux flags
find all processes. We can pipe this to the 'more' command, which will
paginate the data based on your terminal breaks. So, at the prompt,
type 'ps -aux | more'. As you can see, you should get a page full of
data, whith a bytecount at the bottom. Simply press the spacebar to
continue through the listing. (As an alternative, try 'less'. How is it
different? Sometimes, more isn't less.:) )
As an example, consider that you have installed PostgreSQL on your
machine, which is a database server. You think it's running, but you're
not quite sure. How can you check? With the 'ps' command, of course!
We'll pipe the output of ps to grep to find the 'postgres' user, or
anything with 'postgres' in it. Use 'ps -aux | grep postgres'. You
should get a nice output of stuff with postgres.
The ps output can be used for troubleshooting purposes as well. Each of
the columns has information that is useful. For performance reasons,
the %CPU, %MEM and TIME columns are important. The PID column is
important if you want to kill a process, or do some other things for
it. Use 'man ps' for more information on this subject.
There are also some simple tools that I use. I know there are
alternatives for these, but these are easy for me to get to. 'bc' is a
simple calculator. 'cal' will show a calendar of the current month.
'date' shows the current date and time. Not sure what a file is? Use
'file filename'. Displaying the contents of a file? Use 'cat' for
nonstop display, 'more' or 'less' for a paginated display. I'm sure
there are more, but these are ones I use on a regular basis.
A task that I end up doing from time to time is trying to figure out
how many lines there are in a certian document. 'wc' is a command that
can help you do this. So, use 'cat filename | wc -l' to count the lines
in filename. Want to see how many processes are running on your system?
'ps -aux | wc -l'. Want to see how many times the word 'passed' is on a
line in a document? 'cat filename | grep passed | wc -l'.
If you poke around with CLI's very much, you'll eventually get to 'sed'
and 'awk'. Each of these are super-powerful command parsers that have
many many features.However, there are two fairly basic things that I do
with each on a regular basis.
First, with awk, it's really easy to print out a column of data. From
the previous paragraph where I was finding certian processes running. I
mentioned the PID column. As an example, say you want to print just
that column, along with the process.Using 'ps -aux | grep postgres |
awk '{print $2 $11}' ' will do this. The $2 and $11 are split off by
whitespace, so your milage may vary. However, you can easily change the
$2 to $1 to display the first whitespace column of data. This will work
on virtually any file.
Second, sed can be used to do find and replace in files. Where this
gets hairy is that the 'find' portion of this string can be a regular
expression, so it can be very complex to match very specific things in
a file. Regular Expressions are literally a programming language in
themselves, and take a lot of practice to get good at them. But, that
shouldn't stop us from trying. I have a log file from something that
has a bunch of lines with 'passed' in them. I want to change them to
failed. Use 'cat filename | sed -e 's/passed/failed/g' ' to do this.
You should immediatly see an output from this with your changed
strings.
>From any of these commands, if you want to save output to a file for
printing or for emailing to someone or whatever, use 'command >
file.txt'. after that, you'll get everything in the file.
Okay, now for the most complex part of this. Sometimes I have problems
with my cablemodem at home, and I'd like to keep track how long it
takes for pings to return from a certian host. I don't want to do these
forever, just a few pings every so often. Fortunatly, there's the
'cron' facility, which will allow you to schedule tasks at given
intravels. So, I'm going to schedule a ping every 5 minutes.
Type 'crontab -e' and enter in the following line. Of course, edit the
Ip address you'd like to ping:
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /sbin/ping -c 3 -s <somehost>
| grep icmp > /tmp/pinglog.txt
Exit out of your text editor, and it should activate the crontab. Wait
until the next period, and look at /tmp/pinglog.txt. Of course, you
might not want to do it all the time. Maybe just during work hours. Or
maybe weekends. Use 'man 5 cron' to find out. The 5 is important.
As a brief aside, man pages are broken up into sections. One section is
for programmers, one section is for commands, and so on. There's two
'cron' sections. Use 'man cron' and then 'man 5 cron' to display the
different sections.
Next, I'll do a 'good CLI tools to have installed' section as well as a
'slightly more advanced' troubleshooting guide.