For Loop Shell Script Linux
Amabella Tevebaugh
Every sysadmin probably has some skill they've learned over the years that they can point at and say, "That changed my world." That skill, or that bit of information, or that technique just changed how I do things. For many of us, that thing is looping in Bash. There are other approaches to automation that are certainly more robust or scalable. Most of them do not compare to the simplicity and ready usability of the for loop, though.
If you want to automate the configuration of thousands of systems, you should probably use Ansible. However, if you're trying to rename a thousand files, or execute the same command several times, then the for loop is definitely the right tool for the job.
The given set of items can be a literal set of objects or anything that Bash can extrapolate to a list. For example, text pulled from a file, the output of another Bash command, or parameters passed via the command line. Converting this loop structure into a Bash script is also trivial. In this article, we show you some examples of how a for loop can make you look like a command line hero, and then we take some of those examples and put them inside a more structured Bash script.
That's about as simple as it gets and there isn't a whole lot going on there, but it gets you started. The variable $name will contain the item in the list that the loop is currently operating on, and once the command (or commands) in the do section are carried out, the loop will move to the next item. You can also perform more than one action per loop. Anything between do and done will be executed. New commands just need a ; delimiting them.
This loop takes the output of the Bash command ls *.pdf and performs an action on each returned file name. In this case, we're adding today's date to the end of the file name (but before the file extension).
Imagine that you have a file that you want to scp to several servers. Remember that you can combine the for loop with other Bash features, such as shell expansion, which allows Bash to expand a list of items that are in a series. This can work for letters and numbers. For example:
You can also combine iterations. Imagine that you have two data centers, one in the United States, another in Canada, and the server's naming convention identifies which data center a server HA pair lived in. For example, web-us-0 would be the first web server in the US data center, while web-ca-0 would be web 0's counterpart in the CA data center. To execute something on both systems, you can use a sequence like this:
You can also combine some of these ideas for more complex use cases. For example, imagine that you want to copy a list of files to your web servers that follow the numbered naming convention you used in the previous example.
You can accomplish that by iterating a second list based on your first list through nested loops. This gets a little hard to follow when you're doing it as a one-liner, but it can definitely be done. Your nested for loop gets executed on every iteration of the parent for loop. Be sure to specify different variable names for each loop.
There might be other ways to get this done, but remember, this is just an example of things you can do with a for loop. What if you have a mountain of files named something like FILE002.txt, and you want to replace FILE with something like TEXT. Remember that in addition to Bash itself, you also have other open source tools at your disposal, like sed, grep, and more. You can combine those tools with the for loop, like this:
Running for loops directly on the command line is great and saves you a considerable amount of time for some tasks. In addition, you can include for loops as part of your Bash scripts for increased power, readability, and flexibility.
You can also increase the flexibility and reusability of your for loops by including them in Bash scripts that allow parameter input. For example, to rename files like the example More creative renaming above allowing the user to specify the name suffix, use this script:
In this script, the user provides the source file's prefix as the first parameter, the file suffix as the second, and the new prefix as the third parameter. For example, to rename all files starting with FILE, of type .txt to TEXT, execute the script like this :
This is similar to the original example, but now your users can specify other parameters to change the script behavior. For example, to rename all files now starting with TEXT to NEW, use the following:
Hopefully, these examples have demonstrated the power of a for loop at the Bash command line. You really can save a lot of time and perform tasks in a less error-prone way with loops. Just be careful. Your loops will do what you ask them to, even if you ask them to do something destructive by accident, like creating (or deleting) logical volumes or virtual disks.
Ricardo Gerardi is Technical Community Advocate for Enable Sysadmin and Enable Architect. He was previously a senior consultant at Red Hat Canada, where he specialized in IT automation with Ansible and OpenShift. More about me
The opinions expressed on this website are those of each author, not of the author's employer or of Red Hat. The content published on this site are community contributions and are for informational purpose only AND ARE NOT, AND ARE NOT INTENDED TO BE, RED HAT DOCUMENTATION, SUPPORT, OR ADVICE.
Using the output of ls to get filenames is a bad idea. It can lead to malfunctioning and even dangerous scripts. This is because a filename can contain any character except / and the nullcharacter, and ls does not use either of those characters as delimiters, so if a filename has a space or a newline, you will get unexpected results.
The shell expands */ into separate arguments that the for loop reads; even if there is a space, newline, or any other strange character in the filename, for will see each complete name as an atomic unit; it's not parsing the list in any way.
If you want to go recursively into subdirectories, then this won't do unless your shell has some extended globbing features (such as bash's globstar. If your shell doesn't have these features, or if you want to ensure that your script will work on a variety of systems, then the next option is to use find.
Here, the find command will call echo and pass it an argument of the filename. It does this once for each file it finds. As with the previous example, there is no parsing of a list of filenames; instead, a fileneame is sent completely as an argument.
The syntax of the -exec argument looks a little funny. find takes the first argument after -exec and treats that as the program to run, and every subsequent argument, it takes as an argument to pass to that program. There are two special arguments that -exec needs to see. The first one is ; this argument gets replaced with a filename that the previous parts of find generates. The second one is ;, which lets find know this is the end of the list of arguments to pass to the program; find needs this because you can continue with more arguments that are intended for find and not intended for the exec'd program. The reason for the \ is that the shell also treats ; specially - it represents the end of a command, so we need to escape it so that the shell gives it as an argument to find rather than consuming it for itself; another way of getting the shell to not treat it specially is to put it in in quotes: ';' works just as well as \; for this purpose.
Why not set IFS to a newline, then capture the output of ls in an array? Setting IFS to newline should resolve issues with funny characters in file names; using ls can be nice because it has a built-in sort facility.
A Bourne Shell Programming / Scripting Tutorial for learning about using the Unix shell. Learn Linux / Unix shell scripting by example along with the theory. I'll have you mastering Unix shell scripting in no time.
Most languages have the concept of loops: If we want to repeat a tasktwenty times, we don't want to have to type in the code twenty times, with maybe a slight change each time.
As a result, we have for and while loops in the Bourne shell. This is somewhat fewer features than other languages, but nobody claimed that shell programminghas the power of C.
This is well worth trying. Make sure that you understand what is happeninghere. Try it without the * and grasp the idea, then re-readthe Wildcards section and try it again withthe * in place. Try it also in different directories, and with the * surrounded bydouble quotes, and try it preceded by a backslash (\*)
What happens here, is that the echo and read statements will runindefinitely until you type "bye" when prompted.
Review Variables - Part I to see whywe set INPUT_STRING=hello before testing it. This makesit a repeat loop, not a traditional while loop.
The colon (:) always evaluates to true; whilst usingthis can be necessary sometimes, it is often preferable to use a realexit condition. Compare quitting the above loop with the one below;see which is the more elegant. Also think of some situations in which each one would be more useful than the other:
Another useful trick is the while read loop. This example uses the case statement,which we'll cover later. It reads from the file myfile.txt, and for each line, tells you what language it thinks is being used.
My Shell Scripting books, available in Paperback and eBook formats. This tutorial is more of a general introduction to Shell Scripting, the longer Shell Scripting: Expert Recipes for Linux, Bash and more book covers every aspect of Bash in detail.
I guess this may be a naive question but I can't get my head around so I felt like asking...I was searching for some solution to a problem, when I found this very interesting post about why is using [whilefor] loops in bash considered bad practice. There is a very good explanation in the post (see the chosen answer) but I can't find anything that solves the issues that are discussed.
d3342ee215