store shell output of last command(s) in predefined variables
Rahul
(tcsh). Frequently one issues a command with a lot of output and then in
the very next command I want to operate on it. Can I tweak my shell (via
.cshrc or functions etc.) so that it always retains the STDOUT output of
the last command in a variable?
A typical application:
%>find . -name 'foo'
[pageloads of data]
Current workflow
%>find . -name 'foo' | grep 'specific_foo'
This is inefficient especially for large timeconsuming finds. On the other
hand dumping data to a file each time beforehand is annoying since often
one plays with find parameters until one obtains a match one wants.
GOAL:
%>find . -name 'foo'
%>echo $LASTCMD | grep 'specific_foo'
%>echo $LASTCMD1 | grep 'specific_foo' [more ambitious version]
%>echo $LASTCMD2 | grep 'specific_foo' [more ambitious version]
The more ambitious versions could retain a rolling set of the outputs of
the last 10 commands say.
Any sugesstions what might be the best way to do this? My shell is tcsh but
of course I'm open to scripting in bash, perl whatever so long as I can see
the effects on my tcsh commandline.
--
Rahul
Chris Davies
> I had an idea that I was wondering how to impliment for my linux shell
> (tcsh). Frequently one issues a command with a lot of output and then in
> the very next command I want to operate on it. Can I tweak my shell (via
> .cshrc or functions etc.) so that it always retains the STDOUT output of
> the last command in a variable?
Probably not, but read on...
> A typical application:
> %>find . -name 'foo'
> [pageloads of data]
> Current workflow
> %>find . -name 'foo' | grep 'specific_foo'
Use !! to repeat a command (this works in (t)csh and also bash). So,
find . -name foo
!! | grep 'specific_foo'
!! | less
In general, read the man page to find out more about the ! substitution
operators. Some examples are,
!! is the entire previous line
!$ is the last argument of the previous command
!5 is the fifth command in the session history (usually, but not
always, starting from 1)
!!:3 is the third parameter to the previous command
Chris
pk
> Rahul <nos...@nospam.invalid> wrote:
>> I had an idea that I was wondering how to impliment for my linux shell
>> (tcsh). Frequently one issues a command with a lot of output and then in
>> the very next command I want to operate on it. Can I tweak my shell (via
>> .cshrc or functions etc.) so that it always retains the STDOUT output of
>> the last command in a variable?
>
> Probably not, but read on...
>
>
>> A typical application:
>
>> %>find . -name 'foo'
>> [pageloads of data]
>
>> Current workflow
>> %>find . -name 'foo' | grep 'specific_foo'
>
> Use !! to repeat a command (this works in (t)csh and also bash). So,
>
> find . -name foo
> !! | grep 'specific_foo'
> !! | less
I think (but I may be wrong) that his point was to avoid rerunning the
command for efficiency's sake, but somehow still be able to get its output
later.
Maxwell Lol
> I had an idea that I was wondering how to impliment for my linux shell
> (tcsh). Frequently one issues a command with a lot of output and then in
> the very next command I want to operate on it. Can I tweak my shell (via
> .cshrc or functions etc.) so that it always retains the STDOUT output of
> the last command in a variable?
You don't want EVERY command to go into a variable.
i.e.
echo $X
I think Chris gives you an answer.
If the program takes too long, use tee
long_command | tee file | more
You can create an alias like
alias tm "tee file | more"
You can also create an alias like
alias keep 'set A=(`\!-1`)'
Which, when executed, will repeat the last command, and save the
results in variable $A
> Current workflow
> %>find . -name 'foo' | grep 'specific_foo'
>
> This is inefficient especially for large timeconsuming finds. On the other
> hand dumping data to a file each time beforehand is annoying since often
> one plays with find parameters until one obtains a match one wants.
Then save the results of the find to a file, and then
try different variations of grep using the same file.
This is better that storing the results in a variable, because a
variable cannot store as much informaiton as a file.
Robert Riches
It could be rather costly to store the output of the last 10
commands in shell memory. What if the output of each
command is a few megabytes? I would _DISABLE_ such a
feature if it was enabled by default.
If you're not averse to rerunning the first command again,
you can do something like I do quite frequently at work.
Let's say I am one directory down from the root of a
development tree, I have edited a bunch of files in the
tree, and now I want to remove the '~' backup files. I
often do these two commands:
find .. -iname '*~'
rm -i `!!`
The second command reruns the first command (the !! part),
and the backquotes (the `` part) puts the output of that
command into the argument list for the second command.
In this case, if I see from the first few files that all is
well with the command, I control-C out of it, and do
^-i
to start over without the -i iption.
(Yes, I could have used -name rather than -iname. However,
I'm getting in the habit of using -iname rather than -name
for other purposes, and the habit is growing roots.)
If rerunning the first command is not something you want to
do, then how about this:
find .. -iname '*fred*' | tee z91
rm -i `cat z91`
The first commands puts the output of 'find' into file z91
and sends it to your terminal. The second command puts the
contents of each line of the file into a command line
argument for the second command.
--
Robert Riches
spamt...@verizon.net
(Yes, that is one of my email addresses.)
Chris Davies
> I had an idea that I was wondering how to impliment for my linux shell
> (tcsh). Frequently one issues a command with a lot of output and then in
pk <p...@pk.invalid> wrote:
> I think (but I may be wrong) that his point was to avoid rerunning the
> command for efficiency's sake, but somehow still be able to get its output
> later.
Ah. In that case, maybe something like this could help. It's for {k,ba,}sh
shells rather than the {t,}csh variant, but maybe it could be used as
a basis for something the OP could write themself?
Basic function:
t() { eval T$1="/tmp/tee_$1" TN=\$"T$1"; shift 1; "$@" | tee "$TN"; }
Usage:
t 1 find . -iname '*~' # Saves in file #1, i.e. /tmp/tee_1
t 1 find . -iname '*~' # Saves in file #1, i.e. /tmp/tee_1
t 2 grep wibble... <$T1 # T1 is file #1; t 2 saves in file #2
less $TN # TN is always the "last" file saved
Or:
t 1 find . -iname '*~' | t 2 grep wibble | less
This could be extended fairly trivially to accept an "x" as an argument
instead of the file number, such that this then removed all saved
output files.
Chris
Rahul
@news.roaima.co.uk:
> In general, read the man page to find out more about the ! substitution
> operators. Some examples are,
>
Thanks Chris. I already use all of those. Just discovered them a few months
ago and they are all super-useful.
> Use !! to repeat a command (this works in (t)csh and also bash). So,
>
> find . -name foo
> !! | grep 'specific_foo'
> !! | less
Does not work well for my case. This will repeat an execution. That's what
I am trying to avoid. A complex "find" or a "du" on a larger chunk of my
filesystem can easily take upwards of an hour.
--
Rahul
Rahul
> I think (but I may be wrong) that his point was to avoid rerunning the
> command for efficiency's sake, but somehow still be able to get its
> output later.
>
Absolutly right, pk. Effeciency of execution time combined with a
convinience of user syntax is the goal.
--
Rahul
Rahul
> Ah. In that case, maybe something like this could help. It's for
> {k,ba,}sh shells rather than the {t,}csh variant, but maybe it could
> be used as a basis for something the OP could write themself?
Sure. I think if there is a suitable epithet in any other shell I can do
the port to tcsh myself.
> Basic function:
>
> t() { eval T$1="/tmp/tee_$1" TN=\$"T$1"; shift 1; "$@" | tee
> "$TN"; }
>
> Usage:
>
> t 1 find . -iname '*~' # Saves in file #1, i.e.
> /tmp/tee_1 t 1 find . -iname '*~' # Saves in file #1,
> i.e. /tmp/tee_1
>
> This could be extended fairly trivially to accept an "x" as an
> argument instead of the file number, such that this then removed all
> saved output files.
>
Thanks for the code snippet Chris. But it is kind of "invasive" or
"obstrusive" on the workflow in the sense that it requires active
involvement of the user in pre-pending the "t 1" to each of his commands.
A lot of wasted keystrokes. Even before knowing whether that particular
find was successful for or not yet.
Let me try to explain better what I had in mind (maybe it is not
practical at all, who knows, but you guys can correct me!)
Look at PWD or TIME. They are "always there" and available whenever I
want them. But in the background. If I don't need them then I can be on
my shell for a day without noticing or invoking them. And I don't need to
plan *before* a command if I want to use them.
I know this analogy is simplistic but I have made creative use of
similar shell features in the past. Examples are the "precmd" alias
($PRECMD in bash) which I can get to unobstrusively set things for me.
Can something like this be hacked to just trap the STDOUT? I just need
the hook and then I can always add the shell fluff around it.
Crux of the matter: I do have a "postcmd". This will be executed by the
shell right after each command. I could add a wrapper script to postcmd I
feel. I can access exit status etc. in there. But how can I access the
last STDERR output from the buffer?
Response to some other issues other posters brought up:
1. It might be better to use temp files that SHELL VARIABLES. Great idea.
Point taken. A prototype might look like:
%>find . -name 'foo'
%>lastcmd | grep 'specific_foo'
%>lastcmd 1 | grep 'specific_foo' [more ambitious version]
etc.
lastcmd would be of the form:
cat /tmp/lastcmd/cmd1
etc.
2. There is of course, the risk that Robert points out about a runaway
command spewing a few MB of data and messing things up. But this could (I
feel) be handled in the wrapper by adding a circuit-breaker when the
STDERR exceeds a certain LIMIT (something >100k, say, is clearly useless
in most common situations).
Do people have any other ideas for me now?
--
Rahul
Chris Davies
> Sure. I think if there is a suitable epithet in any other shell I can do
> the port to tcsh myself.
That's fine. It's tricky to know how much expertise any given poster has,
so I tend to assume less until I know otherwise.
> Thanks for the code snippet Chris. But it is kind of "invasive" or
> "obstrusive" on the workflow [...]
It's not transparent, that's certainly true. I guess you could get used to
prefixing a command with "t", and it wouldn't be too hard to extend the
function I proffered to treat its first parameter as optional depending
on context (numeric or not).
> I know this analogy is simplistic but I have made creative use of
> similar shell features in the past. Examples are the "precmd" alias
> ($PRECMD in bash) which I can get to unobstrusively set things for me.
> Can something like this be hacked to just trap the STDOUT? I just need
> the hook and then I can always add the shell fluff around it.
I can't see anything that could do this. Maybe someone else can offer
a better suggestion.
Good luck
Chris
Dave Gibson
[snip: capturing a program's output so it can be re-used]
> %>find . -name 'foo'
> %>lastcmd | grep 'specific_foo'
> %>lastcmd 1 | grep 'specific_foo' [more ambitious version]
> etc.
>
> lastcmd would be of the form:
>
> cat /tmp/lastcmd/cmd1
> etc.
script -a -f ~/.typescript
The typescript file will contain the programs' outputs delimited by
shell prompts -- use a numbered prompt and you'll be able to grovel
through the typescript using the prompts as triggers.
set prompt='## %h > '
find . -name 'foo'
lastcmd
A sample lastcmd (assumes a prompt of '## %h > '):
#! /bin/sh
typescript="${HOME}/.typescript"
n=$(tail -n 1 "$typescript")
n=${n#* }
n=${n%% *}
wanted=${1:-0}
awk -v go="^## $(( $n - $wanted - 1 )) >" \
-v stop="^## $(( $n - $wanted )) >" '
$0 ~ stop { exit }
$0 ~ go && !t { t = 1 ; next }
!t { next }
{ print $0 }' "$typescript"
Rahul
news:5jd086x...@perseus.gibson-hrd.abelgratis.co.uk:
> script -a -f ~/.typescript
>
Ah! That's a great idea. I plan to look at script sources to see how they
get a "tee" from the shell.
--
Rahul
John Hasler
> Ah! That's a great idea. I plan to look at script sources to see how they
> get a "tee" from the shell.
man tee
--
John Hasler
jo...@dhh.gt.org
Dancing Horse Hill
Elmwood, WI USA
Hactar
This seems to work (it's only a proof of concept) for bash:
#! /bin/bash
tempfile=/tmp/${0##*/}-$$
touch "$tempfile" || exit
while read line ; do
eval "$line" | tee "$tempfile"
OUTPUT="$(<$tempfile)"
done
rm "$tempfile"
I guess when "echo $OUTPUT" is executed, that happens in the script's
environment. So I could change the definition of $tempfile if I wanted.
There's also no prompt, none of your rc files get read, and so on. And
it's bash, not tcsh. LIS, proof of concept.
--
-eben QebWe...@vTerYizUonI.nOetP royalty.mine.nu:81
Q: Why do black holes never learn?
A: Because they're too dense. -- ZurkisPhreek on Fark
Rahul
> I guess when "echo $OUTPUT" is executed, that happens in the script's
> environment. So I could change the definition of $tempfile if I wanted.
> There's also no prompt, none of your rc files get read, and so on. And
> it's bash, not tcsh. LIS, proof of concept.
>
Thanks Hactar!
--
Rahul