capture the maximum amount of cmd history in bash

[hput]

Can anyone post some examples of how to setup bash configuration
files to capture the maximum amount of command history. When, even
though it is a single user host, often, multiple shells come into
play.

[Janis Papanagnou]

On 22.03.2022 16:25, hput wrote:
> Can anyone post some examples of how to setup bash configuration
> files to capture the maximum amount of command history.

What do you mean by that?

Capture the actual contents? Then use: fc -l 1

Or make history memory for shell windows as large as possible?

You can define any size by setting the HISTSIZE variable, as in

HISTSIZE=1000000

Since the history will be stored in a file (or maybe temporarily in
memory depending on shell), the limit is basically a runtime limit,
depending on your (available and used) system resources.

> When, even
> though it is a single user host, often, multiple shells come into
> play.

It may make sense to increase the default value for HISTSIZE (which
is typically around 500 lines), but often other additional approaches
might also serve one's needs. For example I defined a separate HISTFILE
for every shell instance; for me that makes sense because I usually
have a few windows for compiling stuff, some for testing, some for
playing, some for specific projects, one for managing web content and
servers, etc. So each history file needs not to store that many lines
because if I stay in the domain the commands re-appear frequently and
don't get pushed off the far end of the history queue by commands from
another domain. So I have in a shell resource file that is called from
my $HOME/.profile these lines defined

TTY=$( tty )
TTY=${TTY#/dev/}
TTY=${TTY//\//_}
export HISTFILE=${HOME}/.sh_history_${TTY}
export HISTSIZE=1000

to create history files named individually for each shell window.

Janis

[Eli the Bearded]

In comp.unix.shell, Janis Papanagnou <janis_pa...@hotmail.com> wrote:
> On 22.03.2022 16:25, hput wrote:
>> Can anyone post some examples of how to setup bash configuration
>> files to capture the maximum amount of command history.

In bash I've seen this trick:

PROMPT_COMMAND='history -a >(logger -t "user $(id -u); at $(date +%s); in ${PWD}"'

Then ensure logs are sent off-system.

> Since the history will be stored in a file (or maybe temporarily in
> memory depending on shell), the limit is basically a runtime limit,
> depending on your (available and used) system resources.

That's why you write to system logs an use network logging.

>> When, even though it is a single user host, often,
multiple shells come into
>> play.

/bin/sh or `busybox sh` will frustrate your attempts to keep history.

> For example I defined a separate
> HISTFILE for every shell instance; for me that makes sense because I
> usually have a few windows for compiling stuff, some for testing, some
> for playing, some for specific projects, one for managing web content
> and servers, etc.

For myself, I do something quite similar with tty named history files.

Elijah
------
not a history maximalist

[John-Paul Stewart]

On 2022-03-22 12:35, Janis Papanagnou wrote:
>
> It may make sense to increase the default value for HISTSIZE (which
> is typically around 500 lines), but often other additional approaches
> might also serve one's needs. For example I defined a separate HISTFILE
> for every shell instance; for me that makes sense because I usually
> have a few windows for compiling stuff, some for testing, some for
> playing, some for specific projects, one for managing web content and
> servers, etc.

I've taken the opposite approach with HISTFILE: I want all shell
windows to share a common (and up-to-date) history by setting:

PROMPT_COMMAND='history -a; history -r'

That ensures that each time a new shell prompt is displayed, the history
(i.e., the last command run) gets appended to HISTFILE, which then gets
re-read. That re-reading means that the last command typed in one shell
instance (xterm, in my case) is picked up by other shell sessions the
next time I do something in each.

Appending to the history file in PROMPT_COMMAND also has the side effect
of minimizing history loss in the event of an unclean shutdown. Only
the last command is lost from the history instead of the entire shell
session.

So that's another approach for the OP to consider, depending on what was
meant by "capture the maximum amount of command history".

[Grant Taylor]

It's not an answer to your question. But I would feel remiss if I
didn't point out that:

Shell history files are inherently owned by the user running the shell.
As such, said user will almost invariable have full access to modify the
file(s) to their heart's content.

I strongly suggest that you step back and clearly think about and define
/ state what your goal is in messing with shell history files.

I've worked many places that tried to use shell history as a secondary
means of auditing. It always and invariable failed in one or more ways.
Not the least of which is `kill -9 $$` to avoid most if not all
history logging at time of shell exit.



--
Grant. . . .
unix || die

[Janis Papanagnou]

On 22.03.2022 18:39, Eli the Bearded wrote:
> In comp.unix.shell, Janis Papanagnou <janis_pa...@hotmail.com> wrote:
>>
> In bash I've seen this trick:
>
> PROMPT_COMMAND='history -a >(logger -t "user $(id -u); at $(date +%s); in ${PWD}"'
>
> Then ensure logs are sent off-system.
>
>> Since the history will be stored in a file (or maybe temporarily in
>> memory depending on shell), the limit is basically a runtime limit,
>> depending on your (available and used) system resources.
>
> That's why you write to system logs an use network logging.

I'm not sure whether that's a solution for the history, typed commands
to be easily retrieved for re-running them or re-running them edited.
A logger's purpose is different to a shell history's purpose, isn't it?

I seem to recall that there's an additional issue with bash's history,
where someone gave the workaround 'history -a; history -r' to be able
to reliably obtain [in bash] the actual history image. (Ksh's history,
for example, doesn't seem to require such workarounds.)

Janis

[Eli the Bearded]

In comp.unix.shell, Janis Papanagnou <janis_pa...@hotmail.com> wrote:
> On 22.03.2022 18:39, Eli the Bearded wrote:
>> PROMPT_COMMAND='history -a >(logger -t "user $(id -u); at $(date +%s); in ${PWD}"'
>>
>> Then ensure logs are sent off-system.
>

> I'm not sure whether that's a solution for the history, typed commands
> to be easily retrieved for re-running them or re-running them edited.
> A logger's purpose is different to a shell history's purpose, isn't
> it?

Pretty sure the question was about saving history, not re-running it. :^)

The history-to-logger command above was from an emergency shellhost. The
idea is to assist in logging what was done when responding to system
emergencies for the incident report. I'll interject comments via 'echo'
when using that sort of thing.

echo restarting foo on advice of Bar
aws ec2 restart-instance...

I do that in `script` session logging, too.

Elijah
------
and will use ex or ed when it helps for logging

[hput]

Janis Papanagnou <janis_pa...@hotmail.com> writes:

> On 22.03.2022 16:25, hput wrote:
>> Can anyone post some examples of how to setup bash configuration
>> files to capture the maximum amount of command history.
>
> What do you mean by that?
>
> Capture the actual contents? Then use: fc -l 1
>
> Or make history memory for shell windows as large as possible?
>
> You can define any size by setting the HISTSIZE variable, as in
>
> HISTSIZE=1000000

I only meant that all commands issued in the bash shells that I use
via xterm end up in ~/.bash_history

My history related settings in .bashrc which I should have included
in OP
------- ------- ---=--- ------- -------
# [HP 101709_135944 BEGIN All HISTORY variablesT
HISTIGNORE='ls:cd:clear:fg:exit';export HISTIGNORE
HISTCONTROL='ignoreboth';export HISTCONTROL
HISTSIZE='30000';export HISTSIZE
HISTFILESIZE='100000';export HISTFILESIZE
HISTFILE=~/.bash_history;export HISTFILE

shopt -s histappend
## PROMPT_COMMAND="$PROMPT_COMMAND;history -a" ; export PROMPT_COMMAND
## END ALL HISTORY variables ]
------- ------- ---=--- ------- -------

I'm not sure how to set up the PROMPT_COMMAND line. Somewhere in this
thread has been suggested adding the -r flag to PROMPT_COMMAND. What
effect does that have?

Its commented because at one point something I did to that section
caused Whole PROMPT_COMMAND setup to appear at the end of output every
time I ran something in a shell. So I hoped to get an idea how to set
that up so that all cmds end up in ~/.bash_history

Not sure what is supposed to happen to its accumulated history if an
xterm running bash is killed or crashes somehow.

Or what happens if the ~/.bash_history ever reaches $HISTFILESIZE.

Thinking about how to save that file some how automatically.
Something like how logrotate works would be handy where
~/.bash_history would be rotated by size limit, to as many rotated
versions as I like.

> Since the history will be stored in a file (or maybe temporarily in
> memory depending on shell), the limit is basically a runtime limit,
> depending on your (available and used) system resources.
>
>> When, even
>> though it is a single user host, often, multiple shells come into
>> play.

[...] snipped helpful commentary

> TTY=$( tty )
> TTY=${TTY#/dev/}
> TTY=${TTY//\//_}
> export HISTFILE=${HOME}/.sh_history_${TTY}
> export HISTSIZE=1000
>
> to create history files named individually for each shell window.

Do you then add them to some accumulative file?

It must be clear to all that I need to study up on the whole topic of
shell history.

Can anyone offer some guidance on where I might find examples of
settings effecting the accumulation of shell history and a thorough
write up of how it all is supposed to work?

The coverage in man bash is pretty slim, and entirely devoid of
example settings. Especially anything about the workings of
PROMPT_COMMAND.

[Eli the Bearded]

In comp.unix.shell, hput <hpu...@zohomail.com> wrote:
> I'm not sure how to set up the PROMPT_COMMAND line. Somewhere in this
> thread has been suggested adding the -r flag to PROMPT_COMMAND. What
> effect does that have?

PROMPT_COMMAND is a variable. The command in the variabke is run by the
shell just prior to each PS1 display.

The -r was given to the 'history' bash built in command. It refreshes in
shell history with contents of history file each time. This presumably
helps if you have multiple shelks writing to the same history file.
(That is not my usual style.)

Elijah
------
one history file per tty

[Kenny McCormack]

In article <eli$22032...@qaz.wtf>,
Eli the Bearded <*@eli.users.panix.com> wrote:
...
>... if you have multiple shelks writing to the same history file.

>(That is not my usual style.)
>
>Elijah
>------
>one history file per tty

What is your implementation for that?

I'd like to see it and to compare it to my own.

--
They say compassion is a virtue, but I don't have the time!

- David Byrne -

[Eli the Bearded]

In comp.unix.shell, Kenny McCormack <gaz...@shell.xmission.com> wrote:
> Eli the Bearded <*@eli.users.panix.com> wrote:
>> one history file per tty
> What is your implementation for that?

tty=`tty || :`
...
HISTFILE=$HOME/.kshrc_history.${tty##*/}

Obviously from that, ksh is my preferred shell. I don't think I've
edited those lines in fifteen years. $() is my usual notation now.

Elijah
------
probably did something causing 'not a tty' errors for that first part

[hongy...@gmail.com]

AFAICT, there is no a simple but robust and feature rich solution for this type of work. I've a slightly revised project here[1] based on the original one [2].

[1] https://github.com/hongyi-zhao/ariadne#bash
[2] https://github.com/gawells/ariadne

HZ

[David W. Hodgins]

Thanks! Added the following line to my ~/.bashrc file ...
tty="$(tty || :)" HISTFILE="$HISTFILE"".${tty##*/}"

I now get .bash_history.1 for a konsole terminal, and .bash_history.tty5 after
using alt+ctrl+f5 to switch to a pty.

I doubt the || : is needed as .bashrc is only used in interactive shells, but
it doesn't do any harm.

Regards, Dave Hodgins

[Javier]

hput <hpu...@zohomail.com> wrote:
> Not sure what is supposed to happen to its accumulated history if an
> xterm running bash is killed or crashes somehow.

Even if xterm crashes, or is killed with signal 9 SIGKILL, the child
process running bash will receive SIGHUP (hang up), and receiving SIGHUP
bash will write its history normally as if it was terminated by typing exit.

To really stop the shell from saving its history you need to send
SIGKILL to the shell on purpose. For example you can kill the current
shell process by typing

kill -9 $$

In that case, the history of that shell process would be lost.