Missing log files?

[Karl Schmidt]

I understand that wajig no longer makes a log file... does it also delete the one that is there if
you upgrade?

I wish the log was still there - as it was in ~/.wajig/host/Log - no logrotate removed the history (
a good thing )

Having that saved me several times - would it be possible to bring it back as an option?

As an example of how I've used it - I was looking to see what I installed to get network boot to
work - but the log file is no longer there.


--
--------------------------------------------------------------------------------
Karl Schmidt EMail Ka...@xtronics.com
Transtronics, Inc. WEB http://secure.transtronics.com
3209 West 9th Street Ph (785) 841-3089
Lawrence, KS 66049 FAX (785) 841-0434

Truth is mighty and will prevail.
There is nothing wrong with this,
except that it ain't so.
--Mark Twain

--------------------------------------------------------------------------------

[Tshepang Lekhonkhobe]

On Thu, Jun 6, 2013 at 11:05 PM, Karl Schmidt <ka...@xtronics.com> wrote:
> I understand that wajig no longer makes a log file... does it also delete
> the one that is there if you upgrade?
>
> I wish the log was still there - as it was in ~/.wajig/host/Log - no
> logrotate removed the history ( a good thing )
>
> Having that saved me several times - would it be possible to bring it back
> as an option?
>
> As an example of how I've used it - I was looking to see what I installed to
> get network boot to work - but the log file is no longer there.

APT keeps a log in /var/log/apt/history.log and dpkg keeps one on
/var/log/dpkg.log. Here's the reasons I have for removing it in the
changelog (wajig 2.0.50):

* LIST-LOG: Remove custom logging implementation
- was fragile: sometimes info was lost during a run
- was expensive: checked system cache both before and after the
operation to determine what happened; it also did this for
commands
which didn't even change the system cache
- did not log cache changes when they were done by other tools
(apt, aptitude, & synaptic)
- APT log looks much better

[Karl Schmidt]

On 06/06/2013 04:58 PM, Tshepang Lekhonkhobe wrote:
> APT keeps a log in /var/log/apt/history.log and dpkg keeps one on
> /var/log/dpkg.log. Here's the reasons I have for removing it in the
> changelog (wajig 2.0.50):
>
> * LIST-LOG: Remove custom logging implementation
> - was fragile: sometimes info was lost during a run
> - was expensive: checked system cache both before and after the
> operation to determine what happened; it also did this for
> commands
> which didn't even change the system cache
> - did not log cache changes when they were done by other tools

> (apt, aptitude,& synaptic)

> - APT log looks much better
>

Why were the existing logs were automatically deleted? Not a sweet move.

Others looking for breadcrumbs...
The standard install of apt and dpkg in /etc/logrotate.d/ only keeps the logs for a year -- rotates
so you have to unpack grep multiple files etc - set the size to 0 to prevent rotation - rename after
release upgrade.

[Tshepang Lekhonkhobe]

On Fri, Jun 7, 2013 at 1:01 AM, Karl Schmidt <ka...@xtronics.com> wrote:
> On 06/06/2013 04:58 PM, Tshepang Lekhonkhobe wrote:
>>
>> APT keeps a log in /var/log/apt/history.log and dpkg keeps one on
>> /var/log/dpkg.log. Here's the reasons I have for removing it in the
>> changelog (wajig 2.0.50):
>>
>> * LIST-LOG: Remove custom logging implementation
>> - was fragile: sometimes info was lost during a run
>> - was expensive: checked system cache both before and after the
>> operation to determine what happened; it also did this for
>> commands
>> which didn't even change the system cache
>> - did not log cache changes when they were done by other tools
>> (apt, aptitude,& synaptic)
>>
>> - APT log looks much better
>>
>
>
> Why were the existing logs were automatically deleted? Not a sweet move.

silly move from my side; I did not like the idea of leaving stuff
around, and did not expect anyone to value such stuff

> Others looking for breadcrumbs...
> The standard install of apt and dpkg in /etc/logrotate.d/ only keeps the
> logs for a year -- rotates so you have to unpack grep multiple files etc -
> set the size to 0 to prevent rotation - rename after release upgrade.

maybe that warrants bug reports for those tools

[Karl Schmidt]

It turns out that the apt log system really fails to do the job that the old wajig log did (too much
information is logged - and there appears that it isn't working correctly - there are missing
entries.). I sure wish you would put it back in. I've tried grepping for install with bad results

The wajig log just showed what was installed or removed with a date - all on one line - very useful.

--------------------------------------------------------------------------------
Karl Schmidt EMail Ka...@xtronics.com
Transtronics, Inc. WEB http://secure.transtronics.com
3209 West 9th Street Ph (785) 841-3089
Lawrence, KS 66049 FAX (785) 841-0434

When everything is coming your way, it means you are in the wrong lane.
--------------------------------------------------------------------------------

[Tshepang Lekhonkhobe]

On Sat, Aug 3, 2013 at 11:12 PM, Karl Schmidt <ka...@xtronics.com> wrote:
> It turns out that the apt log system really fails to do the job that the old
> wajig log did (too much information is logged - and there appears that it
> isn't working correctly - there are missing entries.). I sure wish you would
> put it back in. I've tried grepping for install with bad results

If there are missing entries you may wanna file a bug report against APT.

> The wajig log just showed what was installed or removed with a date - all on
> one line - very useful.

Why not just use a pager, and use it's search feature?

[Karl Schmidt]

On 08/03/2013 08:46 PM, Tshepang Lekhonkhobe wrote:
> Why not just use a pager, and use it's search feature?

At one time, wajig used to solve several problems in a simple way (KISS). It had simple commands
that were accessible via the 'commands' command that could be grepped for a one-line reminder. It
did not require keystrokes needing dashes (for commandline switches or otherwise) , it was
scriptable, it simplified backing up and restoring a installed file list. It kept a simple log that
would help figure out what install broke things.

The most important thing it provided was orthogonality to the
package interface - and that is disappearing with a bunch of --commandline switches. The whole point
of wajig was the wajig <command> <argument> simplicity. This was the paradigm of wajig - unify the
apt and dpkg commands into a sane and simple tool.

Now this will sound harsh - and I don't want you to take it personally, but it sad to say that I've
never seen such an excellent tool go so far backwards.

--------------------------------------------------------------------------------
Karl Schmidt EMail Ka...@xtronics.com
Transtronics, Inc. WEB http://secure.transtronics.com
3209 West 9th Street Ph (785) 841-3089
Lawrence, KS 66049 FAX (785) 841-0434

Those that have knowledge don't predict.
Those that predict, don't have knowledge.
Lao tzu


--------------------------------------------------------------------------------

[Tshepang Lekhonkhobe]

On Sun, Aug 4, 2013 at 7:25 PM, Karl Schmidt <ka...@xtronics.com> wrote:
> On 08/03/2013 08:46 PM, Tshepang Lekhonkhobe wrote:
>>
>> Why not just use a pager, and use it's search feature?
>
>
> At one time, wajig used to solve several problems in a simple way (KISS). It
> had simple commands that were accessible via the 'commands' command that
> could be grepped for a one-line reminder. It did not require keystrokes
> needing dashes (for commandline switches or otherwise) , it was scriptable,
> it simplified backing up and restoring a installed file list. It kept a
> simple log that would help figure out what install broke things.
>
> The most important thing it provided was orthogonality to the
> package interface - and that is disappearing with a bunch of --commandline
> switches. The whole point of wajig was the wajig <command> <argument>
> simplicity. This was the paradigm of wajig - unify the apt and dpkg commands
> into a sane and simple tool.
>
> Now this will sound harsh - and I don't want you to take it personally, but
> it sad to say that I've never seen such an excellent tool go so far
> backwards.

ouch! now I feel like a lesser person

[David Liontooth]

Unfortunately I have to concur -- wajig was a great tool that really
simplified Debian management, and for some reason it's being stripped of
some of its core features. The point of wajig was to simplify a vast
number of diverse and complex operations into something simple and
user-friendly. Of course those tasks could be achieved in other ways,
that goes without saying.

A major benefit with wajig used to be tracking when and which packages
had been upgraded, through list-log -- this is gone. At least in my
version 2.7.3, wajig newupgrades is the same as wajig toupgrade, and
toupgrade fails to list packages that for some reason are not
installable. Buggy. Lost features.

The thing is, wajig used to be stable and mature, I was a very happy
user. It was always the first package I would install in a fresh
installation, like a saddle on a horse. It's taken me by complete
surprise that the new maintainer felt the package needed a massive
overhaul. I don't need wajig as much as I used to, but I would have kept
using the old version if it had stayed available.

For package management stability and maturity matters. It's been
"improved" as if it were in alpha, throwing out years of careful and
helpful work. A fork would have been more appropriate, where a project
under development can attract new users who expect bugs and instability.

I appreciate that wajig needs to be updated, since the Debian sysadmin
tools keep changing. It's just that I'm seeing a package with a
completely new direction, focusing on standards rather than narrowly
targeting real utility for its users. George Williams' whole spirit of
making Debian maintenance easier to find and simpler to remember,
packing a wide range of tools into a predictable wrapper, seems to have
gone out the window.

Cheers,
David

[Reuben Thomas]

I've used wajig for a long time, and I am on the whole appreciative of the recent developments: I find I can type fewer commands to get the work done I want.

I have never scripted wajig or used its log.

As far as I can tell, the addition of flags is mostly a matter of preserving backwards compatibility when more powerful functionality is added, and of minimising the number of commands (though there are still many). I don't find them a problem: many commands have command-line switches.

I certainly felt that wajig was ripe for an overhaul, and I'm only sorry it hasn't been overhauled more thoroughly, with more old functionality removed. As a thinnish front-end, I would rather it were able to change, as the underlying tools have to worry about scriptability and backwards-compatibility, it's rather nice if wajig doesn't have to, in order to be able to change and improve.

[Tshepang Lekhonkhobe]

> --
> You received this message because you are subscribed to the Google Groups
> "wajig" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to wajig+un...@googlegroups.com.
> To post to this group, send email to wa...@googlegroups.com.
> Visit this group at http://groups.google.com/group/wajig.
> For more options, visit https://groups.google.com/groups/opt_out.

I am glad someone came to my rescue. I was really starting to feel
bad, especially considering the amount of work I put into wajig
(countless hours and hundreds of commits). It's just a pity that I
didn't start this mailing list early enough so as to ask for feedback
before changing (breaking?) stuff.

[Tshepang Lekhonkhobe]

Sad news that I made your world a little worse than it was. I thought
I was doing good work.

PS: Note that the version of wajig you've grown to love is available
at changeset 0 in the mercurial repository (before I came in to screw
everything up). It will be missing loads of bug fixes and nice
improvements, but then maybe it will make you happier.

[Reuben Thomas]

On 4 August 2013 21:02, Tshepang Lekhonkhobe <tshe...@gmail.com> wrote:

ons, visit https://groups.google.com/groups/opt_out.

I am glad someone came to my rescue. I was really starting to feel
bad, especially considering the amount of work I put into wajig
(countless hours and hundreds of commits). It's just a pity that I
didn't start this mailing list early enough so as to ask for feedback
before changing (breaking?) stuff.

Since I seem to have been one of the most active feature requesters and change suggesters in recent times, it'd be odd indeed if I were unsatisfied with the results; equally, I feel I should take some of the blame for dissatisfaction caused by recent changes.

I'd suggest that those dissatisfied speak up and suggest what could be done to make things better all round, without giving up the considerable improvements we've had recently.

--

http://rrt.sc3d.org

[David Liontooth]

Hi Tshepang,

You've approached the task with admirable energy and skill, and the
changes may well be worth it; let new users judge.

Best wishes,
David

[Karl Schmidt]

On 08/04/2013 03:07 PM, Tshepang Lekhonkhobe wrote:
> PS: Note that the version of wajig you've grown to love is available
> at changeset 0 in the mercurial repository (before I came in to screw
> everything up). It will be missing loads of bug fixes and nice
> improvements, but then maybe it will make you happier.

Now you make me feel bad for saying what I think really needed to be said. I REALLY appreciate the
upgrading work you have done. The problem is not all yours, the system has grown more complex, and
Graham Williams left VERY large shoes to fill. IF this mailing list had existed before the
transition, some of this could have been avoided. ( for instance - if people don't want the nice
clean, terse wajig log - there could be a setting in .wajig/config or even /etc/wajig )

popcon show about 147k of installs - on Debian - for people that have popcon installed.

This mailing list needs more of those people involved to help with the work and debate the direction
before things just happen.

As painful as my first email may be - please take to heart the paragraph about orthogonality.

One of my jobs was as director of engineering that oversaw the development of both complex hardware
and software. It is a reality that the people really close to the code see things that the people
who wrote the specification don't and it is equally true that those closest can miss the 'forest for
the trees'. If the conversation of these conflicting points of view doesn�t happen, the results are
disappointing. So in a way, what I'm saying is the users let you down by not being more involved. If
you had known just how valuable that log was, I don't think you would have removed it.

If nothing else, making versions available (via backports when possible. ) for review before it gets
into testing would be a good idea. ( I should have had a testing box going earlier to see what was
going on )

Just for the record, the default setup apparently logrotated my history away - something that did
not/would not happen with the wajig log. I had changed log rotate on the servers, but I missed one
of the desktops. The idea is I can install wajig - things just work without messing with ever more
configurations - that is not what apt provides. With the wajig log - I had the information I needed
- and just what I needed in one file - no logrotate file to edit and the logs have just what is
needed no more. If I do a release upgrade, I might delete the log and start over - keeping a log of
what was installed and purged over the course of a release is quite useful - I've been able to
reconstruct how bad things happened and when.

Compare these two logs:
wajig:
2013-07-07T13:16 upgrade firmware-linux-free 2.6.32-48squeeze3
2013-07-07T13:16 upgrade flashplugin-nonfree 1:2.8.2+squeeze1
2013-07-07T13:16 remove gecko-mediaplayer 0.9.9.2-1
2013-07-07T13:16 upgrade ghostscript 8.71~dfsg2-9+squeeze1
2013-07-07T13:16 remove gnome-mplayer 0.9.9.2-1
2013-07-07T13:16 upgrade gnupg 1.4.10-4+squeeze1
2013-07-07T13:16 upgrade gnupg-agent 2.0.14-2+squeeze1

Apt:
Start-Date: 2013-08-04 13:53:43
Commandline: apt-get install --auto-remove jackeq
Install: jackeq:amd64 (0.5.9-2)
End-Date: 2013-08-04 13:53:50

Start-Date: 2013-08-04 13:59:10
Commandline: apt-get install --auto-remove jack-tools jack-stdio pulseaudio-module-jack qjackctl
slv2-jack jack-rack vlc-plugin-jack qjackrcd
Install: slv2-jack:amd64 (0.6.6+dfsg1-2), qjackrcd:amd64 (1.0.6~ds0-1), pulseaudio-module-jack:amd64
(2.0-6.1), jack-stdio:amd64 (1.4-1), jack-rack:amd64 (1.4.8~rc1-1), jack-tools:amd64 (20101210-2),
vlc-plugin-jack:amd64 (2.0.6-dmo1), libraptor1:amd64 (1.4.21-7.1, automatic), liblrdf0:amd64
(0.4.0-5, automatic)
End-Date: 2013-08-04 13:59:17


Which one would you want to dig through when debugging at 3AM?

The old wajig log saved my but several times.

Here is an example from a still running squeeze machine:

$ grep bind9-host Log
2012-06-16T15:51 upgrade bind9-host 1:9.7.3.dfsg-1~squeeze5
2012-08-12T11:47 upgrade bind9-host 1:9.7.3.dfsg-1~squeeze6
2012-09-18T15:15 remove bind9-host 1:9.7.3.dfsg-1~squeeze6
2012-09-18T15:20 install bind9-host 1:9.7.3.dfsg-1~squeeze7
2012-10-29T20:03 upgrade bind9-host 1:9.7.3.dfsg-1~squeeze8
2013-07-07T13:16 upgrade bind9-host 1:9.7.3.dfsg-1~squeeze10

We just can't do that with the apt log.

Here is a grep for wajig in .bash_history which could be a source of information about what people
use the most.

wajig listfiles icedove |grep -i css
wajig details icedove
wajig update
wajig source icedove
wajig update
wajig source icedove
wajig update; wajig distupgrade
wajig stop cron
wajig start cron
wajig whichpkg /usr/sbin/service
wajig list screen
wajig details kscreensaver
wajig listfiles xscreensaver-gl
wajig listfiles xscreensaver-gl |grep slide
wajig reportbug xscreensaver-gl
wajig install google-talkplugin_current_amd64.deb
wajig listfiles google-talkplugin_current_amd64.deb
wajig purge google-talkplugin_current_amd64.deb
wajig purge google-talkplugin
wajig listall h265
wajig listall codex
wajig listall 264
wajig listall msx
wajig listall 264
wajig list |grep -i gnome
wajig listorphans
wajig purgeorphans
wajig autoremove
wajig commands |grep -iC3 depend
wajig commands |grep -iC3 hold
wajig commands |grep -iC6 force


Imagine the nasty syntax if all was done without wajig..

--------------------------------------------------------------------------------
Karl Schmidt EMail Ka...@xtronics.com

Transtronics, wajig Inc. WEB http://secure.transtronics.com

3209 West 9th Street Ph (785) 841-3089
Lawrence, KS 66049 FAX (785) 841-0434

The truth is easy to kill. But a lie, well told, is immortal. -- Mark Twain

--------------------------------------------------------------------------------

[Reuben Thomas]

On 5 August 2013 05:21, Karl Schmidt <ka...@xtronics.com> wrote:

As painful as my first email may be - please take to heart the paragraph about orthogonality.

I didn't understand this the first time, and I went back and re-read it and still don't understand it. Could you explain again, please?

[Tshepang Lekhonkhobe]

On Mon, Aug 5, 2013 at 6:21 AM, Karl Schmidt <ka...@xtronics.com> wrote:
> On 08/04/2013 03:07 PM, Tshepang Lekhonkhobe wrote:
>>
>> PS: Note that the version of wajig you've grown to love is available
>> at changeset 0 in the mercurial repository (before I came in to screw
>> everything up). It will be missing loads of bug fixes and nice
>> improvements, but then maybe it will make you happier.

[skip a mouthful]

The commit where I removed your beloved logger is e2ad1bb107af
(Mercurial). You will find my reasoning for removing it, but If you
are feeling energetic, you can re-implement the logging function, and
I will get the package uploaded to Debian soonest. If you intend going
ahead with this, be sure to contact me first, either here or off-list.
I myself do not have that energy, and do find the APT log more than
adequate.

Had I known that the feature was so important to someone, I would not
have removed it. Note that it was actually buggy (I don't remember the
details) and expensive (it slowed wajig down), and fixing it was too
much of a pain, so I didn't think it was worth having. But do forgive
me.

Anyways, for list-commands, let me see if I can make each command
description fit in one line for you. ETA: 10 days.

[Karl Schmidt]

On 08/05/2013 04:10 AM, Reuben Thomas wrote:

> On 5 August 2013 05:21, Karl Schmidt <ka...@xtronics.com <mailto:ka...@xtronics.com>> wrote:
>
>
> As painful as my first email may be - please take to heart the paragraph about orthogonality.
>
>
> I didn't understand this the first time, and I went back and re-read it and still don't understand
> it. Could you explain again, please?

orthogonality: Every command works in a similar fashion.

http://en.wikipedia.org/wiki/Orthogonality_%28programming%29

(as an old assembly programmer - in early Intel machines (which were picked because they were bad)
every register worked a bit different from the other - so assembly routines were always pushing a
popping to get the value in the right register that could do some function.. and programmers tended
to forget what value was in which register - it was a huge feat to write workable compilers. The
Motorola 6800 machines were quite different - there were pointer registers and accumulators that all
worked the same. IBM did not want machines that could compete with their rental contract business so
they picked the worst. The poor design of the Architecture and dyslexic syntax of the assembly
language is one of the reasons why M$ was 11 years behind coming out with a preemptive multitasking
GUI platform (Win95) compared with the Amiga)

Keeping the form for all commands is important

$ wajig command argument

Thus:

$ wajig fileinstall install-list.txt

follows the form - While:

$ wajig install --fileinput install-list.txt

does not.

Keeping with a standard form makes the tool intuitive.

When wajig appeared, we had a large number of commands and switches - both dpkg and apt-get with
dozens of command line switches. To fix some package problem meant having to refer constantly to man
pages and trying to remember if some switch was a capital or small .. yuck.

wajig magically wrapped around that mess and provided a consistent 'orthogonal', intuitive interface.

If some dependency command was foggy in your memory - $ wajig commands |grep depend


Who wants to remember
apt-get -P remove {package-name}
dpkg --info {.deb-package-name}
dpkg -l | grep -i 'sudo'
dpkg -L {package-name}
dpkg -s {package-name} | grep Status

It saved me so much time, I put up a web page about it:

http://wiki.xtronics.com/index.php/Wajig

--------------------------------------------------------------------------------
Karl Schmidt EMail Ka...@xtronics.com

Transtronics, Inc. WEB http://secure.transtronics.com

3209 West 9th Street Ph (785) 841-3089
Lawrence, KS 66049 FAX (785) 841-0434

Socialism is very similar to racism; one group of people is forced
to become servants to another group. kps
--------------------------------------------------------------------------------

[Karl Schmidt]

On 08/05/2013 01:49 PM, Tshepang Lekhonkhobe wrote:
> Had I known that the feature was so important to someone, I would not
> have removed it.

Like I said, it is my/our fault as much as yours.

> Note that it was actually buggy (I don't remember the
> details) and expensive (it slowed wajig down),

Never noticed the speed issue -- BTW the 'update' command speed issue is not in wajig - the
'apt-get update' currently in wheezy has some nasty clinker that makes it take too long.

> But do forgive me.
>
> Anyways, for list-commands, let me see if I can make each command
> description fit in one line for you. ETA: 10 days.

Again - we are talking with email and can miss each other - I sure don't want to sound mean - I hope
what I've been saying is constructive - I'm just a user and really appreciate you sharing your work
with us.

Now, if I could think of someway to bribe you to put the logging back in...

--------------------------------------------------------------------------------
Karl Schmidt EMail Ka...@xtronics.com

Transtronics, Inc. WEB http://secure.transtronics.com

3209 West 9th Street Ph (785) 841-3089
Lawrence, KS 66049 FAX (785) 841-0434

Never let school interfere with your education.
- Mark Twain
--------------------------------------------------------------------------------

[Tshepang Lekhonkhobe]

On Mon, Aug 5, 2013 at 9:26 PM, Karl Schmidt <ka...@xtronics.com> wrote:
> Now, if I could think of someway to bribe you to put the logging back in...

I have a gittip account: https://www.gittip.com/tshepang :)