Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Small victories
154 views
Skip to first unread message
Tim Bradshaw
Feb 12, 2012, 5:05:28 PM2/12/12
to
I've been working on a contract whose project has just gone live. It's
an important system for the company concerned and it has nothing to do
with Lisp of course (it's the usual enterprise crapware).
Needless to say it fell over in the first few hours, the way these
things do (people were making half-serious jokes about whether it would
make it into The Register). It''s all back now but people would like
to understand what went wrong obviously.
I spent a long time on Friday grovelling ineffectively over log files
(really huge log files, simple awk scripts took tens of minutes to
process them) to try and find bad things. Today I realised that
there's a much smaller log file which contains just lists of API calls,
when they started, how long they took and the PID of the process that
handled them (which you can use to find the appropriate big log file to
look in). I realised that, if I could make a picture of the service
times that might be interesting (I can't do anything without pictures).
So I did that, via my standard trick of a tiny awk (or perl) front-end
to convert things from line-based into sexp-based, and then some Lisp
code to spit out something gnuplot could print.
And indeed it was interesting. Generally the service times were a
hundredth of a second or so, but there were some which were tens of
*minutes*. So, some more playing based on the original code and I have
a list of the offending services, times, and PIDs for things taking
more than some number of times the median time. With suitable settings
this give a few tens of calls which I can now go and look at the
detailed logs for to work out what was going on.
None of this is special magic Lisp code: in fact there's really nothing
remarkable about it at all. I could have done it in Perl. Was Lisp
better than Perl? For me, yes, because I'm a better (less bad) Lisp
programmer than Perl programmer and the environment I have for Lisp is
better than the one I have for Perl. But significantly it wasn't
*worse*: nothing was difficult, and nothing got in the way, it just all
worked easily and well. It was just *ordinary*, and I'm coming to the
view that that's what you want: a language which is just there and does
what you want without some glamourous computational pyrotechnics.
Common Lisp actually turns out to be quite good at that.
an important system for the company concerned and it has nothing to do
with Lisp of course (it's the usual enterprise crapware).
Needless to say it fell over in the first few hours, the way these
things do (people were making half-serious jokes about whether it would
make it into The Register). It''s all back now but people would like
to understand what went wrong obviously.
I spent a long time on Friday grovelling ineffectively over log files
(really huge log files, simple awk scripts took tens of minutes to
process them) to try and find bad things. Today I realised that
there's a much smaller log file which contains just lists of API calls,
when they started, how long they took and the PID of the process that
handled them (which you can use to find the appropriate big log file to
look in). I realised that, if I could make a picture of the service
times that might be interesting (I can't do anything without pictures).
So I did that, via my standard trick of a tiny awk (or perl) front-end
to convert things from line-based into sexp-based, and then some Lisp
code to spit out something gnuplot could print.
And indeed it was interesting. Generally the service times were a
hundredth of a second or so, but there were some which were tens of
*minutes*. So, some more playing based on the original code and I have
a list of the offending services, times, and PIDs for things taking
more than some number of times the median time. With suitable settings
this give a few tens of calls which I can now go and look at the
detailed logs for to work out what was going on.
None of this is special magic Lisp code: in fact there's really nothing
remarkable about it at all. I could have done it in Perl. Was Lisp
better than Perl? For me, yes, because I'm a better (less bad) Lisp
programmer than Perl programmer and the environment I have for Lisp is
better than the one I have for Perl. But significantly it wasn't
*worse*: nothing was difficult, and nothing got in the way, it just all
worked easily and well. It was just *ordinary*, and I'm coming to the
view that that's what you want: a language which is just there and does
what you want without some glamourous computational pyrotechnics.
Common Lisp actually turns out to be quite good at that.
namekuseijin
Feb 13, 2012, 9:38:22 AM2/13/12
to
cool, but you missed extra-points for implementing awk in CL... ;)
Paul Wallich
Feb 13, 2012, 5:43:50 PM2/13/12
to
On 2/12/12 5:05 PM, Tim Bradshaw wrote:
[...]
some log files in Lisp because the Perl execution time was up to tens of
minutes. Perhaps they were using naive algorithms (as one does when
groveling) but the Lisp execution times were in the tens of seconds.
paul
[...]
> None of this is special magic Lisp code: in fact there's really nothing
> remarkable about it at all. I could have done it in Perl. Was Lisp
> better than Perl? For me, yes, because I'm a better (less bad) Lisp
> programmer than Perl programmer and the environment I have for Lisp is
> better than the one I have for Perl. But significantly it wasn't
> *worse*: nothing was difficult, and nothing got in the way, it just all
> worked easily and well. It was just *ordinary*, and I'm coming to the
> view that that's what you want: a language which is just there and does
> what you want without some glamourous computational pyrotechnics. Common
> Lisp actually turns out to be quite good at that.
What's the performance? Long ago, some friends brought me in to grovel
> remarkable about it at all. I could have done it in Perl. Was Lisp
> better than Perl? For me, yes, because I'm a better (less bad) Lisp
> programmer than Perl programmer and the environment I have for Lisp is
> better than the one I have for Perl. But significantly it wasn't
> *worse*: nothing was difficult, and nothing got in the way, it just all
> worked easily and well. It was just *ordinary*, and I'm coming to the
> view that that's what you want: a language which is just there and does
> what you want without some glamourous computational pyrotechnics. Common
> Lisp actually turns out to be quite good at that.
some log files in Lisp because the Perl execution time was up to tens of
minutes. Perhaps they were using naive algorithms (as one does when
groveling) but the Lisp execution times were in the tens of seconds.
paul
Tim Bradshaw
Feb 13, 2012, 6:35:32 PM2/13/12
to
Paul Wallich <p...@panix.com> wrote:
> What's the performance? Long ago, some friends brought me in to grovel
> some log files in Lisp because the Perl execution time was up to tens of
> minutes. Perhaps they were using naive algorithms (as one does when
> groveling) but the Lisp execution times were in the tens of seconds.
>
A few seconds, but this is only a day's worth of timing data.
> What's the performance? Long ago, some friends brought me in to grovel
> some log files in Lisp because the Perl execution time was up to tens of
> minutes. Perhaps they were using naive algorithms (as one does when
> groveling) but the Lisp execution times were in the tens of seconds.
>
Michael Moeller
Feb 14, 2012, 9:28:48 AM2/14/12
to
Am 02/12/2012 11:05 PM, schrieb Tim Bradshaw:
> in). I realised that, if I could make a picture of the service times
> that might be interesting (I can't do anything without pictures). So I
> did that, via my standard trick of a tiny awk (or perl) front-end to
> convert things from line-based into sexp-based, and then some Lisp code
> to spit out something gnuplot could print.
>
I agree. In my experience 'visualization' always was, and still is, the
> in). I realised that, if I could make a picture of the service times
> that might be interesting (I can't do anything without pictures). So I
> did that, via my standard trick of a tiny awk (or perl) front-end to
> convert things from line-based into sexp-based, and then some Lisp code
> to spit out something gnuplot could print.
>
method of choice. In a science lab for example one would be completely
lost without it. Ways and means to carry this out are without number.
If one likes doing things in Lisp, CLTL2 offers an extended example on
how to create PostScript code. It needs some minor adjustments, at
least with the Lisps I've tried then, but it might pay off, depending
on the problem given.
Regards,
Michael
Tim Bradshaw
Feb 14, 2012, 11:53:32 AM2/14/12
to
On 2012-02-14 14:28:48 +0000, Michael Moeller said:
> I agree. In my experience 'visualization' always was, and still is, the
> method of choice.
I used to think this, but I now think that it depends on who you are:
> I agree. In my experience 'visualization' always was, and still is, the
> method of choice.
there do seem to be genuinely non-visual people. For a very long
time[*] I used to just assume that these people were some kind of alien
or just not smart, but I think I was entirely wrong about that.
--tim
[*] Actually, not really a very long time by the standards of how old I
now am, if you see what I mean.
Michael Moeller
Feb 14, 2012, 6:29:54 PM2/14/12
to
infancy. Or did you mean 'unwilling' to mess about with graphics stuff?
Regards,
Michael
Alex Mizrahi
Feb 15, 2012, 6:18:23 AM2/15/12
to
>> I used to think this, but I now think that it depends on who you are:
>> there do seem to be genuinely non-visual people. For a very long time[*]
>> I used to just assume that these people were some kind of alien or just
>> not smart, but I think I was entirely wrong about that.
>> there do seem to be genuinely non-visual people. For a very long time[*]
>> I used to just assume that these people were some kind of alien or just
>> not smart, but I think I was entirely wrong about that.
>> [*] Actually, not really a very long time by the standards of how old I
>> now am, if you see what I mean.
> Genuinely non-visual? To me this is a developmental disorder during
> infancy. Or did you mean 'unwilling' to mess about with graphics stuff?
Some people are good at handling speech and textual information, that
>> now am, if you see what I mean.
> Genuinely non-visual? To me this is a developmental disorder during
> infancy. Or did you mean 'unwilling' to mess about with graphics stuff?
is, they can just read it and it makes sense for them without
intermediate representation in a visual form. I'd say it is a strength,
not disorder.
Tim Bradshaw
Feb 15, 2012, 7:46:06 AM2/15/12
to
On 2012-02-14 23:29:54 +0000, Michael Moeller said:
> enuinely non-visual? To me this is a developmental disorder during
> infancy. Or did you mean 'unwilling' to mess about with graphics stuff?
I meant "don't think in pictures".
> enuinely non-visual? To me this is a developmental disorder during
> infancy. Or did you mean 'unwilling' to mess about with graphics stuff?
Tim Bradshaw
Feb 15, 2012, 7:46:54 AM2/15/12
to
On 2012-02-15 11:18:23 +0000, Alex Mizrahi said:
> I'd say it is a strength, not disorder.
Definitely not a disorder. A difference rather than a strength though.
> I'd say it is a strength, not disorder.
Michael Moeller
Feb 15, 2012, 9:03:27 AM2/15/12
to
choose whatever method is most appropriate. Admittedly a matter
of taste in many cases. If a data set is small enough to get
evaluated by simply reading through it we have no problem at all.
Of course there are ways and means to handle data other than by
visualization. For this reason industrial strength statistics
applications offer both methods.
Regards,
Michael
Tim Bradshaw
Feb 15, 2012, 9:27:28 AM2/15/12
to
On 2012-02-15 14:03:27 +0000, Michael Moeller said:
> Admittedly a matter
> of taste in many cases.
Well, my point was that no, it doesn't seem to be. It seems instead to
> Admittedly a matter
> of taste in many cases.
be a matter of differing capacities for various styles of thinking.
Michael Moeller
Feb 15, 2012, 12:05:30 PM2/15/12
to
deliberately changes the style of thinking according to the problem
given. Sooner or later anything else will probably raise attention.
At least mine. This may depend on how close you look at the phenomenon.
Regards,
Michael
Tim Bradshaw
Feb 15, 2012, 12:31:09 PM2/15/12
to
On 2012-02-15 17:05:30 +0000, Michael Moeller said:
> People are of course more or less visually oriented but normally one
> deliberately changes the style of thinking according to the problem
> given. Sooner or later anything else will probably raise attention.
> At least mine. This may depend on how close you look at the phenomenon.
I can't get a parse of the second part of this. However my point was
> People are of course more or less visually oriented but normally one
> deliberately changes the style of thinking according to the problem
> given. Sooner or later anything else will probably raise attention.
> At least mine. This may depend on how close you look at the phenomenon.
really that the differences appear to be really quite significant. I'm
a long way at the visual end of things, and I find, for instance,
programming in an environment which does not have good support for
visualising programs (which is not pretty colours in the source code)
sufficiently unpleasant that I don't really bother to do it any more
unless someone's paying me to. There aren't many such environments I
know of: I suspect most programmers are not very visual because of that
fact.
Anyway, I've probably said more than is useful here.
Michael Moeller
Feb 15, 2012, 1:37:02 PM2/15/12
to
Am 02/15/2012 06:31 PM, schrieb Tim Bradshaw:
> On 2012-02-15 17:05:30 +0000, Michael Moeller said:
>
>> People are of course more or less visually oriented but normally one
>> deliberately changes the style of thinking according to the problem
>> given. Sooner or later anything else will probably raise attention.
>> At least mine. This may depend on how close you look at the phenomenon.
>
> I can't get a parse of the second part of this.However my point was
> On 2012-02-15 17:05:30 +0000, Michael Moeller said:
>
>> People are of course more or less visually oriented but normally one
>> deliberately changes the style of thinking according to the problem
>> given. Sooner or later anything else will probably raise attention.
>> At least mine. This may depend on how close you look at the phenomenon.
>
> really that the differences appear to be really quite significant. I'm a
> long way at the visual end of things, and I find, for instance,
> programming in an environment which does not have good support for
> visualising programs (which is not pretty colours in the source code)
> sufficiently unpleasant that I don't really bother to do it any more
> unless someone's paying me to. There aren't many such environments I
> know of: I suspect most programmers are not very visual because of that
> fact.
>
> Anyway, I've probably said more than is useful here.
>
That's a new aspect. I don't see any connection between writing code and
> long way at the visual end of things, and I find, for instance,
> programming in an environment which does not have good support for
> visualising programs (which is not pretty colours in the source code)
> sufficiently unpleasant that I don't really bother to do it any more
> unless someone's paying me to. There aren't many such environments I
> know of: I suspect most programmers are not very visual because of that
> fact.
>
> Anyway, I've probably said more than is useful here.
>
visualization. Anyway, in an environment such as industrial research and
engineering a rather non-visual person will have a very
hard time.
Regards,
Michael
Paul Rubin
Mar 2, 2012, 5:06:18 AM3/2/12
to
Tim Bradshaw <t...@tfeb.org> writes:
> big log file to look in). I realised that, if I could make a picture
> of the service times that might be interesting (I can't do anything
You might like this:
> big log file to look in). I realised that, if I could make a picture
> of the service times that might be interesting (I can't do anything
http://www.haskell.org/haskellwiki/Timeplot
The related splot (linked from there) might also be of interest.
0 new messages