combining load failures with dependencies information to priorities fixes

[Anton Vodonosov]

"Juan Jose Garcia-Ripoll" <juanjose.g...@gmail.com> on ecls-list:

> Right now it would be very useful to sort the libraries by dependencies:
> those that other depends on and fail will likely cause failures on other
> libraries themselves.

Eventually, as I have tested load status of every library found in Quicklisp,
we can build such a report.

I initially though that "sort the libraries by dependencies" will be just
topological sorting, but it turned out to be not sufficiently illustrative.

The most fruitful systems to fix are those which:
- fails by themselves, i.e. don't have failing dependencies
- have many other systems depending on the given system
- in particular, these depending systems have the given system
as the only failed dependency (I call this blocked-exclusively)

For example, closer-mop fails to load on ECL, and there are
123 other ASDF systems (related to 59 projects), which depend
on closer-mop and closer-mop is the only failing dependency
for them (in my terms, closer-mop block them exclusively).

This means that fixing closer-mop is likely to fix 123 other ASDF
systems (unless these other systems have their own problems,
in this case we will at least reveal these problems).

Here is the report
http://common-lisp.net/project/cl-test-grid/ecl-load-failures.html

In the default sorting is
- number of root-blocker-systems desc,
- number projects of systems blocked exclusively asc
- system name desc.

I.e. the default sorting places the most fruitful systems at the top.

You may change sorting by clicking columns (holding Shift
to sort by multiple columns).

> Again, that should be programatically easy to do, given ASDF's tree
> of dependencies (or quicklisp's for that matter).

Note, there is no way (or at least non trivial) to extract the dependency
information 100% precise. ASDF systems are just lisp code,
sometimes they contain reader conditionals, sometimes
asdf:load-op invocations, instead of just putting the systems
into :depends-on or :defsystem-depend-on. But the information
we can gather is enough to make useful observations. I retrieve
the dependency information from
quicklisp\\dists\\quicklisp\\systems.txt index file.

Best regards,
- Anton

[Juan Jose Garcia-Ripoll]

Hi Anton,

thanks for the hard work on this issue. I believe it will be frutiful. Unfortunately I am quite overloaded with work myself and I am missing this and other email exchanges -- to be sincere I have arrived to it in the wrong order: Pascal's blog post, then Zach's and then this email. This now seems to explain the weird chain of reaction I noted this morning

On Tue, Sep 25, 2012 at 8:06 PM, Anton Vodonosov <avodo...@yandex.ru> wrote:

I initially though that "sort the libraries by dependencies" will be just
topological sorting, but it turned out to be not sufficiently illustrative.

The most fruitful systems to fix are those which:
- fails by themselves, i.e. don't have failing dependencies
- have many other systems depending on the given system
- in particular, these depending systems have the given system
  as the only failed dependency (I call this blocked-exclusively)

The result is a beautiful explanation of what is going on, indeed. I now read the table and I see the blocking libraries and understand Pascal's response to it. It is a pity that my response times and my confusion are inversely proportional to each other.

 

For example, closer-mop fails to load on ECL, and there are
123 other ASDF systems (related to 59 projects), which depend
on closer-mop and closer-mop is the only failing dependency
for them (in my terms, closer-mop block them exclusively).

Wow, I did not expect closer-mop to be such a must for so many libraries, otherwise I would have provided a set of patches together with the changes in ECL. It would have helped solve the bad publicity it is getting right now.

> Again, that should be programatically easy to do, given ASDF's tree
> of dependencies (or quicklisp's for that matter).

Note, there is no way (or at least non trivial) to extract the dependency
information 100% precise. ASDF systems are just lisp code,
sometimes they contain reader conditionals, sometimes
asdf:load-op invocations, instead of just putting the systems
into :depends-on or :defsystem-depend-on. But the information
we can gather is enough to make useful observations. I retrieve
the dependency information from
quicklisp\\dists\\quicklisp\\systems.txt index file.

Understood. This is indeed one of the big problems I pointed out some time ago and which nobody cares to solve.

Thanks again for the great work. I hope I can correspond by fixing ECL's problems ASA my sleep deprivation is cured.

Juanjo

--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain) 
http://juanjose.garciaripoll.googlepages.com

[Anton Vodonosov]

OK.

You are too self-critical, we know that you can only work on ECL
in spare time, still you do huge amount of work on ECL
and the support you provide on ecls-list is great, I think no message is left
unanswered. In particular the feedback you gave me about test-grid
lead to the creation of this load failures + dependencies report.

I also think that it's clear for everyone that ECL progress and improvement
may only be achieved if you have proper rest :) The information we collected
is not going to disappear, it will be there when you have time for it.

Best regards,
- Anton

[redli...@gmail.com]

Juanjo,

Just wanted to echo Anton. You do fantastic work on ECL and though I don't use it often I am glad it and you are a part of the CL community.

Regards,

Brit Butler

[Anton Vodonosov]

Xach, could you please arrange the following
clarification to be forwarded to Planet Lisp? It's necessary
in particular to restore/confirm good names of ECL and Closer to MOP?

When I posted the first message in this thread and
CC'ed Xach, I wanted to present the results
cl-test-grid may provide, but I forgot that the results
deal with failures, and without proper explanation
it may cast shadow of bad impression onto someone's
hard work.

In this case ECL and Closer to MOP were mentioned
in context of large table of failures.

First of all. The absolute minority of the compile-load
failures presented in the report are bugs in ECL (if any).

The failures may be classified by their cause into:
- CL implementation bugs
- library bugs
- library intentionally supports only limited
  set of CL implementations
- absence of certain foreign library
  on the test system.

By priority for the lisp implementation maintainer:

1. CL implementation bugs are of course of interest
   for the maintainer.

   Here I want to say how they are handled in ECL.
   Of course, ECL can not be bug free, especially
   taken into account that in the recent years it saw
   major changes (fully rewritten Lisp-to-C compiler,
   newly introduced bytecode compiler, refactoring
   of multi-threading infrastructure, performance
   improvements and lot of other improvements).

   But when critical bugs are found, they are usually
   fixed very quickly. This demonstrates the great level
   of control Juan Jose has on the ECL code base.

2. Bugs in basic widely used libraries as alexandria;
   portability layers like closer-mop, usocket,
   bordeaux-threads; important libraries as
   hunchentoot or drakma.

   This is not the area of direct "responsibility"
   for the CL implementation developer who didn't started
   the project and didn't make a commitment to
   support it.

   I put "responsibility" into quotes because very
   often we deal with open source projects, which
   are provided AS IS. No-one is obliged to provide
   support. The applies also to the library author.

   Of course, CL implementation developer might
   be interested to support such important libraries
   to achieve wider adoption of his CL implementation.
   The library author is also usually interested.
   And the users of the library, who want to develop
   applications on the given CL implementation are
   also interested to see the library fixed on this CL
   implementation.

   So, this is an area where interest of various parties
   overlaps and we may hope the issue will be fixed
   (depending the time available for these parties
   and presence of necessary knowledge).

   Speaking particularly about Closer to MOP on ECL.
   ECL has improved MOP support recently, but the API
   changes are not backward compatible. According
   to the Pascal's comment in his blog, now Closer to MOP
   will need a lot less code to support ECL. But migration to the
   new API will take some development time.

3. Compile/load failures which do not need to be fixed.

   For a library developer it is often the only possible
   or at least the most reasonable solution to support
   limited set of CL implementations, because supporting
   and testing on every CL requires more time than available.
   
   If there is a library named "my-html-utils" which is
   only used by "my-web-application" and the library
   author works with one particular CL implementation,
   there is no need to bother neither the library author,
   nor other community members with requests to port
   the library onto other CL implementations.

   What we can do is to collect and provide information
   about what works where, and the author will decide
   himself. Or maybe the first user, who whats to
   employ the library on a different implementation
   will port it.   

   I assume that the failures in the category 3 might
   constitute the majority of all the failures.

4. Absence of foreign library on the test system.
   I plan to adjust cl-test-grid agent in the future
   so that CFFI errors will be recognized and
   stored as a special status, so that we will be
   able to exclude them from report + maybe collect
   the list of foreign libraries used by whole Quicklisp.
   But it's not the highest priority, the reports
   in the current form allow to make lot of useful
   conclusions.

That's it. I hope it clarifies any doubts about the
quality of work done by ECL and Closer to MOP developers.

And also on my intentions. I don't call the developers
to spend even more of their time. Other way around.
We should anticipate the fact that developers time is
very limited. If we all be realistic and stay within
limits of possible the results will be more satisfying.
I hope that presence of information will allow to
save developers energy by helping them to choose
areas where the efforts will give maximum outcome;
to decide also what may be avoided.

The load failures + dependencies report was build for
ECL first because it was Juan Jose's proposal to
combine them together to prioritize problems.

I've just published similar reports for some
other CL implementations I have:

http://common-lisp.net/project/cl-test-grid/abcl-load-failures.html
http://common-lisp.net/project/cl-test-grid/acl-load-failures.html
http://common-lisp.net/project/cl-test-grid/ccl-load-failures.html
http://common-lisp.net/project/cl-test-grid/cmucl-load-failures.html
http://common-lisp.net/project/cl-test-grid/sbcl-load-failures.html
http://common-lisp.net/project/cl-test-grid/ecl-load-failures.html


Best regards,
- Anton