I want to get the UK Ingres user association to do more to promote community
development projects. In my new role as acting chairman I've asked Marty
Bowes to be our "Community Projects Czar". I'm really taken with your
suggestion above because most of us have day jobs or other limitations that
would prevent us taking on a major project but as you point out there is
probably lots that we could do for our own amusement or convenience, and in
principle it could relieve other developers.
There are a number of things that stand in the way at the moment. I am also
curious to know how Ingres Corp "passively killed" your first attempt. Can
you explain in more detail what you tried and why you think it didn't work?
Roy
(Acting IUA chairman)
So far, the main problem seems to be the lack of resources on
the Ingres Corp end, perhaps combined with a fair bit of inertia.
Datallegro has attempted to contribute a fair number of fixes and
enhancements to Ingres. (That doesn't just mean me, either!) Some
of the fixes have made it into the mainline, after various fits and
false starts. For pure bug fixes we've been trying a new procedure
that has been working better: namely, sending the specific fix
to the appropriate person in Level 2. I don't know how well that
would translate to an Ingres Janitor feed, but it might.
If Ingres Corp can't pry the time loose, there is some slight
chance that I might be willing to act as a gatekeeper, at least
initially while volumes are low. We'd need some sort of public
repository and I'm not sure how to work that. It would be
better for changes to flow directly into Ingres Corp, but they
need to get themselves un-stuck before that will work.
Karl
Seems appropriate, especially since he tackled the compilation of INGRES
and stuck with it despite little substantive help from Ingres Corp.
I'm really taken with your
> suggestion above because most of us have day jobs or other limitations that
> would prevent us taking on a major project but as you point out there is
> probably lots that we could do for our own amusement or convenience, and in
> principle it could relieve other developers.
Yes, and I think it would serve to spawn a deeper level of expertise of
existing INGRES users as they are exposed to the information garnered
and published as a result of the effort of the Janitors and the Janitor
Mentors.
> There are a number of things that stand in the way at the moment. I am also
> curious to know how Ingres Corp "passively killed" your first attempt. Can
> you explain in more detail what you tried and why you think it didn't work?
Ingres Corp. uses some nifty tool, for which I've forgotten the name, to
parse the INGRES source code to look for issues, e.g. orphaned
structures, syntax errors, compilation warnings, etc. The idea was to
take subsets of the report produced by said nifty tool and attempt to
tackle the "simple" issues. Myself and another non-Ingres Corp. member
were going to take first pass at this and work out the details for
receiving bugs, coding a fix, testing said fix, and then submitting the
fix to Ingres Corp. for integration into the main code base.
There were concerns by Ingres Corp. that exposing these bugs publicly
would perhaps encourage their exploitation by crackers, damaging the
reputation of INGRES technology, and consequently, Ingres Corp.
Opinions on how to minimize this risk were voiced but an official
decision of how Ingres Corp. would be involved in facilitating this core
INGRES code bug list was never made, either for or against the approach.
Additionally, even enhancing some of the Open Source connectivity tools,
e.g. the Perl INGRES DBI, is difficult because traversing the INGRES
code base without being able to reference its specific layout or
consulting a knowledgeable party is frightening. Karl has done a nice
job in a presentation of providing a "Gentle Introduction" or "101"
discussion of how to approach the INGRES code base but as far as I'm
aware, this is the *only* thing of its kind that's publicly available.
Tackling Perl DBI enhancements is one type of challenge where we need an
idea of where we should be looking to determine how INGRES performs X
function; fixing trivial or non-critical bugs in the INGRES code base is
another thing entirely, both from a standpoint of general complexity,
and perhaps proximally, how to identify such bugs to begin to attempt to
fix them.
I think it's a problem with a resolution, and although it's great to see
Karl volunteering to assist, I don't think it should be on Karl's, or
your shoulders, to introduce the INGRES code base to the Open Source
Community, or to foster the Community's involvement. There will be some
bridge that will require the knowledge of INGRES Engineering to cross
and it would nice if Ingres Corp. would acknowledge that A) this is
necessary and B) they aren't the only ones with the skills to bug-fix or
even enhance INGRES technology.
If we had the likes of Karl, you, and the others (who know who they are)
providing a list of bug fixes and/or enhancements to someone like Marty,
who in-turn was distributing said tasks to persons such as myself, we
might all be surprised at how much we're able to accomplish, and how
quickly we gain the inertia required to refine the process and add new
members to the project, promote existing Janitors to Senior Janitors,
Junior Mentors, and so on, and so forth.
But perhaps I'm being too idealistic :)
> Seems appropriate, especially since he tackled the compilation of
INGRES
> and stuck with it despite little substantive help from Ingres Corp.
Actually I had a fair deal of help offline from quite a few people.
Nonetheless it was an 'adventure'. But don't get the idea I'm some C
guru. I'm way short of Yoda on that front, hell on some days I even
forget how to spell 'C'.
But in regards to the rest of Marks comments, I think he's totally hit
the nail on the head. In particular...
> Additionally, even enhancing some of the Open Source connectivity
tools,
> e.g. the Perl INGRES DBI, is difficult because traversing the INGRES
> code base without being able to reference its specific layout or
> consulting a knowledgeable party is frightening.
> Karl has done a nice
> job in a presentation of providing a "Gentle Introduction" or "101"
> discussion of how to approach the INGRES code base but as far as I'm
> aware, this is the *only* thing of its kind that's publicly available.
Yeah, there just has to be a better way than:
cd $ING_SRC; find . -type 'f' -name '*.c' | xargs -n1 -t grep 'whatever'
>& fred
And then weeding the output looking for some relevant bit of code that
may then give you a clue as to what the hell is going on and how it all
interacts.
Karl was kind enough to point me to a some documentation which covered
installing a new normal function. So rand_seq_gen() - my standard
version of 'hello world' in function format - was born. Fun as this
example was, getting the code of such a beast back into the standard
base code is another matter alltogethor.
As an example, I'm trying to make a new aggregate function
percentile_cont(). But this has proven much harder than intially
anticipated as the full horror of aggregate functions, ADF_AG_STRUCT,
ADF_AG_OLAP etc become apparent. Can I alter the ADF_AG_STRUCT to get my
function some extra workspace - what will be the ramifications of this?
Having altered the structure, find the thing that initialises it. Having
done that, convince adecompile.c that it can pass two arguments to an
aggregate function that isn't using OLAP - but am I allowed to do that?
Finally, I have to get the sql syntax checker to accept it must order
the data coming into the function even though the data itself is not
part of the select statement. I'm not even sure where to start looking
for that bit yet...
I get the feeling that the code required to do all of this will be
fairly extensive. And lets say it all seems to hang togethor when I
build it on Linux, x86_64. Whose going to test it on windows, Solaris,
VMS?
By the time little old me finally gets something to hang togethor, how
far out of date will my code changes be? I dragged the Opensource into
my own CVS repository. It could be amusing to compare that with what is
currently in place.
It comes down to how best to get fixes and additions into the code
stream. Fixes generally happen fairly well, but I've had very limited
success with Enhancement Requests over the years, and part of my
thinking is how to improve the speed with which something can be added
to the code set. I'm not sure how to write a 'Business Proposal' to
justify a new function being created! But have I got a better chance
just writing the code and handing that over? Don't know, but it may be a
lot of fun finding out!
Marty
I think that anyone who considers themselves even half-way decent with
INGRES, whether it be the DBMS, ABF (and Co.), and/or OpenROAD, has had
"offline help from quite a few people." The problem is that I think the
information winds up being repeated, repeatedly :), by a select few who
have plenty of generosity (and thanks for that) but barely any time to
tackle their own interests, let alone formulate their ideas into a
knowledge base, or the like.
I'm not so sure that any of us Janitors have to be impressive at X (in
which case, we'd be a Mentor), so much as we need to make the
information that we've all received publicly available in a single
location, suited to the task, so that those more expert at X can assist
those less expert, who will be expert at Y, and do likewise.
Your (our) dilemma is larger than just a repository, and you've
articulated it nicely with a specific example, but I wonder if we could
host the INGRES project on SourceForge? We'd be putting the released
source tree on SourceForge and then referencing changes against that
tree, which is public. You'll notice that INGRES *is* actually hosted
on SourceForge, and yet it has *0* files in the CVS :(
These <http://sourceforge.net/projects/webmin-ingres/>
<http://sourceforge.net/projects/ivdw/>
<http://sourceforge.net/projects/ingres-bind/> are interesting, and
useful, INGRES projects, which all appear to be driven by the same
primary person, Hervé Leclerc (and Jean-Pierre Zuate for the "Ingres
Visual DBA Web" project). Jean-Pierre's name I recognize from this
group, but not Hervé.
Do these guys have the "secret files" or are they just incredibly
enterprising? :)
> It comes down to how best to get fixes and additions into the code
> stream. [snip]
I think your experience with building INGRES and creating OME functions
are excellent beginnings. As an example of a potential INGRES Janitors
Scenario:
---
Karl or Roy deem Bug Y both worthy of effort and within our current
skill bounds, which provides a somewhat specific direction in the maze
of INGRES source. When we hit specific snags we ask specific questions,
receive the needed guidance, and then return to attempting to resolve
the issue.
---
After an initial cycle of as many iterations as it takes to fix Bug Y,
we'd be better at a lot of related things (C, parsing the INGRES source,
finding callers, using C scope, etc.) which will help us more
efficiently frame the process for Bug N, and so on, and so forth.
It doesn't appear that Ingres Corp. Engineering is releasing INGRES
updates at break-neck speed, so although we'd be behind their schedule
for some time, I think we could eventually have something attractive,
even if it's "only" a 'diff'able set of cool functions, like Statistical
Time Series Analysis (such as the STARPAC functions rewritten in C?), or
a comprehensive set of guides about INGRES internals available to the
public that keeps current with the latest GPL'd INGRES source release.
--
Mark R. Winston
www.datavailable.com
This concept is easy to pursue and I've recently posted about it. The wiki
is there and because it is a wiki we don't need any say-so from anyone about
our contributions. Mike Sale has asked that anyone interested contact him
to be put on the list of contributors. I take his word that this is a
temporary situation pending the setting up of a proper anti-spammer
handshaking process. If you don't want to contact
Michael.Sale@you-know-where, contact me at rhann at rationalcommerce dot com
and I will pass on the request.
> Your (our) dilemma is larger than just a repository, and you've
> articulated it nicely with a specific example, but I wonder if we could
> host the INGRES project on SourceForge? We'd be putting the released
> source tree on SourceForge and then referencing changes against that
> tree, which is public. You'll notice that INGRES *is* actually hosted
> on SourceForge, and yet it has *0* files in the CVS :(
>
> These <http://sourceforge.net/projects/webmin-ingres/>
> <http://sourceforge.net/projects/ivdw/>
> <http://sourceforge.net/projects/ingres-bind/> are interesting, and
> useful, INGRES projects, which all appear to be driven by the same
> primary person, Hervé Leclerc (and Jean-Pierre Zuate for the "Ingres
> Visual DBA Web" project). Jean-Pierre's name I recognize from this
> group, but not Hervé.
>
> Do these guys have the "secret files" or are they just incredibly
> enterprising? :)
>
>> It comes down to how best to get fixes and additions into the code
>> stream. [snip]
Emma McGrattan's current blog (http://blogs.ingres.com/emmamcgrattan) is
agitating for more community development. I don't think there's any divide
here. The priorities and the resources and the enthusiasms may not all have
come to together on the first attempt (or even the first few), but I don't
think anyone is opposing community projects in any way. My instinct is that
it is just a matter of organizing to get it done. As I have said, I am
asking the IUA to take more of a role in this. (Feel free to use
comp.databases.ingres or info-ingres to communicate with me on this subject;
that is what it is for.)
[snip]
> Karl or Roy deem Bug Y both worthy of effort and within our current
> skill bounds [...]
Karl for sure, if he wants to be involved. But I am not qualified (or
indeed interested) in the source code. My interests are somewhere else
entirely.
And remember, Karl is a relative late-comer. There are lots of other people
who can (and occasionally do) help out. It would be great to hear from them
more, and from some of their even more shy colleagues.
Roy
Having more of my colleagues respond here would be a good thing for the
company and as a result the product. It would be a good showcase for the
support that we as a company can provide. I am sure that those of you
who work as an external consultant have gleaned new business as a result
of your knowledge shown here - if it works for you it will work for us too.
my 0.02€
grant
p.s. these are my own opinions etc... please test before going live....
--
Grant Croker - Software Engineer - http://ingres.com
It was hard enough to kill a vampire. You could stake them down and turn
them into dust and ten years later someone drops a drop of blood in the
wrong place and *guess who's back*? They returned more times than raw
broccoli.
-- (Terry Pratchett, Feet of Clay)
You and Paul Mason didn't seem to get that message too well. :-) And thank
goodness for that.
Roy
This topic is near and dear to my heart for many reasons. I've been
the fellow integrating the vast majority of Karl's updates, and I'm
also the guy who introduced Klocwork (aka. nifty bug finding tool) to
Ingres.
One of the current things I'm working on is a way to make it easier to
contribute to Ingres. Some of you have been in touch with me about
this.
We are looking at the code repository, email lists, bug tracking, and
other factors. I can't promise miracles daily but I can promise we'll
head in the right direction, listen to good ideas, and make things
progressively better. In the meantime, we're doing a lot of the
legwork to accept patches from contributors and integrate them through
email. Be assured we're motivated to improve this.
I have bugs found by Klocwork that need review and fixing. If you are
interested in working with us, please contact me. I am
andrew.ross@youknowwhere. I'm happy to carve off a few for you to look
at and very grateful for any contributions.
Thank you,
Andrew
Oh this is friggen great mate! That's exactly what I want to hear! So
I'm definitly up for working with you on this.
I'm particularly interested in the mechanics of getting a bug or
enhancement request through the system from start to end. For example if
I gave you a diff for the file(s) that I thought fixed a problem, or
created a new feature, how would you propose to test and finally
incorporate them into the code stream?
At the moment I'm playing with fixing the tups_per_page problem
mentioned earlier on this thread. I should have a solution fairly
shortly. If I just email to you the diff files would you find that
useful?
Martin Bowes
--
Random Farscape Quote #11:
John - Hey Astro-Boy. Fix now, freak later.
Stark - How much later?
Hi All,
Thank you,
Andrew
_______________________________________________
Info-Ingres mailing list
Info-...@kettleriverconsulting.com
http://www.kettleriverconsulting.com/mailman/listinfo/info-ingres
First, to save you time and effort, please send me a brief summary of
what you're planning to do, the files you're going to affect, and how
you intend to test it. I'll review it, depending on what it is, and my
familiarity with that area, I may enlist the help of my esteemed
colleague who has expertise in the area, and give you feedback. The
idea is that I want to be able to tell you up front whether your
approach is correct. If we don't like the approach, we'll give you
guidance on an approach we'd prefer.
Once you're past this up front sanity check, you can develop the
update & test it. Once you're done, please send me the patch (created
with diff) via. email (for now) along with any updated notes on what
it does and tests to check it. I'll take it from there to review it
and integrate it into our main codeline (latest & greatest). Hopefully
because of the up front proposal review, it shouldn't be rejected at
that point.
Thank you for your interest and in advance for your update Martin, we
really appreciate it.
Andrew
On Nov 2, 6:21 am, "Martin Bowes" <martin.bo...@ctsu.ox.ac.uk> wrote:
> Hi Andrew,
>
> Oh this is friggen great mate! That's exactly what I want to hear! So
> I'm definitly up for working with you on this.
>
> I'm particularly interested in the mechanics of getting a bug or
> enhancement request through the system from start to end. For example if
> I gave you a diff for the file(s) that I thought fixed a problem, or
> created a new feature, how would you propose to test and finally
> incorporate them into the code stream?
>
> At the moment I'm playing with fixing the tups_per_page problem
> mentioned earlier on this thread. I should have a solution fairly
> shortly. If I just email to you the diff files would you find that
> useful?
>
> Martin Bowes
>
> --
>
> Random Farscape Quote #11:
> John - Hey Astro-Boy. Fix now, freak later.
> Stark - How much later?
>
> -----Original Message-----
> From: info-ingres-boun...@kettleriverconsulting.com
>
> [mailto:info-ingres-boun...@kettleriverconsulting.com] On Behalf Of
> g...@rogers.com
> Sent: 02 November 2007 01:36
> To: info-ing...@kettleriverconsulting.com
> Subject: Re: [Info-Ingres] Ingres Janitors
>
> Hi All,
>
> This topic is near and dear to my heart for many reasons. I've been
> the fellow integrating the vast majority of Karl's updates, and I'm
> also the guy who introduced Klocwork (aka. nifty bug finding tool) to
> Ingres.
>
> One of the current things I'm working on is a way to make it easier to
> contribute to Ingres. Some of you have been in touch with me about
> this.
>
> We are looking at the code repository, email lists, bug tracking, and
> other factors. I can't promise miracles daily but I can promise we'll
> head in the right direction, listen to good ideas, and make things
> progressively better. In the meantime, we're doing a lot of the
> legwork to accept patches from contributors and integrate them through
> email. Be assured we're motivated to improve this.
>
> I have bugs found by Klocwork that need review and fixing. If you are
> interested in working with us, please contact me. I am
> andrew.ross@youknowwhere. I'm happy to carve off a few for you to look
> at and very grateful for any contributions.
>
> Thank you,
>
> Andrew
>
> _______________________________________________
> Info-Ingres mailing list
> Info-Ing...@kettleriverconsulting.comhttp://www.kettleriverconsulting.com/mailman/listinfo/info-ingres
> FYI
>
> http://community.ingres.com/wiki/Ingres_Janitors_Project
I think your cultural awareness consultants have let you down. Our Irish
readers (Emma for sure) will be offended by the pronunciation guide for FEQ.
This link to the photo of the famous Irish scarecrow will explan the
problem. (Korea? Why??)
http://www.seoulshamrock.co.kr/Index_eng.htm
Roy
A suitably, subtly irreverent way to begin the project, in my opinion :)
Hooray!
Mark R. Winston
www.datavailable.com
> http://www.seoulshamrock.co.kr/Index_eng.htm
I botched that. :-(
This works better:
http://www.tcm.phy.cam.ac.uk/~mdt26/photos/irish_scarecrow.jpg
Roy
OK, a quick blast from the past.
I used to build patches for a particular code line for a manufacturer (or
add a new machine code line altogether). The code line I worked upon was a
branch from the main code line that was (at that time) based on sun4.
At that time I would hunt down bugs against the supplied test cases and
change files as I went along. The really important thing was to update the
file header with a comment about the change (as well as inline comments
where the changes were made). Effectively my patch would be a list of the
files changed in fixing the bug. The split codeline would have it's own
version control.
Patches were made on that basis and basic testing would be done for a quick
patch fix, for full blown releases the whole test suite and comparison of
test cases would be invoked and differences from expected output checked.
In our way of working file differences weren't the prefered way of notifying
the patches, since the code line I worked against was split from the main
code line which was changing independently of me as other changes were added
to it.
The point is that even if you supply before and after changes, for the work
you've done, the chances are that the underlying code you started with will
already have changed in the main code line, so your differences will no
longer be correct.
I think my changes were re-applied to the code by diffing the main codeline
files against my changed files and the important things for a given change
are the list of change files, plus good comments and most importantly
comments in the headers so that you can read the history of changes within
the file back .
I was curious if my changes still exist inside the current code line - and
they do! You can only see them as comments in the headers (pauland), as the
codeline Guru SWM, applied my branch changes back into the main codeline.
Not sure if that is at all relevant, but that what I did way back when .
Sometimes it was real fun to do. Other times it was as boring as can be!
Paul
I strongly recommend that anyone sending in a diff use the unified
diff format, diff -u. The patch program is very good about applying
unified diffs correctly, even when the two originals are out of sync.
>I was curious if my changes still exist inside the current code line - and
>they do! You can only see them as comments in the headers (pauland), as the
>codeline Guru SWM, applied my branch changes back into the main codeline.
I think it would be fun to start a wiki page with all of the
known contributors to Ingres, and their initials in the code.
I know most of the current ones, of course, but every now and
then I see a "joeq" or something in the code and I wonder
who the devil that was. Maybe I'll create a wiki page for
this later today.
(So, who was SWM???)
Karl