Contributing?
Vivian Landers
contributing to an open-source OCR project. Tesseract seems like one
of the best ones currently available, and I like its modular design.
Is there a to-do list anywhere of things that Tesseract still needs?
What research is it based on, and what existing research might
profitably be incorporated into it? What practical work needs to be
done on things like the build system, installer, code refactoring,
documentation, and so on? And how do I get permission to make
contributions? Thanks a lot for your help!
-Vivian
74yrs old
also
Ted Rolle
Ted
Jussi Pakkanen
> profitably be incorporated into it? What practical work needs to be
> done on things like the build system, installer, code refactoring,
> documentation, and so on?
Here are some things which I have considered while looking at the code:
Documenting how the character matcher actually works. AFAICR the
system does very convoluted things before calling the matcher. Stuff
like converting extracted TBLOBs to PBLOBs and back again and calling
matching functions indirectly through function pointers and so on.
Tesseract passes some state variables as function parameters but some
things are in global variables that are scattered all over the code
base. This makes Tesseract very non-threadsafe and difficult to
decipher. Wrapping all global variables in a struct and passing it
along with the functions or something like that would be beneficial.
Like all C programs from the early 90s, Tesseract has its own
implementation of standard containers such as a linked list. (In fact,
it has several implementations which all work slightly differently.)
These are done with preprocessor macros. See for example elst.h.
Converting to STL containers would be very beneficial.
Tesseract has two different build systems: autotools for Unix and
Visual studio project files for Windows. It would make more sense to
have just one build system using CMake, since it supports all these
platforms (plus a bunch of others such as XCode) natively.
> And how do I get permission to make contributions? Thanks a lot for your help!
Usually posting patches to the bug tracker or this list is sufficient.
Ted Rolle
spaghetti code is particularly deserving.
Ted
74yrs old
Ted Rolle
Vivian Landers
On Jan 16, 2:17 am, "Jussi Pakkanen" <jpakk...@gmail.com> wrote:
> Tesseract passes some state variables as function parameters but some
> things are in global variables that are scattered all over the code
> base. This makes Tesseract very non-threadsafe and difficult to
> decipher. Wrapping all global variables in a struct and passing it
> along with the functions or something like that would be beneficial.
>
> Like all C programs from the early 90s, Tesseract has its own
> implementation of standard containers such as a linked list. (In fact,
> it has several implementations which all work slightly differently.)
> These are done with preprocessor macros. See for example elst.h.
> Converting to STL containers would be very beneficial.
help me ramp up on the code base.
> Tesseract has two different build systems: autotools for Unix and
> Visual studio project files for Windows. It would make more sense to
> have just one build system using CMake, since it supports all these
> platforms (plus a bunch of others such as XCode) natively.
system to new platforms this would be a good first step.
> Documenting how the character matcher actually works. AFAICR the
> system does very convoluted things before calling the matcher. Stuff
> like converting extracted TBLOBs to PBLOBs and back again and calling
> matching functions indirectly through function pointers and so on.
base, but it's a good suggestion.
> > And how do I get permission to make contributions?
>
primary authors if they're available.
-Vivian
Ted Rolle
Vivian, you are so enthusiastic! It's refreshing to see.
Grad school? Where? In what?
I believe theraysmith is still about. He's one of the "heavies" for tesseract -- if not the author.
Ray Smith
Thanks for volunteering. For the benefit of everyone who has expressed an interest in contributing, here is a summary of how I have been running the project so far:
In case you didn't know, we are actively developing (and using) Tesseract here at Google, and we are committed to putting our improvements back into the open source codebase. Changes we make at Google go into our own codebase, source control system etc first, and get tested very carefully, but on a limited set of platforms. As a consequence of the dual codebases, someone (usually me) has to merge changes on the two systems periodically, usually at a point of good stability, and test. The merge and release cycle has therefore turned out to be fairly complex and infrequent - at 2-4 months, but at each one, I try to incorporate as many patches that people have supplied and fix as many of the issues that people have reported as possible. Due to having several different responsibilities, I contribute to this forum on a best efforts basis, but I don't get a great deal of time to give to it.
For making small patches for this and that, this forum (and the issues list) are fine with me, but they won't generally get much attention until a release cycle.
If you have more time to contribute, I will add you as a developer to give you access to the subversion repository. I have a list of projects of varying sizes that I carry round in my head. I occasionally post comments on them to this forum, bat that makes them more difficult to find. I will comment on some of them now, and later post them to a new wiki page on the tesseract-ocr site.
Easy starter project:
1. Convert old variables to new. There are 2 implementations of tunable parameters (called "variables"), an old C relic, and a new C++ implementation. It would be nice to get all the old C-style variables ( variables.h) converted to new C++ style (varable.h). This would enable a tidying up of config files, and a tidying up of the initialization process.
Bigger, but still fairly basic:
2. Thread safety. As someone on this thread already mentioned, it would be useful to gather all the nasty globals together into a Tesseract class, and make the TessBaseAPI instance-based, with a view to making it possible to have two instances of Tesseract running together in the same process. This might be a lot harder than it looks on the surface, but does involve a lot of fairly basic grind.
3. An hOCR output converter might be useful to many people. A lot of people ask for bounding box information, and that might be a good standard way of doing it.
Longer term, harder & requiring deeper knowledge:
4. Get rid of the polygonal approximation. Recognition accuracy could be more accurate, probably with virtually no loss of speed, by getting rid of the polygonal approximation. It would be relatively easy to convert the character classifier, but we need a new chopper that works directly on the C_OUTLINEs. Inter-dependent with:
5. Cut the crap. The top-level code uses C++ classes to describe the outlines, blobs, words, etc, but the lower-level classifier code, and the top-level word classifier both use older C-style structs for the same purpose. While it would be nice to get rid of the old completely and use only the new, it would not really be desirable to do it, until we have a chopper that uses the un-approximated edge-step outlines, as then we could eliminate the polygonal approximation step, and the data structures that support them, completely.
6. More integration with ocropus would be useful. Allowing tesseract and ocropus to share the same training data is one possibility.
Things I would NOT recommend working on:
1. Someone suggested the macro-based list stuff as a candidate for replacement with stl. I will not be incorporating any such changes into the main codebase for 2 reasons:
a. We have been trying to keep stl out of tesseract for a specific reason that I probably shouldn't comment on.
b. Compared to the macro-based lists in tesseract, stl lists are very different, very incompatitble, and IMHO a poor abstraction designed to make them as like vectors as possible, and if you use them the way they are used in tesseract, it would be very slow. Although the rest of stl is very useful, reason (a) still keeps it out of the codebase. It might be possible to sensibly convert the macro-based lists to (mostly) use templates though.
2. Although it is very tempting to try to expand tesseract to new languages, if you did so, you would be overlapping significantly with the work going on at Google. Of course that leaves anyone that wants a different language in the difficult position of either waiting for it to be available, or trying to train it themselves. I will be in a much better position to discuss language compatibility after the next release, by which time there will be much more language support.
Ray.
Terve Jussi, thanks for your comments.
Jussi Pakkanen
> In case you didn't know, we are actively developing (and using) Tesseract
> here at Google, and we are committed to putting our improvements back into
> the open source codebase. Changes we make at Google go into our own
> codebase, source control system etc first, and get tested very carefully,
> but on a limited set of platforms.
This is not a very good way of organizing an open source project,
since it does not encourage participation from the community. The open
source mantra is "release early, release often" and this means SVN
development as well. It can be very discouraging for a new developer
to have their hard worked patches dropped because the work was already
done (but kept secret) or because they conflict with certain new
development (which, again, was kept a secret).
> As a consequence of the dual codebases,
> someone (usually me) has to merge changes on the two systems periodically,
> usually at a point of good stability, and test. The merge and release cycle
> has therefore turned out to be fairly complex and infrequent - at 2-4
> months, but at each one, I try to incorporate as many patches that people
> have supplied and fix as many of the issues that people have reported as
> possible.
This is pretty much the same problem as discussed here:
http://thedailywtf.com/Articles/Happy_Merge_Day!.aspx
The lesson: commit early, commit often, merge early, merge often. Not
doing that causes a ton of totally unnecessary work. It would be
beneficial in this case to have a revision control system that handles
merges and private branches better than SVN (that is to say: at all).
However I'm not willing to open that particular can of flameworms at
this time.
As a basis of discussion I propose the following way to handle this issue:
There are three different kinds of changes to Tesseract:
1) those that do not contain any Google Secret Information (TM) such
as bug fixes and general enhancements
2) those with Google Secret Information that will get released when
they are done
3) those with Google Secret Information that will never, ever get released
Type 1 code is the kind that should be pushed to public SVN as soon as
possible. Even better would be if they were developed in the public
SVN and only then imported to Google's internal branch.
Type 2 would be developed inside Google and then pushed to public SVN.
But again, as early as possible. I guess these would be added just
before releases as is currently the case.
Type 3 is Google's internal issue and thus not our problem.
This approach would try to satisfy the two slightly contradicting goals:
- public SVN is as up to date with project development as is possible,
enabling community participation
- Google gets to keep their secret bits secret, keeping corporate bean
counters happy
Vivian Landers
response didn't show up and may have been lost, so just posting again.
I didn't know before about Google's involvement with the project and
your integration schedule, and that info was helpful.
On Jan 18, 5:53 pm, "Ray Smith" <theraysm...@gmail.com> wrote:
> Easy starter project:
> 1. Convert old variables to new.
> 2. Thread safety.
the codebase. The other projects you listed also sound interesting but
I'd like to get some more experience with Tesseract before considering
tackling them.
> a. We have been trying to keep stl out of tesseract for a specific reason
> that I probably shouldn't comment on.
> b. Compared to the macro-based lists in tesseract, stl lists are very
> different, very incompatitble, and IMHO a poor abstraction designed to make
> them as like vectors as possible, and if you use them the way they are used
> in tesseract, it would be very slow. Although the rest of stl is very
> useful, reason (a) still keeps it out of the codebase. It might be possible
> to sensibly convert the macro-based lists to (mostly) use templates though.
I love killing macros and would be happy to introduce templates here,
with suitable testing to prevent performance regression.
> For making small patches for this and that, this forum (and the issues list)
> are fine with me, but they won't generally get much attention until a
> release cycle. If you have more time to contribute, I will add you as a developer to give
> you access to the subversion repository.
substantial time. It would be great if I could get SVN repository
access to help streamline contributions. Please let me know if this is
possible.
Thanks again for your feedback and I look forward to working with you.
-Vivian
>
> - Show quoted text -
