Status of Scalpel and helping.
Germán Diago
implementation (like this one) in C++
of a source code analysis tool. Congratulations for the project, it
looks promising.
I have read all the documentation, compiled the source code, played a
little, and inspected myself
the source code. It looks very organized, but it's quite big. I would
like to answer some questions
about the project:
1.- Which is the status of the project? I mean, there are no commits
since September. Is anyone working
on it?
2.- Which would be the best way to help to the project (I'm mainly
interested in writing source code to
add features).
3.- If I contribute source code, could you guide me in:
- How should I write the tests?
- Which is your reference to write the analyzers (I suppose it's
the standard commitee paper).
- Is the source code exactly (or almost) organized as the working
paper for c++03? I mean if, for example,
where it says declarator-seq, it's the same identifier in the
working paper and so on.
Thanks for your time.
Germán Diago
> about the project:
Sorry, it's obvious, I mean "like to ask".
Florian Goujeon
First of all, thank you for your interest in the project.
I'm currently thinking about the future of Scalpel.
It seems like Clang is a similar project against whom I'm unable to compete.
However, some people have encouraged me not to give up, arguing that
Scalpel could be a good alternative for at least some cases.
Before I answer to your questions, I would like to know whether you
considered using Clang as a C++ source code analyzer, and if so, why you
discarded it.
german diago
> Before I answer to your questions, I would like to know whether you
> considered using Clang as a C++ source code analyzer, and if so, why you
> discarded it.
which nowadays, is
more feature-complete for c++0x). Besides that,
I think it's far more portable for IDEs to have an analyzer as an
independent
library that just depends on boost and c++ standard library. If this
project
gets completed, we coul even think of a gcc plugin to analyze C++
code.
I think, even with Clang, the project covers other use cases as well.
Can Clang code analyzer be used independently from the compiler, just
as a library? I think the answer is yes,
but I don't know.
There will be some cases where Clang will not available (I think, but
I'm not sure), and
this library would be useful, What do you think?
Florian Goujeon
> I think, even with Clang, the project covers other use cases as well.
> Can Clang code analyzer be used independently from the compiler, just
> as a library? I think the answer is yes,
> but I don't know.
Apparently yes, it can.
> There will be some cases where Clang will not available (I think, but
> I'm not sure), and
> this library would be useful, What do you think?
Actually, I don't know Clang in details.
I only know that Scalpel seems to be a redundant project in relation to
Clang. Well, more precisely, that's what I thought since september, when
I introduced Scalpel to the Boost mailing list [1]. Then I started to
wonder whether Scalpel was worthy of being maintained.
Recently, I've talked with people who seem more aware about the
differences between Clang and Scalpel. According to them, Scalpel still
deserves to be developed, for reasons I didn't expect.
This is why I'm in a strange situation where *I* ask potential
contributors why Scalpel should be maintained ;).
Anyway, I'll finally answer to your questions!
On 11/04/2010 04:27 PM, Germ�n Diago wrote:
> 1.- Which is the status of the project? I mean, there are no commits
> since September. Is anyone working
> on it?
Indeed, I paused the development in september because I was looking for
contributors.
A contributor plans to work on the syntax analyzer in december. He's
going to fix bit-field and template-ids parsing (in light green in the
feature matrix) and then he will port the grammar from Spirit classic to
Spirit v2.
> 2.- Which would be the best way to help to the project (I'm mainly
> interested in writing source code to
> add features).
Good question. There is a lot of things to do. Perhaps the best method
would be to let you pick which feature you would feel comfortable to
implement.
First of all, how much are you familiar with the design and the source code?
The shortest term goal is the implementation of the semantic analysis of
every C++ features excluding Standard conversions, Expressions,
Statements, Templates and Exception handling.
> 3.- If I contribute source code, could you guide me in:
>
> - How should I write the tests?
The tests are written with Boost.Test. They mainly consist of:
- creating a syntax tree (by directly dealing with syntax node types);
- analyzing it using the semantic analyzer (or maybe internal functions
used by the semantic analyzer, depending on what you want to test);
- creating an instance of the expected semantic graph;
- checking whether the output of step 2 is equal to the previously
created instance.
You can find examples of test cases in test/src/name_lookup/ (these ones
test the name lookup functions, as you may have guessed).
> - Which is your reference to write the analyzers (I suppose it's
> the standard commitee paper).
Exactly ;).
Of course, I use my knowledge of the C++ language and I do some tests
with GCC when I have a doubt as well.
> - Is the source code exactly (or almost) organized as the working
> paper for c++03? I mean if, for example,
> where it says declarator-seq, it's the same identifier in the
> working paper and so on.
Yes it mostly is, but the example you've named is an exception.
The C++ grammar contains left-recursions. Left recursion cannot be
managed by LL parsers (Spirit is an LL parser). If you want to use an LL
parsers on such grammars, you have to reformulate it to eliminate
left-recursions.
The reformulated grammar is strictly equivalent to the original one though.
You can consult Spirit's documentation for further information about
left-recursions : [2] (I didn't find any occurrence of "left-recursion"
in the v2 doc. Strange.).
For convenience, I've also factorized some reccuring patterns of the C++
grammar in new rules/syntax nodes.
> Thanks for your time.
Thanks for yours, Germ�n ;).
[1] http://article.gmane.org/gmane.comp.lib.boost.devel/208217
[2] http://boost-spirit.com/old_docs/v1_6/doc/faq.html#left_recursion
Florian Goujeon
On 01/18/2011 04:22 AM, AsmWarrior wrote:
> I have fully read the discussion on boost.
> I admit that parsing c++ language was too hard if no semantic
> information is used.
> I have post a message in the user maillist:
> http://groups.google.com/group/scalpel-users/browse_thread/thread/1231dcdd7d951009
> several months ago.
Sorry about that. I did read your message, but at this time I was about
to give up.
When I finally got back to work, I thought it was too late to answer you.
I'll fix my mistake right now.
> My major interest is the codecompletion feature.But currently what
> codecompletion plugin was done was just a heuristic guess without
> semantic info. (we even do not have symbol table)
>
> about clang, it is a hand-written recursive descent parser, but it
> handling all the semantic info.
You could indeed give Scalpel a try.
Scalpel isn't fully usable yet, but we could work in collaboration with
each other.
> BTW: there is another open source c++ compiler in berlios, see:
> http://developer.berlios.de/projects/zcplusplus/
I just gave it a look. It seems like I'm not the only mad trying to
implement C++ all by oneself!
However, ZC++ apparently aims to be a 'black-box' compiler, with no
intention to be used by IDEs as a library, right?
asm warrior
Hi AsmWarrior,
On 01/18/2011 04:22 AM, AsmWarrior wrote:
I have fully read the discussion on boost.Sorry about that. I did read your message, but at this time I was about to give up.
I admit that parsing c++ language was too hard if no semantic
information is used.
I have post a message in the user maillist:
http://groups.google.com/group/scalpel-users/browse_thread/thread/1231dcdd7d951009
several months ago.
When I finally got back to work, I thought it was too late to answer you.
I'll fix my mistake right now.
My major interest is the codecompletion feature.But currently whatYou could indeed give Scalpel a try.
codecompletion plugin was done was just a heuristic guess without
semantic info. (we even do not have symbol table)
about clang, it is a hand-written recursive descent parser, but it
handling all the semantic info.
Scalpel isn't fully usable yet, but we could work in collaboration with each other.
http://en.wikipedia.org/wiki/Parsing_expression_grammar
to parse the c++ source code.
I have personally use a quite similar framework, see:
http://code.google.com/p/ell/
the ell is quite similar to the boost::spirit frame. I even use a very fast lexer, see:
http://sourceforge.net/projects/quex/, the lexer is at least 2X faster than the flex lexer.
But I was finally give up on trying the ell, because
1, it is really hard to do the error handling, it was really hard to do a recover and user friendly from parser error or lexer error.
2, since the parsing expression grammar is a ll(n) grammar, I need to keep all the lexes from the start of the source code, this is not efficient to what I have see.
so, I guess scaplel may encounter the same problem....
BTW: there is another open source c++ compiler in berlios, see:I just gave it a look. It seems like I'm not the only mad trying to implement C++ all by oneself!
http://developer.berlios.de/projects/zcplusplus/
However, ZC++ apparently aims to be a 'black-box' compiler, with no intention to be used by IDEs as a library, right?
ZC++ 's source code was so complex as I see.
Currently, I still have no idea to pick up a best codecompletion frame work.
currently, I have done a simple test to libclang, see my post:
http://forums.codeblocks.org/index.php/topic,14046.0.html
but any way, we need a lot of work.
asmwarrior