Perl6 and support for Refactoring IDE's
J Matisse Enzer
available for Java, C#, etc?
I've been using Perl since 1994 and for the past several years have
used it to build a number of complex mod_perl applications. I love
Perl.
The following may be considered heresy, but I believe that these days
the quality of the whole development environment is just as important,
maybe *more* important than the language itself. I mean that the
availability of *easily integrated* tools including IDE, automated
testing tools, refactoring, deployment tools, etc. are a HUGE factor in
determining what language gets used for a large project
I've become scared that if Perl is to continue to be viable for large,
complex, multi-developer projects that the tools need to serious
catch-up with what is available for Java, for example. Things like:
- Refactoring Support (see http://www.refactoring.com/)
- CVS and/or Subversion integration
- Support for integrating regression tests and auto-building
- Automated syntax and dependency checking
I've been using Eclipse, with the EPIC plugin
(http://e-p-i-c.sourceforge.net/) and so far I like it. It uses
Devel::Refactor to support "extract subroutine", but a lot more is
needed to match what you can do with Java these days.
What are others' thoughts on this?
-------------------------------------------------------
Matisse Enzer <mat...@matisse.net>
http://www.matisse.net/ - http://www.eigenstate.net/
Luke Palmer
> I've become scared that if Perl is to continue to be viable for large,
> complex, multi-developer projects that the tools need to serious
> catch-up with what is available for Java, for example. Things like:
>
> - Refactoring Support (see http://www.refactoring.com/)
> - CVS and/or Subversion integration
> - Support for integrating regression tests and auto-building
> - Automated syntax and dependency checking
>
> I've been using Eclipse, with the EPIC plugin
> (http://e-p-i-c.sourceforge.net/) and so far I like it. It uses
> Devel::Refactor to support "extract subroutine", but a lot more is
> needed to match what you can do with Java these days.
>
> What are others' thoughts on this?
I think you're absolutely right. Perl should have an IDE with
Eclipse-like context-sensitivity and refactoring support. However,
it's hardly in Perl's philosophy or interest to bless one.
One thing is for sure. Perl 6 is providing enough introspection and
parsing capabilities to make it possible to write a context-sensitive
IDE, unlike Perl 5 (well, Perl 5 made it *possible*, I suppose, but
Perl 6 will make it obvious). Perl 6 is exposing its whole grammar at
the language level, so you can say "give me a syntax tree for this
chunk of code" and it will. Even if there are modules that change the
syntax with macros (though your editor might have trouble
understanding what the macros mean).
In other words, Perl 6 is open to the possibility of such an IDE, and
is going to provide the machinery necessary to build a really good
one, but I doubt it will become a development milestone.
Luke
Fagyal Csongor
> Will Perl 6 help us have tools that are as good or better than the
> ones available for Java, C#, etc?
>
> I've been using Perl since 1994 and for the past several years have
> used it to build a number of complex mod_perl applications. I love Perl.
>
> The following may be considered heresy, but I believe that these days
> the quality of the whole development environment is just as important,
> maybe *more* important than the language itself. I mean that the
> availability of *easily integrated* tools including IDE, automated
> testing tools, refactoring, deployment tools, etc. are a HUGE factor
> in determining what language gets used for a large project
Heresy or not, I think you are (mostly) right. Actually I would say your
are right *unfortunately*.
Available tools are a huge factor, yes - not in a way as which language
to select for a project, but which one to *deselect*. And currently Perl
(I mean Perl5) is not doing very well compared to others.
> I've become scared that if Perl is to continue to be viable for large,
> complex, multi-developer projects that the tools need to serious
> catch-up with what is available for Java, for example. Things like:
>
> - Refactoring Support (see http://www.refactoring.com/)
> - CVS and/or Subversion integration
> - Support for integrating regression tests and auto-building
> - Automated syntax and dependency checking
IMHO subversioning does not have too much to do with the language
itself. Subversioning operates on files. An IDE might integrate some
interface for this task, but that is a different question.
Syntax checking in general is very hard to do in Perl5/6 because of the
great amount of line noise there (consider yourself in the middle of
writing a regexp :)) The perl6 compiler must be able to do syntax
checking, though :-), and the design of Perl6, as I understand,
definetely gives you more possibilities compared to Perl5.
For the others, some more knowledgeable folks will probably provide some
answers and ideas. However, I think that the language design and
implementation does not have too much to do with IDE availability.
> I've been using Eclipse, with the EPIC plugin
> (http://e-p-i-c.sourceforge.net/) and so far I like it. It uses
> Devel::Refactor to support "extract subroutine", but a lot more is
> needed to match what you can do with Java these days.
>
> What are others' thoughts on this?
Have you considered using the "DONATE" button on
http://e-p-i-c.sourceforge.net/ ? :)))
- Fagzal
J Matisse Enzer
Thanks for your comments - I was afraid I'd get flamed for suggesting
something wasn't perfect about Perl :-)
On May 6, 2005, at 9:31 AM, Fagyal Csongor wrote:
>
> IMHO subversioning does not have too much to do with the language
> itself. Subversioning operates on files. An IDE might integrate some
> interface for this task, but that is a different question.
You are right - it's not a language issue, sorry I left that in.
> Have you considered using the "DONATE" button on
> http://e-p-i-c.sourceforge.net/ ? :)))
Yes :-) I've donated :-)
I contributed some changes to Devel::Refactor (added rename_subroutine
and a test suite) and now I have loaded the Java source for the EPIC
editor into Eclipse and started looking at how to add support for
'rename_subroutine' - but my Java skills are less than minimal :-)
In another post, On May 6, 2005, at 9:26 AM, Luke Palmer wrote:
>
> I think you're absolutely right. Perl should have an IDE with
> Eclipse-like context-sensitivity and refactoring support. However,
> it's hardly in Perl's philosophy or interest to bless one.
and I agree that Perl should not bless any one IDE, and I am *very*
happy to read that Perl6 will make it much easier for good tools to be
created for Perl, as Luke Palmer said:
> One thing is for sure. Perl 6 is providing enough introspection and
> parsing capabilities to make it possible to write a context-sensitive
> IDE, unlike Perl 5 (well, Perl 5 made it *possible*, I suppose, but
> Perl 6 will make it obvious). Perl 6 is exposing its whole grammar at
> the language level, so you can say "give me a syntax tree for this
> chunk of code" and it will. Even if there are modules that change the
> syntax with macros (though your editor might have trouble
> understanding what the macros mean).
So, perhaps that leave me (us) in the following situation:
1) Do what I can do help Perl6 happen faster (not much I can do, the
current types of programming required is beyond my skills)
Maybe I can help evangelize Perl6.
2) Try and improve the tools that do exist for Perl5 - I am doing this
with Devel::Refactor and EPIC, but I don't know if I'll learn
enough
Java to make a difference in EPIC any time soon.
3) Try and keep Perl alive in other ways so it's still around when
Perl6 arrives.
Yesterday I was at an engineering Open House at Google, and I asked one
of the engineers "What can Perl do for Google?", and he basically said
(I'm paraphrasing) "Not much, we are a Java and Python shop. Getting a
real refactoring IDE for Perl 5 is more compelling a story then a
virtual machine (Parrot) that will run multiple dynamically-typed
languages." (they already have Jython)
Now that was just one person's statement, but this fellow (John Brewer,
www.jera.com) is not a Perl hater - he uses Perl, and he's bright,
level headed and was not trying to be hostile. In fact he said these
things to me and to Larry Wall who was listening very politely :-)
Maybe this isn't the right place to keep discussing this, so I'll take
pointers to other places. I'm very worried about the continued
viability of Perl for major projects and am trying connect with other
people and see what can be done about it.
Larry Wall
: In other words, Perl 6 is open to the possibility of such an IDE, and
: is going to provide the machinery necessary to build a really good
: one, but I doubt it will become a development milestone.
I think that, just as Perl 1 built in all the system interfaces, and
Perl 5 allowed interfaces to every database under the sun, we should
just make sure that Perl 6 can interface easily to every existing
and future IDE, including any that might be written particular with
Parrot in mind.
Larry
Fagyal Csongor
Just one note before we take this off-list:
> Maybe this isn't the right place to keep discussing this, so I'll take
> pointers to other places. I'm very worried about the continued
> viability of Perl for major projects and am trying connect with other
> people and see what can be done about it.
I share some of your fears, and would be very much interested in talking
about them with you (and others). Actually I gave a small lightening
speech on this topic at last year's Perl gathering at Budapest, which I
will hopefully make into an article some day...
<please_spare_my_life>IMHO Perl needs some more marketing
:)</please_spare_my_life>
- Fagzal
Piers Cawley
One of the 'mental apps' that's been pushing some of the things I've been
asking for in Perl 6's introspection system is a combined
refactoring/debugging/editing environment for the language. One of the
annoyances of the 'only perl can parse Perl' thing is not so much the truth of
the statement, but that perl 5 doesn't allow you to ask about the parsed code
in ways that would be useful for an IDE. Perl 6 promises to change that -- it
should be possible to either write a fantastic Perl 6 IDE in perl itself, or to
write a codegrokker object that can be used by some pre existing IDE.
> In other words, Perl 6 is open to the possibility of such an IDE, and
> is going to provide the machinery necessary to build a really good
> one, but I doubt it will become a development milestone.
What about the debugger?
Deborah Pickett
> One of the 'mental apps' that's been pushing some of the things I've been
> asking for in Perl 6's introspection system is a combined
> refactoring/debugging/editing environment for the language. One of the
> annoyances of the 'only perl can parse Perl' thing is not so much the truth of
> the statement, but that perl 5 doesn't allow you to ask about the parsed code
> in ways that would be useful for an IDE. Perl 6 promises to change that -- it
> should be possible to either write a fantastic Perl 6 IDE in perl itself, or to
> write a codegrokker object that can be used by some pre existing IDE.
While I agree with this wholeheartedly, I am afraid to say that I
haven't seen a lot of evidence that introspection has been given much
thought in the Perl 6 design process. There was a flurry at one point
about the execution of BEGIN blocks, but then silence again.
One thing is for sure: although B::Deparse is a wonder to behold, it
isn't something that we should try to be repeating. It doesn't give you
control over how much of the code to parse, whether you trust the code
your're parsing, or in the case of BEGIN blocks even whether you have
the choice to execute them or not. It doesn't give you the ability to
load parts of another program into the running program's namespace, and
it doesn't give you enough hooks and callbacks to be able to transform
syntax trees. At least, if it does do any of those things, then I
haven't seen ways of doing them in the documentation for B and O.
I'm afraid that because of the dynamic parse/execute nature of Perl, it
may be a theoretically intractable problem to parse Perl safely. Don't
run the BEGIN blocks and you change the way future source is parsed
compared with true runtime. Do run them and you risk someone inserting
dangerous code into one. It isn't good if turning on syntax
highlighting on some found-in-the-wild code in your Perl-aware editor
could wipe your hard disk.
Oh, and I don't have a lot of faith in the Safe module, either, which is
a marvel of kludge engineering and broken, both at the same time.
What's the plan for that module's moral equivalent in Perl 6?
I'd love it if someone could set my mind at ease here. I have to
process syntax trees of Perl code quite often in my line of work and I'd
love something less finicky than a Safe/B::* combination.
Stevan Little
> One of the 'mental apps' that's been pushing some of the things I've
> been
> asking for in Perl 6's introspection system is a combined
> refactoring/debugging/editing environment for the language.
Maybe I have been reading too much about Smalltalk meta-classes lately,
but in doing some draft ideas of the Perl6 meta-model for Pugs, I
realized that given a really solid class/metaclass introspection system
and access to the interpreter level meta-meta-classes, it would be
possible to easily write a class browser much like Smalltalk.
And to extend that to also be a refactoring browser would require the
meta-meta-class system to be able to generate and emit code, which, if
we properly model the meta-meta-classes should also be fairly simple as
well.
And of course all this is even simpler if the interpreter is written in
Perl6 (although the cyclical nature of that can be dizzying at times).
However adding debugging support to this is another matter all
together, and it is only partially useful to the non-OO programmer.
Anyway, just wanted to add my 2 cents. Back to $work for me :)
Stevan
Luke Palmer
> I'm afraid that because of the dynamic parse/execute nature of Perl, it
> may be a theoretically intractable problem to parse Perl safely.
Yep. It's not really possible for the parser to distinguish between:
BEGIN {
%main::{'&foo'} = sub { print "Hello!" };
}
And:
BEGIN { for 1... {} }
Or even:
BEGIN { system 'rm -rf /' }
So, like most things in Perl, we can trust the author of the code not
to do such things, or we can not trust them and lose something while
we're at it (the choice is given to the IDE user, not to us
designers). BEGIN blocks should never do INITialization, opening
filehandles, etc., precisely for this reason. This becomes even more
important in Perl 6 where bytecode will be more prevelant.
You may certainly parse Perl syntax trees without executing BEGIN
blocks, but you have to bite the bullet if the BEGIN changes the
syntax in a way that you can no longer parse it if you don't execute
it. And if you can't accept that, I see only one option: give up
BEGIN blocks. And we're not going to do that (but you're free to
place that restriction on whoever's code you're parsing).
Luke
Piers Cawley
> On May 25, 2005, at 5:39 AM, Piers Cawley wrote:
>> One of the 'mental apps' that's been pushing some of the things I've been
>> asking for in Perl 6's introspection system is a combined
>> refactoring/debugging/editing environment for the language.
>
> Maybe I have been reading too much about Smalltalk meta-classes lately, but in
> doing some draft ideas of the Perl6 meta-model for Pugs, I realized that given
> a really solid class/metaclass introspection system and access to the
> interpreter level meta-meta-classes, it would be possible to easily write a
> class browser much like Smalltalk.
Yes. The application in my head looks very a Smalltalk environment. The way I
see it working is that the language itself has a bunch of minimal hooks that
get triggered by various phases of compilation etc. Your editor then becomes
something that instruments the compiler in such a way that loading a file for
editing compiles it, but the compilation process hangs populates the data
structures you need for editing. So, when you go to edit, say:
class Collection {
method inject_into ($self: $initial_value is copy, *&block) {
$self.each
:{ $initial_value = &block($initial_value, $_) }
return $initial_value;
}
}
You'd end up with a Class object which would have all sorts of interesting
things added to it, including collections of instance variables, class
variables, methods etc. Methods would have metadata about the environment in
which they were evaluated (so when you edit a method you could recompile it in
the correct environment), the file they're found in, their source code
(possibly attributed to allow for syntax highlighting), the resulting AST,
etc. Once you have such a rich set of metadata to hand, it becomes a simple
matter of programming to add all sorts of handy features, refactorings,
breakpoints, whatever...
The point being that you don't really need a massive amount of support from the
core language to do all this. At a pinch you can take advantage of the fact
that the grammar is replaceable and core classes are extensible (and if they're
not, you just stick them in boxes for the editor's purposes).
> And to extend that to also be a refactoring browser would require the
> meta-meta-class system to be able to generate and emit code, which, if we
> properly model the meta-meta-classes should also be fairly simple as well.
We've got eval for that. Assuming you can do eval with arbitrary bindings (and
if necessary you can drill down to parrot for it) you just do something like:
Class.can('fred').environment.eval($freds_new_sourcecode)
Of course, if you alter a macro you're going to have to reevaluate pretty much
all the source code, but so what?
> And of course all this is even simpler if the interpreter is written in Perl6
> (although the cyclical nature of that can be dizzying at times).
It really doesn't have to be. It'd be nice, but not necessary. You just have to
make sure you can extend the runtime using perl. And macros already do that.
> However adding debugging support to this is another matter all together, and it
> is only partially useful to the non-OO programmer.
Not that hard to be honest. The same instrumented interpreter techniques can be
used to write the debugger.
J Matisse Enzer
On May 26, 2005, at 10:03 AM, Piers Cawley wrote:
> Stevan Little <ste...@iinteractive.com> writes:
>
> ... The way I
> see it working is that the language itself has a bunch of minimal
> hooks that
> get triggered by various phases of compilation etc. Your editor then
> becomes
> something that instruments the compiler in such a way that loading a
> file for
> editing compiles it, but the compilation process hangs populates the
> data
> structures you need for editing.
Sorry to have dropped out of this thread for so long. I'm glad to see
such smart people thinking about this too.
I like what Piers is suggesting, not because I am in any way qualified
to say how practical it is, but because the idea he presents really
would make it possible to have much better tools for Perl.
The idea that the language will allow an editor (an IDE) to really find
out the "knowable" information about the source code is the crucial
thing.
Frankly, there are huge improvements in tools possible for Perl5, just
by accepting some limitations - we probably cannot duplicate the
refactoring support available for Java in Perl5, but we can do one heck
of a lot better than just "extract subroutine" (which is the *only*
refactoring available now in a Perl IDE.)
We are still not even at the "simple things should be easy" stage of a
refactoring IDE for Perl, and I'd be happy to at least get there before
worrying about the "hard things are possible" stage :-)
Once I saw how cool the features in something like Eclipse are for
another language it really has made me see how far behind Perl is in
this area. I'm really scared that Perl will be an unacceptable choice
for larger projects in less than a two years.
-Matisse
-------------------------------------------------------
Matisse Enzer <mat...@matisse.net>
http://www.matisse.net/ - http://www.eigenstate.net/
415-225-6703 (work/cellphone)
415-401-8325 (home)