[RELEASE] Pugs 6.2.2 released!

[Autrijus Tang]

On behalf of the Pugs team, I am delighted to announce the release of
Pugs 6.2.2, the first version with a thread-safe, deadlock-free internal
storage, based on software transactional memory (STM).

Also, the performance is much, much better with this release; it is at
least 10x faster than the default 6.2.1 build, and more than 2x faster
than optimized 6.2.1 build for common applications.

Because optimized build is now the default, compiling Pugs will take
longer than before; please be patient. :-)

You can download Pugs from a nearby CPAN mirror, or from pugscode.org:

http://pugscode.org/dist/Perl6-Pugs-6.2.2.tar.gz
SIZE = 786055
SHA1 = 0cda153c52590ea7ffc17b48195d413c8adc9e16

Much thanks to all the lambdacamels for making this release possible.
It has been a wonderful three months working with you folks, and I look
forward for many more months to come.

Enjoy,
/Autrijus/

== Changes for 6.2.2 - May 1, 2005

=== Pugs Internals

* Many performance speed-ups, including:
** New, much much faster implementation type for Pad
** Replace Data.HashTable with STM Map
** Restructured the Eval monad to be reusable on compilers and interpreters
** Position handling and statement reduction logic re-written
** Add strictness to core Monad and object structures
* Functions declared as unary are now parsed as such
* Global subroutine declaration is now visible in the whole file
* Improved Parrot compiler backend support for mod_pugs
* Much better error messages, tracking source file range for expressions
* New loop control routines: `next` and `redo`
* On Win32, `-s` and `-z` now work correctly
* Optimized build is now default; use `make unoptimized` to turn it off
* Pugs can now build on Cygwin
* `#line 123` and `#line 123 "filename"` both work
* `&infix:<%>` raises trappable exception when modulus is zero
* `(undef, $x) = (1, 2)` is now supported
* `@array.end` implemented
* `BEGIN {...}` blocks now work on both statement-level and term level
* `for @a { say }` is now parsed as `for @a -> $_ is rw { say }`
* `for @a -> () { ... }` now consumes one element at a time
* `my $var ::= expr` works
* `print (4)+4` now prints 8, because the whitespace is no longer skipped
* `use Module;` supported and happens at `BEGIN` time
* `|=` `^=` `&=` for junctions

=== Tests, Examples and Documentations

* Many new tests and cleanup of older tests, we are now at 4600+ tests
* API documentation available in `docs/haddock/`
* Added an example of IRC bot into `examples/network`
* Added simple POD parser into `ext/Pod-Stream-Parser`
* Added support for generating Haskell documentation with `make haddock`
* Added the tic-tac-toe game from perlmonks into `examples/games/`
* Parallel `make smoke` when `%ENV<PUGS_TESTS_CONCURRENT>` is greater than 1
* Patched local `Test::Harness` to display number of TODO tests
* Progressive powerset generator from perlmonks added to `examples/`
* Smoke testing tool now includes timing data in YAML harness data
* German talk /Perl 6, genau jetzt!/ finished

=== Bug Fixes

* Constant references now automatically dereference when used in rvalue context
* Junctive `^` should join junctions instead of autothread them
* Lone block does not count as `simple expression` anymore
* Repair single statements in shell to preserve lexical context
* Repair the `list` context hinter
* The dot in `123.ref` is now parsed correctly as `(123).ref`
* `$_` in subroutine signatures is no longer slurpy by default
* `$x = \$x` no longer causes an infinite loop
* `NaN` calculations no longer causes infinite loop
* `last if foo` now means `last() if foo()` instead of `last(if(foo()))`
* `one(1,1).pick` bug fixed; it should return `undef`
* `splice([], 1)` no longer causes division by zero errors

[Vadim]

> You can download Pugs from a nearby CPAN mirror, or from pugscode.org:
>
> http://pugscode.org/dist/Perl6-Pugs-6.2.2.tar.gz

How can I figure out which patchlevel it corresponds to?

'pugs -v' for intermediate versions shows it (like 2616) but not CPAN
version.

Thanks!

[Rob Kinyon]

I noticed that Makefile.PL has deny_cygwin() very early on in the
process. As I hate developing in native Win32, I've been trying to
make this problem go away.

Please correct me if I'm wrong, but the only problem with Pugs on
Cygwin is the pathing requirements that GHC has, all of which are
fixable using cygpath
(http://www.reed.edu/~carlislp/ghc6-doc/users_guide/x11221.html).

The steps I've taken thus far are:
1) Link /bin/ghc to the GHC install location.
2) Created a /bin/cygghc that applies cygpath to every parameter that
begins with '/', then calls ghc.
3) Turn POSIX off.

This allows Pugs to compile and it will run the basic "Hello world"
programs. However, Pugs can't run the test suite because @INC isn't
cygpath'ed. I don't have enough Haskell-fu or compiler-fu to fix it,
but I just wanted it documented what the problems are.

I'm willing to do any testing needed to get and keep a Cygwin port
happy. If this is something the group wants to pursue, I'll post my
cygghc.

Rob

[Gaal Yahas]

On Mon, May 02, 2005 at 09:06:42AM -0400, Rob Kinyon wrote:
> I'm willing to do any testing needed to get and keep a Cygwin port
> happy. If this is something the group wants to pursue, I'll post my
> cygghc.

Pugs builds on Cygwin as of 6.2.2 (or svn r2460). pugscc needs some
cygpath love and if you're willing to work on that, that's great (I got
sidetracked). Do you have a committer bit? Drop by #perl6 and we'll get
you one if not.

--
Gaal Yahas <ga...@forum2.org>
http://gaal.livejournal.com/

[Autrijus Tang]

On Mon, May 02, 2005 at 01:44:43PM -0400, vadim wrote:
> > You can download Pugs from a nearby CPAN mirror, or from pugscode.org:
> >
> > http://pugscode.org/dist/Perl6-Pugs-6.2.2.tar.gz
>
> How can I figure out which patchlevel it corresponds to?

Well, you can grep the svn log for "6.2.2". :-)

r2604: * This be Pugs 6.2.2 for real.

Thanks,
/Autrijus/

[Rob Kinyon]

Gaal -

I pulled svn r2461 and it does compile on cygwin, yes. But, the
@INC problem is still there, preventing 'make test' from running
successfully. Do you want me to look at that?

Rob

[Autrijus Tang]

On Tue, May 03, 2005 at 07:36:24AM -0400, Rob Kinyon wrote:
> I pulled svn r2461 and it does compile on cygwin, yes. But, the
> @INC problem is still there, preventing 'make test' from running
> successfully. Do you want me to look at that?

Please do, thanks. :-)

Also there is the problem with script/pugscc/ passing non-cygpathed
include paths to GHC. Try:

make install
pugscc -e "say 123"

To see the error message on Cygwin. It's around line 80 of
script/pugscc, and it'd be great if you can look into that too.

Thanks,
/Autrijus/

[Rob Kinyon]

Oh - one more thing. hsc2hs needs to be done by hand on Cygwin. You
cannot allow it to try and built through because the last step
(executing the .exe redirected to the .hs) will fail and that cannot
be cygpath'ed (I don't think). Having the makefile execute each step
separately may be a good option. I can try to cygpath the filenames
before they get passed to hsc2hs, but I don't think that's best.

Rob

[Rob Kinyon]

I've found the problem with "make test" not working. The problem is
the pathname not being cygpath'ed. Basically, every call to
doesFileExist needs to be cygpath'ed. Unfortunately, I have no idea
how to do that. All the calls to doesFileExist seem to be in
src/Pugs/Prim.hs lines 204,1250,1256. The one on line 204 is for
require, which is what's preventing "use Test;" from working. I don't
know where fileTestExists and fileTestIsFile are used, but they should
be modified as well.

As for pugscc, the only changes that need to be made are:
1) Use of the cygpath() function on $core and $base.
sub cygpath {
my $in = shift;
return $in if $^O ne 'cygwin';
open my $path, "cygpath -m $in |"
or die "Cannot fork to 'cygpath $in': $!\n";
chomp( my $val = <$path> );
return $val;
}
2) Any use of backticks needs to have all \r stripped out. This is
primarily seen on line 75, where you have "my $archlib = `pugs
-V:archlib`;" Personally, I would convert that to named pipes, but
cygpath() could also be converted to backticks with \r stripped.

I am afraid to commit because with those changes, I get to Pugs, but I
receive the following errors and I don't know enough about Pugs to get
it working. (This is with r2765, built about 2 hours ago.)

Rob

$ ./script/pugscc -e 'say 123'

*** Using the 'Pugs' backend.

ghc -v0 -o a.out --make -main-is MainCC.mainCC
-LC:/cygwin/home/Customer/pugs
-LC:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs
-LC:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/pcre
-LC:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck
-IC:/cygwin/home/Customer/pugs
-IC:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs
-IC:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/pcre
-IC:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck
-iC:/cygwin/home/Customer/pugs
-iC:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs
-iC:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/pcre
-iC:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck -static
-Wall -fno-warn-unused-imports -fno-warn-unused-binds
-fno-warn-missing-signatures -fno-warn-name-shadowing
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/pcre/pcre.o
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck/syck.o
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/UnicodeC.o
C:/cygwin/home/Customer/pugs/MainCC.hs

C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/Text/PGE.o(.text+0x1326):ghc3916.hc:
undefined reference to `pge_gen'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/Text/PGE.o(.text+0x145d):ghc3916.hc:
undefined reference to `pge_parse_new'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/Text/PGE.o(.text+0x159f):ghc3916.hc:
undefined reference to `pge_parse_free'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/Text/PGE.o(.text+0x5a06):ghc3916.hc:
undefined reference to `pge_init'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/Text/PGE.o(.text+0x5b0a):ghc3916.hc:
undefined reference to `pge_parse_free'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/Text/PGE.o(.text+0x5ba8):ghc3916.hc:
undefined reference to `pge_parse'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/Data/Yaml/Syck.o(.text+0x1965):ghc3916.hc:
undefined reference to `syck_map_read'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/Data/Yaml/Syck.o(.text+0x1a44):ghc3916.hc:
undefined reference to `syck_map_read'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/Data/Yaml/Syck.o(.text+0x1c51):ghc3916.hc:
undefined reference to `syck_seq_read'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/Data/Yaml/Syck.o(.text+0x1ef1):ghc3916.hc:
undefined reference to `syck_str_read'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck/syck.o(.text+0x302):syck.c:
undefined reference to `st_init_numtable'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck/syck.o(.text+0x319):syck.c:
undefined reference to `st_insert'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck/syck.o(.text+0x349):syck.c:
undefined reference to `st_lookup'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck/syck.o(.text+0x364):syck.c:
undefined reference to `syck_free_node'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck/syck.o(.text+0x392):syck.c:
undefined reference to `st_foreach'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck/syck.o(.text+0x39d):syck.c:
undefined reference to `st_free_table'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck/syck.o(.text+0x3bf):syck.c:
undefined reference to `st_foreach'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck/syck.o(.text+0x3ca):syck.c:
undefined reference to `st_free_table'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck/syck.o(.text+0x3f6):syck.c:
undefined reference to `st_free_table'
C:/cygwin/lib/perl6/cygwin-thread-multi-64int/CORE/pugs/syck/syck.o(.text+0x862):syck.c:
undefined reference to `syckparse'
Died at ./script/pugscc line 111.

[Gaal Yahas]

On Mon, May 02, 2005 at 09:06:42AM -0400, Rob Kinyon wrote:

> I'm willing to do any testing needed to get and keep a Cygwin port
> happy. If this is something the group wants to pursue, I'll post my
> cygghc.

Okay, here's the Cygwin situation right now. This is a summary of
discussions with Rob and others on #perl6 and around.

Building Pugs on Cygwin doesn't result in a Cygwin executable at
all. It doesn't depend on the Cygwin DLL; and doesn't have any fun POSIX
functionality. The key points are:

* It compiles with the .msi GHC, which is MSYS-based;
* HAVE_POSIX is undefined, just like in an MSYS or native win32 build.

The original reason it's even possible to build Pugs on Cygwin is that
it's just more convenient to use Cygwin as a development environment than
MSYS: newer perl5, modern shell and editor, and so on. What's primarily
misleading is that $*OS is currently set to 'cygwin'; this is a bug in
pugs ($*OS should be the OS running under, not the one this pugs was
built with).

To make a Cygwin pugs, we would need to use a Cygwin GHC.[1] To the best
of my knowledge, nobody maintains public binaries of that; and there is
also the somewhat scary warning of a binary GHC being "a moving target"
since cygwin1.dll often changes in ABI-breaking ways. In short, a real
Cygwin port of pugs is something of a challenge. It might become easier
if done in cooperation with cygwin.com.

Until we find a champion for that cause, I think it would still be useful
to allow pugs to be built, as today, on Windows using Cygwin's gcc and
perl5 (just like Corion makes a native build with -- I think -- cc1.exe
and a homebrew perl5). The path question becomes the same as it is for
any other non-Cygwin tool used in a Cygwin shell; how is it solved there?

[1] See:
http://www.haskell.org/ghc/docs/5.04/html/building/winbuild.html
http://www.reed.edu/~carlislp/ghc6-doc/users_guide/x11221.html

[Rob Kinyon]

> To make a Cygwin pugs, we would need to use a Cygwin GHC.[1] To the best
> of my knowledge, nobody maintains public binaries of that; and there is
> also the somewhat scary warning of a binary GHC being "a moving target"
> since cygwin1.dll often changes in ABI-breaking ways. In short, a real
> Cygwin port of pugs is something of a challenge. It might become easier
> if done in cooperation with cygwin.com.

This is not going to be me. I don't know Haskell or cygwin1.dll. I
have no desire to learn the latter and have the desire but no tuits to
learn the former.

> Until we find a champion for that cause, I think it would still be useful
> to allow pugs to be built, as today, on Windows using Cygwin's gcc and
> perl5 (just like Corion makes a native build with -- I think -- cc1.exe
> and a homebrew perl5). The path question becomes the same as it is for
> any other non-Cygwin tool used in a Cygwin shell; how is it solved there?

I think the solution is to provide a Cygwin version of the Directory
and IO Haskell modules. I don't know if this is possible in Haskell.
I'd prefer to have a PugsDirectory and a PugsIO module and have them
be aliases for the standard Directory and IO modules when it's not
Cygwin. If it is Cygwin, then wrap every function call with a call to
cygpath if the directory or file name begins with '/'.

Is this possible? Desirable?

Rob

[Gaal Yahas]

On Tue, May 10, 2005 at 10:00:45AM -0400, Rob Kinyon wrote:
> > Until we find a champion for that cause, I think it would still be useful
> > to allow pugs to be built, as today, on Windows using Cygwin's gcc and
> > perl5 (just like Corion makes a native build with -- I think -- cc1.exe
> > and a homebrew perl5). The path question becomes the same as it is for
> > any other non-Cygwin tool used in a Cygwin shell; how is it solved there?
>
> I think the solution is to provide a Cygwin version of the Directory
> and IO Haskell modules. I don't know if this is possible in Haskell.
> I'd prefer to have a PugsDirectory and a PugsIO module and have them
> be aliases for the standard Directory and IO modules when it's not
> Cygwin. If it is Cygwin, then wrap every function call with a call to
> cygpath if the directory or file name begins with '/'.
>
> Is this possible? Desirable?

It is possible to put cygpath-like capabilities inside pugs and compile
it in on any win32 build, but it sounds to me like a runtime hit on a
lot of IO, unless someone comes up with the magic to choose which path
mode to work with at pugs start time.

The alternative is even in Cygwin to put things like C:\ paths in
PERL6LIB and friends. pugscc and the rest of the p5 supporting tools
would need to ignore the fact that they're running under Cygwin and feed
pugs only Windows paths. (BTW, pugscc looks like it can be ported to p6
already.)

Let's make a list of things that are broken. For me things mostly work:
the test suit does, if I use the yaml harness (p6); 'make test' and
pugscc do not. Do you know of anything else?

Are there other issues besides paths?

[Rob Kinyon]

I have a "working" pugscc and cannot run the test suite. Should I
check in my pugscc changes or post them to the list so that Gaal can
run them against a Pugs that has a working test suite?

Rob