Welcome to the next generation of the Slate programming language

[goo...@timmy.tmbx.com]

Over the last year, I've been rewriting the slate virtual machine in
order to make it faster and portable to my 64-bit machine. I've gotten
to the point where I can now bootstrap slate on my machine and run
slate faster than I ever could in the past. I would like to invite
everyone to participate because I don't have enough gas to keep this
running myself.

There are a lot of changes I made in the last year and I'm sure I've
left out some:

* I got rid of pidgin and now the VM is in pure C. It is a lot cleaner
now. It is also easy to maintain and you don't have to worry about
generating the whole thing through pidgin again.

* I rewrote the garbage collector to a simple generational collector.
Young objects are allocated in another region and compacted until they
have to move into the old region (where objects don't move). When
moving young to old, we scan the whole old area so freed old holes are
used before growing the old size.

* I changed the compiler to produce SSA register based bytecode
instead of stack based bytecode. It seems to run faster and I think
this bytecode is easier to read and will be easier to inline if we
decide to do that for performance in the future.

* I changed the binary format of the image file to waste more space
and do less bitmasking of object fields. I removed the payload type
object and now I just have another field for the object size. This
made the VM/gc/etc simpler.

* I've rewritten the GUI (graphics.slate/SceneElement) code so that
now the SDL/Cairo library is about up to par with what we left off
with. We have listboxes, textboxes, buttons, and I'm working on a
LineEditorMorph which is like a multiline text editor.

* I grew the global function cache and now I've added callee caches to
each method. This is pretty much like a polymorphic inline cache. I
thought I fixed a bug where it flushes the cache when callee get
redefined but I'll have to check and make sure that code works.

* I removed delegate slots because they were complicated and hard to
follow. Now you just have "delegates" which is an array in the map of
an object. Easy to modify and understand.

* I added primitives to get the time in microseconds since the epoch,
fork the process, and create sockets. I started implementing a socket
interface to postgres but lost interest. You can log in to a database
and execute simple queries.

* I gave the VM much better debugging capabilities, you can now print
a stacktrace from within the VM with print_backtrace and you can look
at objects with print_detail and print_stack_types. These make
debugging the VM a LOT easier. There are also a lot of build flags
that you can find in the Makefile.

* I migrated the source to a git repository which seems a little more
stable than darcs was for me. They probably fixed those problems with
patches taking forever to merge a few years ago but git is more
widespread now.

* The VM is now in one C file less than 7000 LOC. Some people might
not like this style but it's a lot easier to understand and find
things I think.

* There have been lots of various bugfixes to slate files and we
probably need a unit test framework if we're going to find them all.

There might be more things but that is the gist of it. I only have an
amd 64 box to test on, so please try your luck by downloading a kernel
image from the google code page and compiling the vm sources.

[Brian T. Rice]

You've been doing excellent work, Timmy! Thanks for keeping at it, and
I'm definitely a fan of a direction that makes things more
understandable and approachable.

There will definitely be a relaunch of the site over time and
hopefully breath new life into the project and make it more useful.

[Mohamed Samy]

Thanks for the excellent work!

I've tried compiling it on my aging Pentium 4 (32bit). The VM refused
to compile, with gcc citing an error in vm.c:340 about float_t being
inconsistently redefined. A quick and dirty fix was to change it from:
typedef float float_t;
to:
#define float_t float

After that it seems to bootstrap fine with the 32bit image and I'm now
playing with the REPL.

-Mohamed

[Timmy Douglas]

I'm renaming float_t to float_type in my code so that it doesn't
conflict with other definitions. Changes have been pushed to the git
repos.

[Mohamed Samy]

On Dec 9, 12:33 am, Timmy Douglas <goo...@timmy.tmbx.com> wrote:
> I'm renaming float_t to float_type in my code so that it doesn't
> conflict with other definitions. Changes have been pushed to the git
> repos.

Great! It compiles from the first time now, but after it bootstraps it
fails to save the image:

Slate 1> Image saveNamed: 'my.image'.
Saving image to my.image
GC Freed 28821 words and coalesced 239 times
GC tenured 747 objects (8775 words)
Segmentation fault

I think this is expected as the code hasn't really reached maturity.
Do I add a bug report in the Google code site?
-Mohamed

[Brian T. Rice]

It should be fine to use Google's issue tracker. It's better than
anything Slate's had before...

[Timmy Douglas]

Is there a way you can give me some more info? This works on my amd64
machine.
Perhaps you could:

make vm.debug
gdb vm.debug
b prim_save_image
r kernel32.image
(wait for bootstrap to finish and type Image saveNamed: 'my.image'.)
(wait for break point)
type 'n' and tell me what line crashes

[Mohamed Samy]

On Dec 9, 3:31 am, Timmy Douglas <goo...@timmy.tmbx.com> wrote:
> Is there a way you can give me some more info? This works on my amd64
> machine.

Hi, and sorry for the late reply.
I followed your instructions and the GDB output was:

5881 struct Object* name = args[1];

Hope that helps! I'll try to file the bug as soon as possible.

[Mohamed Samy]

On Dec 10, 3:17 am, Mohamed Samy <samy2...@gmail.com> wrote:
> On Dec 9, 3:31 am, Timmy Douglas <goo...@timmy.tmbx.com> wrote:
>
> > Is there a way you can give me some more info? This works on my amd64
> > machine.

> I followed your instructions and the GDB output was....

Oops, I'm unfamiliar with GDB (coming from a Windows dev. background)
and didn't realize that I should keep pressing 'n' until the program
crashes!
The correct output after segfault is line 5779. More details below if
they might be of help :)

Program received signal SIGSEGV, Segmentation fault.
0x08059157 in forward_pointer_hash_get (table=0x9bb0a008,
forwardPointerEntryCount=536009, fromObj=0x9ec03008) at vm.c:5779
5779 if (entry->fromObj == fromObj || entry->fromObj == NULL)
return entry;

[Timmy Douglas]

To understand where you are exactly, could you type: 'bt'
to print a backtrace?

[Mohamed Samy]

On Dec 10, 3:41 am, Timmy Douglas <goo...@timmy.tmbx.com> wrote:
> To understand where you are exactly, could you type: 'bt'
> to print a backtrace?

With pleasure!
Here it is:

#0 0x08059157 in forward_pointer_hash_get (table=0x9bb0a008,

forwardPointerEntryCount=536009, fromObj=0x9ec03008) at vm.c:5779

#1 0x080591e2 in forward_pointer_hash_add (table=0x9bb0a008,
forwardPointerEntryCount=536009, fromObj=0x9ec03008,
toObj=0x9bf21008)
at vm.c:5795
#2 0x0805928e in copy_used_objects (oh=0xb7c04008,
writeObject=0xbf9611c8,
memoryStart=0x9ec03008 "", memorySize=6666520, table=0x9bb0a008,
forwardPointerEntryCount=536009) at vm.c:5810
#3 0x08059777 in prim_save_image (oh=0xb7c04008, args=0xbf9614f8,
n=2,
opts=0x0, resultStackPointer=22458) at vm.c:5924
#4 0x08058c33 in send_to_through_arity_with_optionals
(oh=0xb7c04008,
selector=0x9ed57650, args=0xbf9614f8, dispatchers=0xbf9614f8,
arity=2,
opts=0x0, resultStackPointer=22458) at vm.c:5676
#5 0x0805a595 in interpret (oh=0xb7c04008) at vm.c:6341
#6 0x0805b8f0 in main (argc=2, argv=0xbf961724) at vm.c:6808

[Timmy Douglas]

Well i've spent a little more than an hour or two skimming over the
code again and nothing sticks out to me.
I don't have a good 32-bit system to try it on either. You might be on
your own here until we get more people
that can look at it and debug it.

[Mohamed Samy]

On Dec 10, 5:44 am, Timmy Douglas <goo...@timmy.tmbx.com> wrote:
> Well i've spent a little more than an hour or two skimming over the
> code again and nothing sticks out to me.
> I don't have a good 32-bit system to try it on either. You might be on
> your own here until we get more people
> that can look at it and debug it.

I'm not familiar with the code yet, but perhaps I can take a look in
the coming days. I hope I or some other contributor could can come up
with something, since having to load the image every time kind of
discourages experimentation with Slate itself...oh well :)

[Timmy Douglas]

I think Brian was able to run the x86 slate without problems. If you
are interested in debugging I could help you through it.

[Mohamed Samy]

On Dec 11, 2:42 am, Timmy Douglas <goo...@timmy.tmbx.com> wrote:

> I think Brian was able to run the x86 slate without problems. If you
> are interested in debugging I could help you through it.

I hope I can allocate a few hours in the coming week to explore the
code base and see if I can help with the debugging.

BTW was anyone able to successfully compile the VM under Visual C++ on
Windows? It'd be nice to be able to work within the (comfortable)
Visual Studio debugger. I think I'll try my luck compiling it on VC
when I have the opportunity.

[steve...@gmail.com]

On Dec 9, 5:24 pm, Mohamed Samy <samy2...@gmail.com> wrote:
> forwardPointerEntryCount=536009, fromObj=0x9ec03008) at vm.c:5779
> 5779 if (entry->fromObj == fromObj || entry->fromObj == NULL)

I had the same problem. "hash" and "index" go negative, so "entry" is
invalid. Changing the "word_t" cast to "uintptr_t" when calculating
"hash" fixed it (line 5774). I though it might be even better might
be to define "word_t" as "uintptr_t" instead of "intptr_t", since that
might've shut up all those GCC warnings about printfs, but it didn't
seem to change that. (I didn't have time to find out if that had
adverse effects otherwise.)

I also had to comment out the "ARCH=-m64" line in the Makefile. Why
is that even needed? Wouldn't "-m64" be used implicitly in a 64-bit
environment?

[Timmy Douglas]

On Dec 12, 10:16 pm, "st...@folta.net" <stevefo...@gmail.com> wrote:
> On Dec 9, 5:24 pm, Mohamed Samy <samy2...@gmail.com> wrote:
>
> >     forwardPointerEntryCount=536009, fromObj=0x9ec03008) at vm.c:5779
> > 5779        if (entry->fromObj == fromObj || entry->fromObj == NULL)
>
> I had the same problem.  "hash" and "index" go negative, so "entry" is
> invalid.  Changing the "word_t" cast to "uintptr_t" when calculating
> "hash" fixed it (line 5774).  I though it might be even better might
> be to define "word_t" as "uintptr_t" instead of "intptr_t", since that
> might've shut up all those GCC warnings about printfs, but it didn't
> seem to change that.  (I didn't have time to find out if that had
> adverse effects otherwise.)

Wow thanks! I typedef'd a uword_t and I'm casting it to that on that
line now.

You shouldn't be getting warnings about printf statements because %ld
should be signed like word_t. I would like to see those warnings
you're getting if that's the case.

Also, I'm not going to change it globally because I use word_t for
smallints every where and a global change like that would mess up
smallint operations.

> I also had to comment out the "ARCH=-m64" line in the Makefile.  Why
> is that even needed?  Wouldn't "-m64" be used implicitly in a 64-bit
> environment?

I originally had it like that but it seems that it isn't implicit (or
implicitly the right setting?) on Brian's mac so I put it in like
that.