xpost3 current challenges

[luser- -droog]

I've got 2 trains going at the same time here, and kinda stalled on both.

Track 1 is finishing up this postscript translation of the xpost2 scanner. It's running into issues with building procedures. And I've really painted myself into a corner with lots of code and virtually no comments. I suppose the obvious next step is to print it out and re-type it (fixing the uglies as they arise).

Track 2 is taking a stab at multiprocessing. Both NeWS and Display Postscript appear to have the same semantics for 'fork' and 'join'. A simple incrementing processID should work for unique IDs. The context structures should be ok in a simple linked-list.

But there's a problem with the way I've tied the stacks to specific entries in the memory table. Two processes can't share the same memory space like this. And I can't suspend/resume the interpreter without the context information safely stored in VM.

And the garbage collector will have to define its root set differently if the stacks aren't at special table entries. Ach!


And all of this is really a distraction from the fact that I can't make sense of the Kajiya paper about intersecting rays with Bezier patches, because I don't have enough algebra to understand Bezout's Theorem.

Hmm. Maybe the rbuss sources would help with the context issues...

[luser- -droog]

And NeWS implements processtype objects as magic dictionaries. So maybe that's what I should be working on. Should be pretty straightforward. dicget and dicput need to recognize a special value (object type) that refers to a getter/setter struct, something like

struct gsetter {
object (*get)(void);
void (*set)(object);
};

[luser- -droog]

On further investigation, DPS and NeWS are quite different in their precise semantics. And DPS seems simpler, so I'll start there.

DPS describes 3 ways of creating a new context:

Method 1: Private Global VM, Private Local VM

Method 2: Shared Global VM, Private Local VM

Method 3: Shared Global VM, Share Local VM

So as a basic model, I'm starting with a sequence of forking by each method, producing 4 contexts.

context0: Global0 Local0
---> fork method 1
context0: Global0 Local0
context1: Global1 Local1
---> fork method 2
context0: Global0 Local0
context1: Global1 Local1
context2: Global1 Local2
---> fork method 3
context0: Global0 Local0
context1: Global1 Local1
context2: Global1 Local2
context3: Global1 Local2

But the garbage collector needs a different view

Global0: context0
Global1: context1 context2 context 3
Local0: context0
Local1: context1
Local2: context2 context3

garbage collection requiring knowledge of the appropriate "root sets" in use.

So there appears to be a need for these 3 master tables for the global interpreter state. It doesn't really belong in any of the VM files.

-context-table
-current-context-id
-global-table
-local-table

this could be stored in yet another memory file. So the complete interpreter state would also be reflected in the disk files:

g0.mem g1.mem
l0.mem l1.mem l2.mem
itp.mem

[luser- -droog]

On Saturday, May 11, 2013 12:53:31 AM UTC-5, luser- -droog wrote:

>
> So there appears to be a need for these 3 master tables for the global interpreter state. It doesn't really belong in any of the VM files.
>
>
>
> -context-table
>
> [ ] -current-context-id

---^ should've indented this one. There's 3 tables, but 4 'variables'.

>
> -global-table
>
> -local-table
>
>
>
> this could be stored in yet another memory file. So the complete interpreter state would also be reflected in the disk files:
>
>
>
> g0.mem g1.mem
>
> l0.mem l1.mem l2.mem
>
> itp.mem

This is going to involve a bit of redesign of contexts and mfiles, as well as a cool name for the master super-duper data structure that in the darkness binds them.

But I think I'll keep itp.c pretty much as-is, and the testing main() will for now be the entry-point to the single-process interpreter. The "ghostscript"-mode, if you will. The multiprocessing stuff really needs to be a new module. So it needs a module-name, too. Going by my pattern so far, it should be a 3-letter identifier. "srv" maybe, or ... not coming up with anything else. "srv" it is!

I eventually want to make a "quick-launch" option for itp. But there needs to be some provision for validating a pre-initialized VM file. And depending on the severity of the validation checking, "quick-launch" may be a failure in the end. But it feels like *the thing to do*, you know?

[luser- -droog]

PLRM, 2ed, 7.2 Encoded User Names
[...]

The meaning of a given user name index is local to a specific PostScript
execution context—precisely, to a context’s local VM. If several contexts
share the same local VM, a user name index defined in one context
may be used in another context. In this case, it is the application
programmer’s responsibility to synchronize execution of the contexts
so definition and use occur in the correct order.

Entries placed in the user name table by defineusername accumulate
during the lifetime of the context’s local VM; they do not obey save and
restore, and they cannot be removed or replaced. These restrictions are
intentional: they permit the Client Library to manage user name definitions
without understanding the semantics of the PostScript language
program being generated.
--------------------------

So, I've got to move my names from global-vm to local-vm. And the operator table probably doesn't need to be in global-vm, either, since it cannot be bound to a name object like I'm doing now (In addition to the systemdict:name->operator binding, I put a copy of the name-index in the operator object itself).

So this'll require some re-design.

[luser- -droog]

I hacked my string scanner into the eval-loop, and discovered a bug that xpost2 almost certainly has too.

The problem was `forall`.

string proc forall
array proc forall

but what if instead you do this?

string cvx proc forall
proc proc forall

My pseudocode goes like

forall(src, proc)
push(es, forall)
push(es, cvx)
push(es, cvlit(proc))
push(es, getinterval(src, 1, src.sz-1))
push(es, proc)
push(os, src[0])

So the "getinterval" of the source object gets pushed on the exec stack, expecting it to spill back onto the op stack as data. But if the source object is flagged executable, it gets executed instead.

The fix is to always convert this source object to literal when pushing it to the exec stack; and add a
`cvx` if it was executable before conversion.

This fixed, I might now turn my attention to getting files to work. Or maybe fix up the garbage collector.
Last I looked, I hadn't written a `markdict` routine yet, so I set the period really high (basically de-activating the whole darned thing) until I felt like diving back into that.

[luser- -droog]

On Monday, June 24, 2013 2:48:49 AM UTC-5, luser- -droog wrote:
> I hacked my string scanner into the eval-loop, and discovered a bug that xpost2 almost certainly has too.

Nope. xpost2 had already faced that one. forgot to check. :)

The current challenge is finishing-up a new translation of the scanner,
this time back to C.

The issue is building procedures. This is where the scanner calls
itself recursively, becoming a "parser" in so doing (where everywhere
else it may be treated as a deterministic automaton).

I've got two versions to draw from. The recent translation into
postscript

/* ({)0 get {
pop
mark exch % [ s'
{ % [ ... s'
toke % [ ... s' t b
not { syntaxerror } if % [ ... s' t
dup (}) cvn eq { % [ ... s' t
pop % [ ... s'
counttomark 1 add 1 roll % s' [ ...
] cvx exit
} if % [ ... s' t
exch % [ ... t s'
} loop % s' {}
%true % s' {} true
} */

And the earlier C version.


case '{': {
object *a;
size_t na = 100;
size_t i;
object proc;
object fin;

fin = consname(st,"}");
(a = malloc(na * sizeof(object))) || (fatal("failure to malloc"),0);
for (i=0 ; objcmp(st,a[i]=toke(st,src,next,back),fin) != 0; i++) {
if (i == na-1)
(a = realloc(a, (na+=100) * sizeof(object))) || (fatal("failure to malloc"),0);
}
proc = consarray(st,i);
{ size_t j;
for (j=0; j<i; j++) {
a_put(st, proc, j, a[j]);
}
}
free(a);
return proc;
}

Very different. I intend the result to be a hybrid of the two.
Using the stack to build the array, since it's already extensible
(and not 64k limited), but being in C, which the second ...
... uh ... is.

The second one doesn't have a lot going for it. It's a lot better
than the linked-list nonsense it evolved from. It's pretty short,
fairly easy to read (if you pardon the not-very-C-like
malloc||fatal idioms); but, uh. There's something not quite right.
Can't quite put a finger on it. It's not defining a crazy
local structure type like previous versions. But it's allocating
memory! And it's not using any of my painstakingly-designed
memory architecture! It just skips my whole garbage collector!
My beautiful tables. I like this code. I reared it like a child.

Now begone, wretched (w)realloc!

This is going nowhere.

My vacation ends today. :(

[luser- -droog]

On Monday, July 22, 2013 5:33:10 AM UTC-5, luser- -droog wrote:
> On Monday, June 24, 2013 2:48:49 AM UTC-5, luser- -droog wrote:
>
> > I hacked my string scanner into the eval-loop, and discovered a bug that xpost2 almost certainly has too.
>
> Nope. xpost2 had already faced that one. forgot to check. :)
>
> The current challenge is finishing-up a new translation of the scanner,
> this time back to C.
>
> The issue is building procedures. This is where the scanner calls
> itself recursively, becoming a "parser" in so doing (where everywhere
> else it may be treated as a deterministic automaton).
>
>

There we go. That wasn't so bad once I started typing. :)

case '{': { // This is the one part that makes it a recursive-descent parser
object tail;
tail = consname(ctx, "}");
push(ctx->lo, ctx->os, mark);
while (1) {
object t = toke(ctx, src, next, back);
if (objcmp(ctx, t, tail) == 0)
break;
push(ctx->lo, ctx->os, t);
}
arrtomark(ctx); // ie. the /] operator
return cvx(pop(ctx->lo, ctx->os));
}

[luser- -droog]

On Saturday, May 4, 2013 11:59:02 PM UTC-5, luser- -droog wrote:

The scanner is working nicely. And amusingly, the diff highlighter just doesn't know what to make of it:
http://code.google.com/p/xpost/source/diff?spec=svnd2a0aad28b370a5751a229dabf72c9e22f2adfac&r=d2a0aad28b370a5751a229dabf72c9e22f2adfac&format=side&path=/optok.c

So track 1 has arrived, cargo intact.

Now the relatively separate tracks are:

multi-processes (fork and join and wait, and setjmp and longjmp)

a nice, generic device interface. (dream dream dream, and wait for amazon to bring C Interfaces and Implementations)

prototyping the graphics... where's that stupid sd card??!!

[luser- -droog]

I separated names into global and local banks.
Finding all the places that cooked up name objects
was pretty painful. After a week of debugging, it
runs (test.ps) again.