[Proposal] JIT, exec core, threads, and architectures
Leopold Toetsch
- all JIT platforms *except* i386 have a register reserved for the
runtime interpreter
- Parrot register addressing is done relative to that CPU register
- that would allow to reuse the JITted code for different threads
aka interpreters
- but because of i386 is using absolute addresses that's not done
- the latter point also is the cause for zillions of text relocations
currently needed for the EXEC/i386 run core. I can't imagine that
this will speed up program start :)
Proposal:
* we mandate that JIT code uses interpreter-relative addressing
- because almost all platforms do it
- because some platforms just can't do anything else
- and of course to avoid re-JITting for every new thread
* src/jit.c calls some platform interface functions, which copy between
Parrot registers and CPU registers. These are called at the begin
and end of JITted code sections and are now based on absolute
memory addresses.
This should be changed to use offsets. To accomodate platforms that
have the interpreter cached in a CPU register, I'm thinking of the
following interface:
MACRO int Parrot_jit_emit_get_base_reg_no(jit_info *pc)
// possibly emit code and return register number of base pointer
// this register should be one of the scratch registers
// this must be a macro used as C<reg = foo(pc);>
Parrot_jit_emit_mov_MR(,..int base_reg_no, size_t offset, int src_reg_no)
... other 3 move functions similar
Register addressing is done relative to the base pointer, which currently
is ®_INT(0) or the interpreter, but that might change.
The code to load this register will be emitted just before the actual
register moves are done. It's currenlty a noop for all but i386, which
would need one instruction "mov 16(%ebp), %eax"
* Currently all platforms are using homegrown defines to calculate the
register offset, some of these are even readable ;)
To get rid of that, we should provide a set of macros that calculate
the register offset relative to the base pointer.
REG_OFFS_INT(x) // get offset for INTVAL reg no x
REG_OFFS_STR(x)
...
* Implementation
First the framework should be implemented.
To allow some transition time the old register move semantics remain
for some time. Depending on the defined()ness of
C<Parrot_jit_emit_get_base_reg_no> the new code will be used.
Comments welcome
leo
Jeff Clites
> Proposal:
> * we mandate that JIT code uses interpreter-relative addressing
> - because almost all platforms do it
> - because some platforms just can't do anything else
> - and of course to avoid re-JITting for every new thread
FYI, the PPC JIT does already do parrot register addressing relative to
the interpreter pointer, which as you said is already in a CPU
register. This is actually less instructions than using absolute
addressing would require (one rather than three).
We do still re-JIT for each thread on PPC, though we wouldn't have to
(just never changed it to not). But, we use this currently, because
there is one issue with threads: With a thread, you don't start from
the "beginning" of the JITted code segment, but rather you need to
start with a specific Parrot function call, somewhere in the middle.
But you can't just jump to that instruction, because it would not have
the setup code needed when entering the JITted section. So currently,
we use a technique whereby the beginning of the JITted section has,
right after the setup code, a jump to the correct starting address--in
the main thread case, this is just a jump to the next instruction
(essentially a noop), but in the thread case, it's a jump to the
function which the thread is going to run. So right now the JITted code
for a secondary thread differs by one instruction from that for the
main thread. We'll need to work out a different mechanism for handling
this--probably just a tiny separate JITted section to set things up for
a secondary thread, before doing an inter-section jump to the right
place.
JEff
Leopold Toetsch
> On Oct 14, 2004, at 12:10 PM, Leopold Toetsch wrote:
>> Proposal:
>> * we mandate that JIT code uses interpreter-relative addressing
>> - because almost all platforms do it
>> - because some platforms just can't do anything else
>> - and of course to avoid re-JITting for every new thread
> FYI, the PPC JIT does already do parrot register addressing relative to
> the interpreter pointer, which as you said is already in a CPU
> register. This is actually less instructions than using absolute
> addressing would require (one rather than three).
Yes, and not only PPC, *all* but i386.
> We do still re-JIT for each thread on PPC, though we wouldn't have to
> (just never changed it to not).
Doing that or not depending on a specific JIT platform is error prone
and clutters the source code.
> ... But, we use this currently, because
> there is one issue with threads: With a thread, you don't start from
> the "beginning" of the JITted code segment,
This isn't a threading issue. We can always start execution in the
middle of one segment, e.g. after an exception. That's already handled
on almost all JIT platforms and no problem. The code emitted in
Parrot_jit_begin gets the C<cur_opcode *> as argument and has to branch
there, always.
> JEff
leo
Leopold Toetsch
> We do still re-JIT for each thread on PPC, though we wouldn't have to
The real problem that all JIT architectures still have is a different
one: its called const_table and hidden either in the CONST macro or in
syntax like NUM_CONST, which is translated by the jit2h.pl utility.
String, number (and PMC) constants are all addressed in terms of the
compiling interpreter. Basically everywhere, where the exec code adds
text relocations we aren't safe (e.g. load_nc in the PPC jit_emit code).
When we do an eval() e.g. in a loop, we have to create a new constant
table (and recycle it later, which is a different problem). Running such
a compiled piece of code with different threads would currently do the
wrong thing.
Access to constants in the constant table is not only a problem for the
JIT runcore, its a lengthy operation for all code, For a string constant
at PC[i]:
interpreter->code->const_table->constants[PC[i]]->u.string
These are 3 indirections to get at the constants pointer array, and
worse they depend on each other, emitting these 3 instructions on an
i386 stalls for 1 cycle twice (but the compiler is clever and
interleaves other instructions)
For the JIT core, we can precalculate the location of the constants
array and store it in the stack or even in a register (on not so
register-crippled machines like i386). It only needs reloading, when an
C<invoke> statement is emitted.
OTOH it might be better to just toss all the constant table access in
all instructions, except:
set_n_nc
set_s_sc
set_p_pc # alias set_p_kc
This would reduce the interpreter size significantly (compare the size
of core_ops_cgp.o with core_ops_cg.o). The assembler could still allow
all constant variations of opcodes and just translate it.
leo
Jeff Clites
> Jeff Clites <jcl...@mac.com> wrote:
>> ... But, we use this currently, because
>> there is one issue with threads: With a thread, you don't start from
>> the "beginning" of the JITted code segment,
>
> This isn't a threading issue. We can always start execution in the
> middle of one segment, e.g. after an exception. That's already handled
> on almost all JIT platforms and no problem. The code emitted in
> Parrot_jit_begin gets the C<cur_opcode *> as argument and has to branch
> there, always.
I was remembering wrong--we do this on PPC too.
On Oct 16, 2004, at 4:47 AM, Leopold Toetsch wrote:
> String, number (and PMC) constants are all addressed in terms of the
> compiling interpreter.
...
> When we do an eval() e.g. in a loop, we have to create a new constant
> table (and recycle it later, which is a different problem). Running
> such a compiled piece of code with different threads would currently
> do the wrong thing.
The correct constant table depends on the code segment, rather than the
specific interpreter, right? That means that referencing the absolute
address of the const table entry would be correct for JIT code no
matter the executing thread, but getting the const table from the
compiling interpreter is wrong if that interpreter isn't holding a
reference to the corresponding code segment.
> Access to constants in the constant table is not only a problem for
> the JIT runcore, its a lengthy operation for all code, For a string
> constant at PC[i]:
>
> interpreter->code->const_table->constants[PC[i]]->u.string
>
> These are 3 indirections to get at the constants pointer array, and
> worse they depend on each other, emitting these 3 instructions on an
> i386 stalls for 1 cycle twice (but the compiler is clever and
> interleaves other instructions)
>
> For the JIT core, we can precalculate the location of the constants
> array and store it in the stack or even in a register (on not so
> register-crippled machines like i386). It only needs reloading, when
> an C<invoke> statement is emitted.
For PPC JIT, it seems that we are putting in the address of the
specific const table entry, as an immediate.
> OTOH it might be better to just toss all the constant table access in
> all instructions, except:
>
> set_n_nc
> set_s_sc
> set_p_pc # alias set_p_kc
>
> This would reduce the interpreter size significantly (compare the size
> of core_ops_cgp.o with core_ops_cg.o).
Reducing the size is good, but this doesn't overall reduce the number
of accesses to the constant table, just changes which op is doing them.
> The assembler could still allow all constant variations of opcodes and
> just translate it.
For this we'd need a special register to hold the loaded constant, so
that we don't overwrite a register which is in use.
JEff
Leopold Toetsch
>
> On Oct 16, 2004, at 4:47 AM, Leopold Toetsch wrote:
>
>> String, number (and PMC) constants are all addressed in terms of the
>> compiling interpreter.
>
> ...
>
>> When we do an eval() e.g. in a loop, we have to create a new constant
>> table (and recycle it later, which is a different problem). Running
>> such a compiled piece of code with different threads would currently
>> do the wrong thing.
>
>
> The correct constant table depends on the code segment, rather than the
> specific interpreter, right?
Yes.
> ...That means that referencing the absolute
> address of the const table entry would be correct for JIT code no matter
> the executing thread, but getting the const table from the compiling
> interpreter is wrong if that interpreter isn't holding a reference to
> the corresponding code segment.
Thinking more about that I've to admit that my above conclusion is very
likely wrong. When a piece of code gets compiled, we can say that we are
creating a read-only data structure (code, constants, metadata). This
data structure can be shared between different threads and using
absolute addresses for the constants is ok.
Nethertheless we have to create managed objects (a Packfile PMC) so that
we can recycle unused eval-segments. And we have to protect the packfile
dictionary with mutexes, when this managing structure changes i.e. when
new segments gets chained into this list or when they get destroyed.
>> Access to constants in the constant table is not only a problem for
>> the JIT runcore, its a lengthy operation for all code, For a string
>> constant at PC[i]:
>>
>> interpreter->code->const_table->constants[PC[i]]->u.string
So for JIT and prederefed cores that's not a problem.
>> OTOH it might be better to just toss all the constant table access in
>> all instructions, except:
>>
>> set_n_nc
>> set_s_sc
>> set_p_pc # alias set_p_kc
>>
>> This would reduce the interpreter size significantly (compare the size
>> of core_ops_cgp.o with core_ops_cg.o).
>
>
> Reducing the size is good, but this doesn't overall reduce the number of
> accesses to the constant table, just changes which op is doing them.
Not quite, e.g:
set P0["foo"], "bar"
set S0, P0["foo"]
are 3 accesses to the constant table. It would be
set S1, "foo"
set S2, "bar"
set P0[S1], S2
set S0, P0[S1]
with 2 accesses as long as there is no pressure on the register allocator.
>> The assembler could still allow all constant variations of opcodes and
>> just translate it.
>
> For this we'd need a special register to hold the loaded constant, so
> that we don't overwrite a register which is in use.
No, just the registers we have anyway,
> JEff
leo
Jeff Clites
> Jeff Clites wrote:
>> On Oct 16, 2004, at 4:47 AM, Leopold Toetsch wrote:
>
> Nethertheless we have to create managed objects (a Packfile PMC) so
> that we can recycle unused eval-segments.
True, and some eval-segments are "done" as soon as they run (eval "3 +
4"), whereas others may result in code which needs to stay around (eval
"sub {....}"), and even in the latter case not _all_ of the code
generated in the eval would need to stay around. It seems that it may
be hard to determine what can be recycled, and when.
> And we have to protect the packfile dictionary with mutexes, when this
> managing structure changes i.e. when new segments gets chained into
> this list or when they get destroyed.
Yes, though it's not clear to me if all eval-segments will need to go
into a globally-accessible dictionary. (e.g., it seems the "3 + 4" case
above would not.)
>>> OTOH it might be better to just toss all the constant table access
>>> in all instructions, except:
>>>
>>> set_n_nc
>>> set_s_sc
>>> set_p_pc # alias set_p_kc
>>>
>>> This would reduce the interpreter size significantly (compare the
>>> size of core_ops_cgp.o with core_ops_cg.o).
>> Reducing the size is good, but this doesn't overall reduce the number
>> of accesses to the constant table, just changes which op is doing
>> them.
>
> Not quite, e.g:
>
> set P0["foo"], "bar"
> set S0, P0["foo"]
>
> are 3 accesses to the constant table. It would be
>
> set S1, "foo"
> set S2, "bar"
> set P0[S1], S2
> set S0, P0[S1]
>
> with 2 accesses as long as there is no pressure on the register
> allocator.
Sure, but you can do this optimization today--narrowing it down to just
those 3 ops isn't required. But it only helps if local re-use of the
same constants is frequent, and it may not be. (But still, it's a good
optimization for a compiler to implement--it just may not have a huge
effect.)
Also, there's some subtlety. This:
set S1, "foo"
set S2, "foo"
isn't the same as:
set S1, "foo"
set S2, S1
but rather:
set S1, "foo"
clone S2, S1
since 'set' copies in the s_sc case. (That's not a problem, just
something to keep in mind.)
>>> The assembler could still allow all constant variations of opcodes
>>> and just translate it.
>> For this we'd need a special register to hold the loaded constant, so
>> that we don't overwrite a register which is in use.
>
> No, just the registers we have anyway,
For PIR yes, but the PASM assembler can't know for sure what register
would be safe to use--the code could be using its own obscure calling
conventions.
JEff
Leopold Toetsch
> On Oct 17, 2004, at 3:18 AM, Leopold Toetsch wrote:
>> Nethertheless we have to create managed objects (a Packfile PMC) so
>> that we can recycle unused eval-segments.
> True, and some eval-segments are "done" as soon as they run (eval "3 +
> 4"), whereas others may result in code which needs to stay around (eval
> "sub {....}"), and even in the latter case not _all_ of the code
> generated in the eval would need to stay around. It seems that it may
> be hard to determine what can be recycled, and when.
Well, not really. As long as you have a reference to the code piece,
it's alive.
>> And we have to protect the packfile dictionary with mutexes, when this
>> managing structure changes i.e. when new segments gets chained into
>> this list or when they get destroyed.
> Yes, though it's not clear to me if all eval-segments will need to go
> into a globally-accessible dictionary. (e.g., it seems the "3 + 4" case
> above would not.)
It probably depends on the generated code. If this code creates globals
(e.g. Sub PMCs) it ought to stay around.
[ toss constant op variations ]
> For PIR yes, but the PASM assembler can't know for sure what register
> would be safe to use--the code could be using its own obscure calling
> conventions.
PASM would need rewriting to only use the available ops, basically.
> JEff
leo
Jeff Clites
> Jeff Clites <jcl...@mac.com> wrote:
>> On Oct 17, 2004, at 3:18 AM, Leopold Toetsch wrote:
>
>>> Nethertheless we have to create managed objects (a Packfile PMC) so
>>> that we can recycle unused eval-segments.
>
>> True, and some eval-segments are "done" as soon as they run (eval "3 +
>> 4"), whereas others may result in code which needs to stay around
>> (eval
>> "sub {....}"), and even in the latter case not _all_ of the code
>> generated in the eval would need to stay around. It seems that it may
>> be hard to determine what can be recycled, and when.
>
> Well, not really. As long as you have a reference to the code piece,
> it's alive.
Yes, that's what I meant. In the case of:
$sum = eval "3 + 4";
you don't have any such reference. In the case of:
$sub = eval "sub { return 7 }";
you do. In the case of:
$sub = eval "3 + 4; sub { return 7 }";
you've got a reference to the sub still, but the "3 + 4" code is no
longer reachable, so wouldn't need to stay around.
But it's possible that Parrot won't be able to tell the difference, and
will have to keep around more than is necessary.
>>> And we have to protect the packfile dictionary with mutexes, when
>>> this
>>> managing structure changes i.e. when new segments gets chained into
>>> this list or when they get destroyed.
>
>> Yes, though it's not clear to me if all eval-segments will need to go
>> into a globally-accessible dictionary. (e.g., it seems the "3 + 4"
>> case
>> above would not.)
>
> It probably depends on the generated code. If this code creates globals
> (e.g. Sub PMCs) it ought to stay around.
Yes, that's what I meant by not all--some yes, some no.
JEff