Q: best method for system calls?...
BGB / cr88192
I am considering running code in an interpreter, but may want to keep open
the option of using native processes.
so, here is the issue:
which method is 'best' for the handling of syscalls?...
possible options:
interrupts: convinient, conventional, but "could" be problematic with native
processes (I could likely handle them, but have not fully investigated
this).
"magic jumps", where in the interpreter the jump itself is magic, but in a
process it refers to actual code to initiate a remote call. problem: there
is no "good" way I know of to make these visible to a process.
"fake" DLL (faking a particular DLL), but the issue is that this forces a
PE/COFF based image/loader.
...
comments?...
James
i suggest running defrag
BGB / cr88192
"James" <james@dne> wrote in message
news:4acad4c0$0$1783$afc3...@news.optusnet.com.au...
I fail to see the reason for this comment...
granted, my first message was vague, mostly as I ran out of time suddenly
while writing it.
well, anyways, I have noted that Windows does not give as much control over
processes as would be ideal, so using native processes for running raw code
is likely unworkable (besides, it would make 'sandboxing' problematic, as it
would allow direct access to the real OS).
so, dynamic translation would be needed just to enforce sandboxing, and at
this point, one can easily rewrite interrupt instructions into calls if
needed...
but, for now, probably plain x86 interpreter.
so, for now, I have just decided on using interrupts as the main syscall
mechanism.
granted, loading PE/COFF images will avoid this issue, since for PE/COFF I
will probably either use fake DLLs or stub DLLs (mostly which will wrap the
interrupt instructions).
lacking anything better, for now I am considering 'int 0xC4'...
James
yep
Rod Pemberton
news:hadv48$5e9$1...@news.albasani.net...
> so, here is the issue:
> I am considering running code in an interpreter, but may want to keep open
> the option of using native processes.
>
> so, here is the issue:
> which method is 'best' for the handling of syscalls?...
>
> possible options:
>
> interrupts: convinient, conventional, but "could" be problematic with
> native
> processes (I could likely handle them, but have not fully investigated
> this).
>
Other than not being able to install your own interrupt handler, I'm not
sure what the problems are that you mention.
In your post after this, you said:
> lacking anything better, for now I am considering 'int 0xC4'...
The size of the interrupt table can be limited. So, my first concern here
is that that the "large" value of 0xC4 might be outside the installed
interrupt range of an OS. You'll have to check the size of Window's or
Linux's interrupt table limit. Typically, int 0x34 to int 0x6F should be
free and installed, as well as others below int 0x80.
My second concern is that you may be using v86 or VME. In which case, using
0xCC, e.g., the debug breakpoint version of int 0x03, might offer more
control than int 0xC4. Also, there is also no way to prevent int 03 from
being within the interrupt table.
> "magic jumps", where in the interpreter the jump itself is magic, but in a
> process it refers to actual code to initiate a remote call. problem: there
> is no "good" way I know of to make these visible to a process.
>
It seems NTVDM, DOSBox, Virtual PC, etc. have special instruction sequences
which function as an escape from normal instruction emulation. So, this
method probably works fairly well.
Virtual PC uses 0x0f 0x3f xx xx
NTVDM uses 0xc4 0xc4 xx xx
DOSBox uses 0xfe 0x38
> "fake" DLL (faking a particular DLL), but the issue is that this forces a
> PE/COFF based image/loader.
>
Not sure I like this one...
Rod Pemberton
BGB / cr88192
"Rod Pemberton" <do_no...@nohavenot.cmm> wrote in message
news:haf39n$i9j$1...@aioe.org...
> "BGB / cr88192" <cr8...@hotmail.com> wrote in message
> news:hadv48$5e9$1...@news.albasani.net...
>> so, here is the issue:
>> I am considering running code in an interpreter, but may want to keep
>> open
>> the option of using native processes.
>>
>> so, here is the issue:
>> which method is 'best' for the handling of syscalls?...
>>
>> possible options:
>>
>> interrupts: convinient, conventional, but "could" be problematic with
>> native
>> processes (I could likely handle them, but have not fully investigated
>> this).
>>
>
> Other than not being able to install your own interrupt handler, I'm not
> sure what the problems are that you mention.
>
this is the main issue actually.
I am not certain if Linux and Windows provide the ability for user processes
to capture interrupts in other processes...
> In your post after this, you said:
>
>> lacking anything better, for now I am considering 'int 0xC4'...
>
> The size of the interrupt table can be limited. So, my first concern here
> is that that the "large" value of 0xC4 might be outside the installed
> interrupt range of an OS. You'll have to check the size of Window's or
> Linux's interrupt table limit. Typically, int 0x34 to int 0x6F should be
> free and installed, as well as others below int 0x80.
>
> My second concern is that you may be using v86 or VME. In which case,
> using
> 0xCC, e.g., the debug breakpoint version of int 0x03, might offer more
> control than int 0xC4. Also, there is also no way to prevent int 03 from
> being within the interrupt table.
>
possibly, but by this point, I was like, "oh well, whatever, I am running in
an interpreter so I will just do whatever...".
as for INT 3, yes, this one can be used, but using it as a "functional"
interrupt would essentially conflict with its usability as a breakpoint
interrupt.
>> "magic jumps", where in the interpreter the jump itself is magic, but in
>> a
>> process it refers to actual code to initiate a remote call. problem:
>> there
>> is no "good" way I know of to make these visible to a process.
>>
>
> It seems NTVDM, DOSBox, Virtual PC, etc. have special instruction
> sequences
> which function as an escape from normal instruction emulation. So, this
> method probably works fairly well.
>
> Virtual PC uses 0x0f 0x3f xx xx
> NTVDM uses 0xc4 0xc4 xx xx
> DOSBox uses 0xfe 0x38
>
yes, I have not considered this as such, but this may be better than what I
had considered (essentially, jumps to a "magic" address range which was
trapped...).
in an interpreter, it would just be special logic.
in an OS, it would likely involve finding a way to handle remote page
faults.
now, as for making the call itself special, this may actually be better,
since it does not require address-range trapping, and is unlikely to be
stumbled on "accidentally..."
granted, it would require some level of virtualization in order to work, but
alas, I am using an interpreter anyways, so it does not so much matter...
>> "fake" DLL (faking a particular DLL), but the issue is that this forces a
>> PE/COFF based image/loader.
>>
>
> Not sure I like this one...
>
not sure I do either, since it is a bit much overhead to force.
granted, with any option, this may be what is used in the case of PE/COFF
images (although, another option, namely a "stub DLL", becomes possible as
well), just I don't want to force everything to be PE/COFF...
for my early testing, I had been using '.COM' like images (basically, no
headers, loaded to a particular address).
I had wanted to retain a strategy which "still works" with this approach,
where I was left with a few options:
interruprts;
jump tables;
...
in the jump-table case, the pointer to the table would be loaded into a
small header just before the image.
for example:
Page 0 (VA 0x00000000 - 0x00000FFF), reserved, NULL page.
Page 1 (VA 0x00001000 - 0x00001FFF), image header.
Page 2 (VA 0x00002000 - 0x00002FFF), first page of loaded image.
Page 3 ...
typically, at a higher address, I had been placing the stack (in my tests,
generally at 0x00100000), ...
other options exist:
PIC images, which are similar to the above case, but position independent;
PE/COFF images (the same as normal EXE's and DLL's);
COFF objects (linked directly into the address space, this being similar to
how my JIT works with my dynamic assembler).
the most convinient means for interfacing my interpreter to my existing
dynamic assembler will likely be via the use of COFF objects, although this
would require including a dynamic linker into the interpreter.
I guess said linker will do 2 tasks:
linking COFF objects;
loading PE/COFF images.
'.COM' does not need loading, since this is just mapped into the image and
jumped to...
BGB / cr88192
"Rod Pemberton" <do_no...@nohavenot.cmm> wrote in message
news:haf39n$i9j$1...@aioe.org...
> news:hadv48$5e9$1...@news.albasani.net...
>> so, here is the issue:
>> I am considering running code in an interpreter, but may want to keep
>> open
>> the option of using native processes.
>>
>> so, here is the issue:
>> which method is 'best' for the handling of syscalls?...
>>
<snip>
>
> It seems NTVDM, DOSBox, Virtual PC, etc. have special instruction
> sequences
> which function as an escape from normal instruction emulation. So, this
> method probably works fairly well.
>
> Virtual PC uses 0x0f 0x3f xx xx
> NTVDM uses 0xc4 0xc4 xx xx
> DOSBox uses 0xfe 0x38
>
ok, here is what I have done here:
I have gone and "borrowed" the 0F3F case, as above, but I was unable to find
a valid opcode name for it.
so, for my uses, I have dubbed it 'intxt', where it basically operates like
a 16-bit 'int' instruction.
I have then gone and reserved a vector range of 0x1000-0x3FFF for special
handlers, with values outside this range presently throwing a #UD exception.
this will then allow registering handlers, and would be less effort to work
with than using an interrupt and EAX to select the handler (as well, it
allows EAX to be free to pass values, if needed...).
I may go and further divide this range, with 0x1000-0x1FFF being for
statically-assigned handlers, and 0x2000-0x3FFF for dynamic handlers (very
likely, for trampolined functions and methods).
that or I could make 0x1000-0x3FFF all for static, and allow 0x4000-0x7FFF
for dynamic, leaving 0x8000-0xFFFF as #UD...
I am not sure of a good way to "sandbox" using this feature though, since it
itself is inherently a means of bypassing the sandbox (it would depend
purely on the handlers to ensure that security is observed...).
one possibility is to use pseudo-random assignment of dynamic handlers, and
assuming the use of dynamic handlers for any "secure" features, as well as a
static handler to get the index of one of these dynamic handlers (which
could verify security). that or only allowing secure handlers by accessing
via interpreter-provided thunks, for which these will only be provided to
"worthy" code.
as such, static handlers would either:
A, provide their own security checking;
B, implement facilities which are not readily exploitable.
another security option could be to selectively disable the operation of
this opcode (in which case it would always be #UD or #GP).
reason all this is such a concern:
I may want to be able to do 'Genetic Programming' in the interpreter as
well, and I am well aware of GP's tendency to "exploit" any holes or bugs in
one's interpreter (it is an odd effect to see in practice, as the evolver
starts generating rather odd fragments with even more odd results). so, I
have been partially trying to "harden" the interpreter against this case.
(it is worth noting that my prior fiction story, referenced on
comp.programming, was partly motivated by some of my past experiences with
GP evolvers...).
of note: a static handler with EAX giving a method ID could also be used
here (allowing 32-bit indices to handlers...). this could also have some
uses...
one downside:
this opcode will not be supported by NASM, which is inconvinient for my
testing...
(as is, using my assembler for testing would require adding ability to
produce flat images, or using PIC...).
could add it though I guess, but I am not sure if any of the other NASM
dev's would care about this one (given it is not a 'real' opcode, ...).
Rod Pemberton
>
> "Rod Pemberton" <do_no...@nohavenot.cmm> wrote in message
> news:haf39n$i9j$1...@aioe.org...
> > "BGB / cr88192" <cr8...@hotmail.com> wrote in message
> > news:hadv48$5e9$1...@news.albasani.net...
> >> so, here is the issue:
> >> I am considering running code in an interpreter, but may want to keep
> >> open
> >> the option of using native processes.
> >>
> >> so, here is the issue:
> >> which method is 'best' for the handling of syscalls?...
> >>
>
> <snip>
>
> >
> > It seems NTVDM, DOSBox, Virtual PC, etc. have special instruction
> > sequences
> > which function as an escape from normal instruction emulation. So, this
> > method probably works fairly well.
> >
> > Virtual PC uses 0x0f 0x3f xx xx
> > NTVDM uses 0xc4 0xc4 xx xx
> > DOSBox uses 0xfe 0x38
> >
>
> find a valid opcode name for it.
>
> so, for my uses, I have dubbed it 'intxt', where it basically operates
> like a 16-bit 'int' instruction.
> I am not sure of a good way to "sandbox" using this feature though, since
> it itself is inherently a means of bypassing the sandbox (it would
> depend purely on the handlers to ensure that security is observed...).
Sleepy... I'm really not sure if I'm answering the correct question. You
could require a "magic" value in some register like eax or memory location,
prior to instruction use. If it's not there, then the instruction wasn't
legitimately executed.
> this opcode will not be supported by NASM, which is inconvinient for my
> testing...
> (as is, using my assembler for testing would require adding ability to
> produce flat images, or using PIC...).
>
%macro intxt 0
db 0x0f
db 0x3f
%endmacro
If intxt is only the two bytes, I think that should work... groggy.
Usually, I throw an underscore on them somewhere. If the sequence is four
bytes, well, I'm not sure if you can insert parameters into the instruction
stream with NASM... It might be possible. NASM's preprocessor has lots of
undocumented funtionality. So, I'd probably try (untested) something like
this:
%macro intxt 2
db 0x0f
db 0x3f
db %1
db %2
%endmacro
NASM will probably complain that %1 and %2 are unknown. In which case, you
may be able to use an "assign", "equ", or "define" to come up with a
workaround somehow...
Rod Pemberton
BGB / cr88192
"Rod Pemberton" <do_no...@nohavenot.cmm> wrote in message
yeah, like maybe a security token?... (such as a random magic value only
known to 'secure' code).
I had considered probably locking based on virtual address (maybe by putting
info in the page tables or similar?...).
this would then be vaguely similar to the x86 "ring" notion, but would not
depend on segmentation.
R W X: Read, Write, Execute
so:
bit 56: X-U/S
bit 57: W-U/S
bit 58: R-U/S
bit 59: RSV
bit 60/61: BPL
bit 62: RSV
bit 63: EXB / NX
this would then use a bi-level scheme:
0=User (Ring 3)
1=Supervisor (Ring 0/1/2)
(Note: inverse of other U/S flag).
R/W/X control access:
setting them prevents their access for Ring 3, but allows it for other rings
(R/W, U/S, and NX, effect all access).
BPL is different, as it will essentially allow overriding the CPL, meaning
that if EIP/RIP falls within this page, it will be temporarily boosted to
the given level (it will not restrict the CPL though, allowing Ring3 to be a
no-op case).
it would likely be bit-reversed:
0=Ring 3 (or no-op)
1=Ring 2
2=Ring 1
3=Ring 0
this could allow localized transitions to a kernel-like access-level,
without having to use interrupts or gates.
then again, X may also need 2 bits is Rings 1 and 2 are to have much
restriction.
hmm...
I can then make access to only certain functions accessible from Ring3 (sort
of like with interrupts, ...).
yes, ok...
%macro intxt 1
db 0x0f, 0x3f
dw %1
%endmacro
(or similar) should probably work...
since, in my case, I am interpreting the parameter as a single 16-bit unit
(much like an "extra large" interrupt).
I may make it so that for the 0-255 range, it is equivalent to an 'int'
instruction, thus making it 'special' outside the 0-255 range.
I am considering making the whole 32768-65535 range off-limits.
likely, attempting to call a bad handler will be made a #GP exception,
rather than #UD (which may be inappropriate since my interpreter will
actually parse the opcode as an instruction regardless of whether or not it
is valid...).