Spectrem-Dr: Announcing the worlds first dynamically recompiled Spectrum emulator....
Andrew Davidson
Anyway, seeing as Spectrem was sloooooow I thought I'd start again and write
a nice
fast version and here it is, in its totally buggy glory.
The reason I'm telling you all about it is that I'm just about fed up with
trying to patch it up all by myself and I'd really appreciate the knowledge
and comments of the Z80 coders around here...
Currently it sucks. It runs very very little perfectly (so far I've found
Lancelot runs spot on and Feud runs with some graphical glitches). Most of
this is due to horrible self-modifying code which every Spectrum program
seems to use in vast wodges... I was wondering if anyone could point out
common ways this was done to improve my handling of all the self-modifying
stuff (Currently all I do is monitor immediate opcode bytes/words and
self-modify the dynamically generated x86 machine code if they are changed).
This would be dead neat.
On the up side I've managed to smoothly (constant 50fps framerate) emulate a
Speccy 48k with a 49mhz cpu on my P2-266... good for a laugh if very little
else.
So if some of you chappies would be kind enough to mosey on over to my
Spectrem-Dr page and download the source I'd be most grateful. (It should
compile fine in djgpp with the allegro library). Please bear in mind that
the code is totally unoptimised at the moment and that, currently, it will
only run on a Pentium upwards with a Vesa2 compliant graphics card (unless
you want to recompile the source, just change the VIDMODE #define in
drspec.h to GFX_AUTODETECT for non vesa2 boards and comment out the
profiling code for 486 or less cpus). The Z80 can also be overclocked by
fiddling with the Z80->ifreq variable.
Oh yeah, if you don't want to/can't compile it then there's a precompiled
binary on the site to...
You can get them both at http://www.lemure.freeserve.co.uk/spectremdr.html
Thanks for your time with this,
Andrew
kio
>
> I have WAY to much time on my hands, eh? :)
>
>
> Currently it sucks. It runs very very little perfectly (so far I've found
> Lancelot runs spot on and Feud runs with some graphical glitches). Most of
> this is due to horrible self-modifying code which every Spectrum program
> seems to use in vast wodges... I was wondering if anyone could point out
> common ways this was done to improve my handling of all the self-modifying
> stuff (Currently all I do is monitor immediate opcode bytes/words and
> self-modify the dynamically generated x86 machine code if they are changed).
> This would be dead neat.
>
> On the up side I've managed to smoothly (constant 50fps framerate) emulate a
> Speccy 48k with a 49mhz cpu on my P2-266... good for a laugh if very little
> else.
Hi !
i think it's very noteworthy that you try to compile the z80
instructions into 586 assembler code ad hok. but i'd like to point out
that this is probably no good.
as you have pointed out, there is a lot of self-modifying code out
there. to be precise: you must take into account that each and every
write access to memory might change an already precompiled z80
instruction. even pushing data on the stack!
so you need to examine every write access to memory and invalidate
precompiled instructions as required. THIS TAKES A LOT OF TIME.
IT IS COMPLICATED.
and: you produce a large array of precomiled code executable by the 586
processor. THIS FREQUENTLY LEADS TO CACHE MISSES!
from my experience the z80 emulation even as an interpreter and
entierely written in C takes LESS THAN 10% OF THE PROCESSING POWER.
another 10% go to the sound interface and the rest is screen update, if
you try to make 50 screens a second, potentially with a huge
magnification scale.
my z80 interpreter is approx. 32k in size so it fits sneakly into the
1st level cache of the cpu and it runs like fire. an interpreter is
simpler and it is not neccessarily slower!
you definitely should focus on graphics speed and keep the z80 engine
simple.
> So if some of you chappies would be kind enough to mosey on over to my
> Spectrem-Dr page and download the source I'd be most grateful. (It should
> compile fine in djgpp with the allegro library). Please bear in mind that
> the code is totally unoptimised at the moment and that, currently, it will
> only run on a Pentium upwards with a Vesa2 compliant graphics card (unless
> you want to recompile the source, just change the VIDMODE #define in
> drspec.h to GFX_AUTODETECT for non vesa2 boards and comment out the
> profiling code for 486 or less cpus). The Z80 can also be overclocked by
> fiddling with the Z80->ifreq variable.
ahem .. and you should think about portability.
and why do you try to write an emulator which runs on 16 MHz+ systems if
you require a Pentium ?
... kio !
--
k...@odn.de (private)
Andrew Davidson
kio <k...@odn.de> wrote in message news:373D64A3...@odn.de...
> Andrew Davidson wrote:
> >
> > I have WAY to much time on my hands, eh? :)
> >
> [snip]
> >
>
> i think it's very noteworthy that you try to compile the z80
> instructions into 586 assembler code ad hok. but i'd like to point out
> that this is probably no good.
>
> as you have pointed out, there is a lot of self-modifying code out
> there. to be precise: you must take into account that each and every
> write access to memory might change an already precompiled z80
> instruction. even pushing data on the stack!
>
> so you need to examine every write access to memory and invalidate
> precompiled instructions as required. THIS TAKES A LOT OF TIME.
>
> IT IS COMPLICATED.
Well, it certainly is complicated but it doesn't take very much time. I
handle self-modifying Z80 code using self-modifying x86 code so there really
isn't that much of a speed hit (maybe 40-50 cycles tops when writing to
memory).
> and: you produce a large array of precomiled code executable by the 586
> processor. THIS FREQUENTLY LEADS TO CACHE MISSES!
Yup. You've got me bang to rights there but, as I said in the original post,
it isn't optimised yet. The final optimised version won't recompile old code
thus taking up a load less space. And anyway, the cache on a 586+ is plenty
big enough to stick most of the code I generate in the level 1 cache as any
given block is normally no more than a few kilobytes in size. Remember that,
after optimisation, the execution will remain within the machine code as
long as possible, rather than popping out to the C code. In fact, one of the
beauties of dynamically recompiled code is that it is MORE likely to fit in
the cache. Hand optimised machine code will almost certainly be smaller than
compiler generated C code.
> from my experience the z80 emulation even as an interpreter and
> entierely written in C takes LESS THAN 10% OF THE PROCESSING POWER.
> another 10% go to the sound interface and the rest is screen update, if
> you try to make 50 screens a second, potentially with a huge
> magnification scale.
On rougly how powerful a CPU? I'm doing a line by line redraw, rather than
screen by screen which left the Z80 emulation on my (deliberately
unoptimised) C implementation at between 20% and 40% and my unoptimised
dynarec implementation at around 5%-7% on a Pentium 2.
> my z80 interpreter is approx. 32k in size so it fits sneakly into the
> 1st level cache of the cpu and it runs like fire. an interpreter is
> simpler and it is not neccessarily slower!
The level 1 code cache on a P2 is only 16k... I THINK the code cache on a
Pentium is a mere 8k but don't quote me on that. So it wouldn't fit. And
anyway, with an interpreted emulation EVERY opcode you have to go and do
other stuff (fetch and decode opcodes, for starters which will need lookup
tables or switch jump tables which are on the sizeable side). An interpreter
is far far simpler, I'll admit. But I doubt very much if it is faster. The
only way you'd get close is to write it pretty exclusively in hand optimised
assembler.
I plan on using some of the assembler Z80 emulators out there by plugging
them into my current implementation of Spectrem-Dr and comparing cpu usage.
Currently I think there will be very little in it and that they'll probably
get the upper hand but, as I've already said a thousand times. My emulator
is unoptimised. Their's are optimised to death.
And when it comes to overclocking. Well... I know X128 at times struggles to
provide 4x overclock on my P2-266 without the little speed warning pixel
coming on. I've already got a comfortable 14x overclock using my
implementation. Not really much use for a Spectrum emulator (although I
can't wait to have a go at some of the freescape games at this speed) but I
don't intend to use it purely for Spectrum emulation.
> you definitely should focus on graphics speed and keep the z80 engine
> simple.
Should I heck! :) Graphics on a Speccy is dead simple. I couldn't squeeze
all that much more speed out of the current asm implementation really.
> > So if some of you chappies would be kind enough to mosey on over to my
> > Spectrem-Dr page and download the source I'd be most grateful. (It
should
> > compile fine in djgpp with the allegro library). Please bear in mind
that
> > the code is totally unoptimised at the moment and that, currently, it
will
> > only run on a Pentium upwards with a Vesa2 compliant graphics card
(unless
> > you want to recompile the source, just change the VIDMODE #define in
> > drspec.h to GFX_AUTODETECT for non vesa2 boards and comment out the
> > profiling code for 486 or less cpus). The Z80 can also be overclocked by
> > fiddling with the Z80->ifreq variable.
>
> ahem .. and you should think about portability.
> and why do you try to write an emulator which runs on 16 MHz+ systems if
> you require a Pentium ?
Because I only stuck the emulator up on my website to ask for peoples help.
Obviously I'll strip out all the 486+ instructions from the code and tart up
the screen redraw before I release a fully functional version. As it is,
people without CPUs that can handle the rdtsc operation aren't of much use
to me as I can't find out how intensively the Z80 is eating up their CPU
time. At least until I write a profiler which doesn't require that op.
Andrew
lo...@my.sig
: I have WAY to much time on my hands, eh? :)
Re: Spectrem-Dr: Announcing the worlds first dynamically recompiled Spectrum
emulator....
Making claims like that though....
Tut tut tut...
I hate to break this to you, but yours is NOT the first dynamically compiled
spectrum emulator. There's been one available for the QL for over two years.
It's called ZMhT (Zx eMulator high Technology) by Ergon developments of
Spain.
--
| |What to do if you find yourself stuck in a crack|
|u5...@teach.cs.keele.ac.uk|in the ground beneath a giant boulder, which you|
| |can't move, with no hope of rescue. |
| Andrew Halliwell |Consider how lucky you are that life has been |
| Finalist in:- |good to you so far... |
| Computer Science | -The BOOK, Hitch-hiker's guide to the galaxy.|
-----------------------------------------------------------------------------
|GCv3.12 GCS>$ d-(dpu) s+/- a C++ US++ P L/L+ E-- W+ N++ o+ K PS+ w-- M+/++|
|PS+++ PE- Y t+ 5++ X+/X++ R+ tv+ b+ DI+ D+ G e>e++ h/h+ !r!| Space for hire|
Joe Mackay
Twiglet before excreting the following via fingers:
>In his obvious haste, Andrew Davidson <and...@lemure.freeserve.co.uk> babbled
>thusly:
>: I have WAY to much time on my hands, eh? :)
>Re: Spectrem-Dr: Announcing the worlds first dynamically recompiled Spectrum
>emulator....
>Making claims like that though....
>Tut tut tut...
>I hate to break this to you, but yours is NOT the first dynamically compiled
>spectrum emulator. There's been one available for the QL for over two years.
Well over, in fact. It's mentioned in the penultimate YS from 1993 (IIRC).
I wonder how it copes with self-modifying code, though?
--
+----------------------- Joe Mackay ------------------------+
| joe dot mackay at lineone dot net | ICQ: 28479458 |
|"Basically it's crap": http://web.ukonline.co.uk/joe.moore2|
+-----------------------------------------------------------+
If the Truth is out there, what the hell have we got in here???
Andrew Davidson
<lo...@my.sig> wrote in message news:7hk6na$4hb$5...@cfs2.kis.keele.ac.uk...
> In his obvious haste, Andrew Davidson <and...@lemure.freeserve.co.uk>
babbled thusly:
> : I have WAY to much time on my hands, eh? :)
>
> Re: Spectrem-Dr: Announcing the worlds first dynamically recompiled
Spectrum
> emulator....
>
> Making claims like that though....
>
> Tut tut tut...
>
> I hate to break this to you, but yours is NOT the first dynamically
compiled
> spectrum emulator. There's been one available for the QL for over two
years.
>
> Spain.
Oops :)
Have you got a homepage or anything? I'd be dead interested to take a look
at that...
Thomas Harte
> screen by screen which left the Z80 emulation on my (deliberately
> provide 4x overclock on my P2-266 without the little speed warning pixel
> coming on. I've already got a comfortable 14x overclock using my
> implementation. Not really much use for a Spectrum emulator (although I
Scanline by scanline correct at 14x? So whats that you have? A 700Hz
monitor?
-Thomas
Andrew Davidson
Thomas Harte <T.H...@btinternet.com> wrote in message
news:373DFF6D...@btinternet.com...
> > On rougly how powerful a CPU? I'm doing a line by line redraw, rather
than
> > screen by screen which left the Z80 emulation on my (deliberately
struggles to
> > provide 4x overclock on my P2-266 without the little speed warning pixel
> > coming on. I've already got a comfortable 14x overclock using my
> > implementation. Not really much use for a Spectrum emulator (although I
>
> monitor?
Of course not! (That'd be nice though... :)
14x CPU overclock, not 14x Spectrum overclock. In effect it's like having a
Speccy with a 49mhz cpu (3.5hmz being the standard Z80a clockspeed in your
normal Spectrum). So you simply execute 224*14 or 3136 cycles per scanline
instead of the usual 224.
Andrew
kio
[snip - quoted as needed...]
Hoops ...
you wrote a lot, i replied a lot and i got a lot back...
i'll try to reply without quoting your message too much.
first:
i have not visited your home page. i miss the required platform to give
your emu a try! all i say is general speaking, not specific to your
programme!
speed:
please distinguish between the speed benefits you get from precompiling
the z80 instructions into 586 code and those from writing in assembler,
not in C. you get a performance lift of approx. factor 3 in assembler,
depending on your C compiler and your assembler writing skills.
cache hits:
i can tell that, even in the C version, my z80 engine is approx. 32kB in
size - at least it was on the PowerPC. The 486 version seems to be
slightly larger. written in assembler, maybe optimized, it might be
squeezed to something like 16kB. just to compare assembler against
assembler, not against C.
if you roll out the z80 opcode decoding loop by precompiling them, you
get an executable 20 times larger in size (estimated). given a programme
which consists of 16kB program code and rest data, this means approx.
160kB. and 20 bytes/instruction seems to be a fair estimation, since you
must check in each instruction whether the executed memory locations
have changed, do what the opcode should do, adjust flags, increment the
r register and count cpu clock cycles to eventually refresh a scanline
and so on.
since you say your compiled code will fit into the 1st lvl cache you
probably do not cache the entire program.
by the way:
what do you do when the program jumps inside a precompiled opcode, e.g.
at the location $+1 of ld hl,NN? can you handle it?
what do you do if your program runs into the stack area, e.g. executes a
return address push on the stack by a subroutine call. do you handle it?
have you implemented a paged memory model as for the zx128? what happens
if you write to location $3fff? to location $ffff? if you pop a value at
that address from the stack? if you execute it? anyone any other nasty
ideas?
if you handle it all correct in your precompiling emu, you've done a
good job.
>> from my experience the z80 emulation even as an interpreter and
>> entierely written in C takes LESS THAN 10% OF THE PROCESSING POWER.
>> another 10% go to the sound interface and the rest is screen update, if
>> you try to make 50 screens a second, potentially with a huge
>> magnification scale.
>
>On rougly how powerful a CPU?
this does not matter. cpu power tells us only, how much cpu time is left
or missing... the ratio z80--sound--screen will be the same. (though in
reality, i drop screen updates if time is short)
>And when it comes to overclocking. Well... I know X128 at times struggles to
>provide 4x overclock on my P2-266 without the little speed warning pixel
>coming on. I've already got a comfortable 14x overclock using my
>implementation. Not really much use for a Spectrum emulator (although I
>can't wait to have a go at some of the freescape games at this speed) but I
>don't intend to use it purely for Spectrum emulation.
since the time required for screen refresh does NOT increase with higher
and higher emulated z80 cpu clock, this does not mean that your emulator
is 3.5 times ((14/4)) faster than X128. it means, your z80 engine is 3.5
times faster (assuming that both emulators need the same time for all
the other stuff). if the z80 emulation requires 10% of total emulation
time, this means that your emulator is only 35% faster in total...
you may compare your z80 emu agains my 68000 assembler version: it ran
at approx. 100% speed on my Macintosh LC III (16MHz if i'm right). to be
honest with no paged memory, no rom write protection, no r register
emulation and no screen update. Ahhh, the LC III was too slow. THERE a
precompiling emu would have made sense!
>> you definitely should focus on graphics speed and keep the z80 engine
>> simple.
>
>Should I heck! :) Graphics on a Speccy is dead simple. I couldn't squeeze
>all that much more speed out of the current asm implementation really.
assumed that 80% of the emulation time is screen refresh, this is
definitely the best point for optimisation. ok, if you have already done
it, the z80 is a good target for further optimisation.
finish:
if you make screen refresh 10% faster it has the same effect as if you
make your z80 engine 80% faster. that's basically what i wanted to point
out.
q.e.d.
... kio !
p.s.: as you may have noticed, i tried to write in British English ...
--
k...@odn.de (private)
lo...@my.sig
: compiled
:> spectrum emulator. There's been one available for the QL for over two
: years.
:>
:> It's called ZMhT (Zx eMulator high Technology) by Ergon developments of
:> Spain.
: Oops :)
: Have you got a homepage or anything? I'd be dead interested to take a look
: at that...
Not off hand. But you can get more info (and even download a shareware
version from Thierry Godefroys web site).
I can't remember that either, but there's a link to Thierrys page on
http://www.geocities.com/SiliconValley/vista/4807
--
______________________________________________________________________________
|u5...@teach.cs.keele.ac.uk| "Are you pondering what I'm pondering Pinky?" |
| Andrew Halliwell | |
| Finalist in:- | "I think so brain, but this time, you control |
| Computer Science | the Encounter suit, and I'll do the voice..." |
------------------------------------------------------------------------------
|GCv3.12 GCS>$ d-(dpu) s+/- a C++ US++ P L/L+ E-- W+ N++ o+ K PS+ w-- M+/++ |
|PS+++ PE- Y t+ 5++ X+/X++ R+ tv+ b+ DI+ D+ G e>e++ h/h+ !r!| Space for hire |
------------------------------------------------------------------------------
Thomas Harte
> Speccy with a 49mhz cpu (3.5hmz being the standard Z80a clockspeed in your
> normal Spectrum). So you simply execute 224*14 or 3136 cycles per scanline
> instead of the usual 224.
Surely that makes a lot of things not work or look silly? Also : is
this the normal way of doing frameskip in Spectrum emulators? Or is it
more normal to do 312 lines as normal, then do the next n scanlines with
correct interrupt placing and interrupt generation but no screen
copying?
-Thomsa
Andrew Davidson
It does stop most things from working, yes. As is the case with early PC
games, the Spectrum coders very rarely added in speed throttle code (Which
would be a bit pointless really, given that it was unlikely Zilog would be
releasing a 300mhz Z80 and Sinclair installing it into the next batch of
ZX81s). I have noticed that, if you ramp the speed up high enough, certain
games will crash in weird ways, but there are games that use the halt op to
sync to the vertical retrace which means they will never try to draw more
than 50 screen updates every second and these look pretty neat. (Uridium is
one. Try overclocking the cpu to 4x in X128. It suddenly gets arcade machine
quality smoothness.). Primarily this is an advantage in the games that
really never should have been released on the Spectrum. Driller, for
instance, is one of the early freescape 3d games and makes a snail look like
its on steroids when run on a normal Speccy. However, try X128's 4x
overclock and (assuming your PC can cope with it) it gets a LOT more
playable.
As for frameskip, this doesn't really have that much to do with it, does it?
If you want to emulate frameskip its simplest just to check your current
framerate and, if it's less than 50 increase the frameskip value by a given
variable (depending on how much less than 50 it is). Then you put a line of
code at the start of the screen drawer that basically does exactly what you
said. It checks the current frameskip value and, if its set to anything,
jumps over the draw code every n times. For instance, with a frameskip of 1
you could jump every 1 in 12 screen updates, increasing this value as the
frameskip increases. But yeah, you should definately keep generating
interrupts at the appropriate times.
AndyC
> by the way:
> what do you do when the program jumps inside a precompiled opcode, e.g.
> at the location $+1 of ld hl,NN? can you handle it?
> what do you do if your program runs into the stack area, e.g. executes a
> return address push on the stack by a subroutine call. do you handle it?
> have you implemented a paged memory model as for the zx128? what happens
> if you write to location $3fff? to location $ffff? if you pop a value at
> that address from the stack? if you execute it? anyone any other nasty
> ideas?
> if you handle it all correct in your precompiling emu, you've done a
> good job.
What about instructions that modify the actual opcode itself eg:
res 0,sm_instr
jp c,useBreg
loop ld hl,sm_instr
set 0,(hl) ; change xor b to xor c
...
sminstr xor b
Or instructions which modify the parameter:
ld (branch_to+1),hl
...
branch_to
jp NNNN
(I have to admit I do this quite a lot)
or LDDRs that overwrite the LDDR mid instruction? (rare, but sometimes
useful)
Also do you have to flush the Pentium's cache if you modify the x86
code? I'd have thought that would cause problems in very tight
self-modifying code.
AndyC
Andrew Davidson
kio <k...@odn.de> wrote in message news:373EB855...@odn.de...
> Andrew Davidson wrote:
> [snip - quoted as needed...]
>
> Hoops ...
>
> you wrote a lot, i replied a lot and i got a lot back...
> i'll try to reply without quoting your message too much.
Hmm... I've never been any good at snipping I'm afraid. Sorry if this reply
is a bit long too :)
> speed:
> please distinguish between the speed benefits you get from precompiling
> the z80 instructions into 586 code and those from writing in assembler,
> not in C. you get a performance lift of approx. factor 3 in assembler,
> depending on your C compiler and your assembler writing skills.
Well, technically I'm compiling to 386 code. It just requires a 586 at the
moment because the profiling code uses a 586 instruction (rdtsc). I'm still
unsure as to the performance increase because all I really have to test it
against is my hideous C implementation which was written specifically for
readability, not speed. It's about 5-10 times faster than that but then
that's no contest really. However, the beauty of dynamically recompiled code
is that, the more tstates you try and execute at once, the more 'bang for
your buck' you get as you spend your whole time in the level 1 cache, never
needing to jump out of it to decode the next instruction. Tight Z80 loops
(which the majority of Z80 based code uses as it was almost exclusively
written in assembler or machine code originally) turn into tight native CPU
loops.
> cache hits:
> i can tell that, even in the C version, my z80 engine is approx. 32kB in
> size - at least it was on the PowerPC. The 486 version seems to be
> slightly larger. written in assembler, maybe optimized, it might be
> squeezed to something like 16kB. just to compare assembler against
> assembler, not against C.
Yup, but you still have fetch/decode loop overheads. All that is done only
once in a dynamically recompiled emulation (disregarding self-modifying
code, of course). Now on a PC with interpreted code you're looking at a jump
table of 256*4 or 1024 bytes for each opcode set. With 7 sets thats 7k
straight out of your cache just for jump table storage. Or, using dynamic
recompilation, 0k. And even worse... are you planning on storing all 64k of
Spectrum memory in the cache? If you're interpreting the code you have to
read from this memory between 1 and 4 times EVERY opcode, no matter whether
the opcode accesses memory or not. This is already done with dynamic
recompilation. This is probably the biggest reason for the performance boost
really, as most coders will use registers before memory wherever possible
because of the speed boost to their code.
> if you roll out the z80 opcode decoding loop by precompiling them, you
> get an executable 20 times larger in size (estimated). given a programme
> which consists of 16kB program code and rest data, this means approx.
> 160kB. and 20 bytes/instruction seems to be a fair estimation, since you
> must check in each instruction whether the executed memory locations
> have changed, do what the opcode should do, adjust flags, increment the
> r register and count cpu clock cycles to eventually refresh a scanline
> and so on.
Slightly bigger than 20 bytes actually. Probably closer to 30 for a hyper
optimised bit of code that could handle all the various flags/obscure
registers/etc. Mine has an overhead of 27 bytes for tstate counter, flags,
and scanline refresh check (no r reg but this shouldn't take more than a
couple of bytes). And that's only for opcodes that set EXACTLY the same
flags in x86 code as they do in Z80 code.
> since you say your compiled code will fit into the 1st lvl cache you
> probably do not cache the entire program.
Too damn right :) I doubt very much if someone would write 16k of code that
never jumps/calls/returns/resets/whatever. If they did it'd be a pretty
useless bit of code. Only the block that is currently being executed needs
to be stored in cache memory.
This is also a somewhat extreme example. Very few programs would have used 1
6k of memory for code. They needed that for graphics, game maps, and such
stuff. Even if they do use a lot of memory, remember the 80/20 rule? 80% of
the code is only executed 20% of the time and vice-versa. I simply end up
caching the 20% of the code that is regularly used. Any extreme example will
always fox a dynamically recompiled emulator. Its real strength lies in
being designed to run real world code, rather than the more obscure code
examples.
> by the way:
> what do you do when the program jumps inside a precompiled opcode, e.g.
> at the location $+1 of ld hl,NN? can you handle it?
> what do you do if your program runs into the stack area, e.g. executes a
> return address push on the stack by a subroutine call. do you handle it?
> have you implemented a paged memory model as for the zx128? what happens
> if you write to location $3fff? to location $ffff? if you pop a value at
> that address from the stack? if you execute it? anyone any other nasty
> ideas?
> if you handle it all correct in your precompiling emu, you've done a
> good job.
Heh :) This is the sort of thing I'm after. Right. Currently I can handle
jumping into absolute addresses, albeit with rather a lot of memory
wasteage. The compiler only registers addresses as having been compiled if
they are at the start of the opcode, so code within the opcode will be
registered as uncompiled and handled correctly. However, if you then wrote
to the immediate byte/word you would be unable to execute correctly from
that address as there is currently no support for code so clever that it
modifies the actual opcodes, only for code that modifies immediate values.
If the code reaches the stack area it will try to compile it like normal
code, then try to execute it. This will almost certainly fail as I can't
really think of any legitimate reason for running code in the stack area
other than an error in the program, but I can't think of any emulator that
wouldn't in that situation (only my code will almost certainly do a far more
spectacular job of falling flat on its face).
I've not bothered with paged memory as the emulator is only intended to
handle a 48k Spectrum but, as with most dynarec problems, it can be easily
be solved by simply chucking a few extra kilobytes of x86 memory at it. And
as for my handling of 0x3fff/0xffff... guilty as charged, I'm afraid.
However, when I can be bothered to type out the few bytes of machine code
compiling it will require (the structure is all there. I just need to bother
joining the dots) I plan on implmenting an optional C based memory
read/write funtion to allow neat stuff like contended memory emulation. The
Z80 emulation is intended as a Z80 emulation, not a Spectrum based Z80a/ULA
emulation, and thus I don't want to stick stuff like that in it. You are
right though. I ought to cope with that. Shouldn't be all that hard, though.
>> from my experience the z80 emulation even as an interpreter and
> >> entierely written in C takes LESS THAN 10% OF THE PROCESSING POWER.
> >> another 10% go to the sound interface and the rest is screen update, if
> >> you try to make 50 screens a second, potentially with a huge
> >> magnification scale.
> >
Are you saying you're looking at a 10:10:80 split between cpu, sound, and
screen. Bloody hell but you code damn fast CPU emulations (or really slow
screen redrawers ;). I'd think more like 40:10:50 or something, surely?
[Snipped some stuff about Z80 overclocking]
> since the time required for screen refresh does NOT increase with higher
> and higher emulated z80 cpu clock, this does not mean that your emulator
> is 3.5 times ((14/4)) faster than X128. it means, your z80 engine is 3.5
> times faster (assuming that both emulators need the same time for all
> the other stuff). if the z80 emulation requires 10% of total emulation
> time, this means that your emulator is only 35% faster in total...
Well... not really. As you overclock a dynamically recompiled emulator it
becomes more and more efficient as it can stay inside the emulation inner
loop for a lot longer, performing better and better when compared with an
interpreted emulator which will simply take (original time*overclock value)
time. And anyway, I've already disagreed with your estimation of time taken
on the various tasks. And anyway anyway, I wasn't comparing the speed of the
emulators. I was comparing the speed of the CPUs they could optimally
emulate. It's just that when I get excited I gabble a bit and get very
unclear about certain things... sorry :)
> you may compare your z80 emu agains my 68000 assembler version: it ran
> at approx. 100% speed on my Macintosh LC III (16MHz if i'm right). to be
> honest with no paged memory, no rom write protection, no r register
> emulation and no screen update. Ahhh, the LC III was too slow. THERE a
> precompiling emu would have made sense!
Ahh... well I've got rom write protection in. It's always the first thing I
add as my emulators tend to write all over rom every chance they get before
I start the bugfixing process. This way allows me to see them working as
they really should be...
What do you mean no screen update? No interrupt or no actual emulator screen
update (Which, as you've pointed out, counts for a hell of a lot of
processing time). Mind, good luck to you if you're writing emulators on a
16mhz cpu. Nightmare ;). That's the other reason for dynamically recompiled
emulators. You can run a graphically detailed (scanline by scanline or even
pixel perfect accuracy) Spectrum emulator on a really low end 386 (or less
if you're willing to go for icky real mode code). And as I've already
mentioned, this isn't just for plugging into a Spectrum emulator. Various
other machines used Zilog processors, sometimes two or three in a machine
(My favourite arcade of all time, Gyruss, used 2 Z80s and one of the old
Intel cpus that the Z80 was based on). Whilst there are problems with
interaction between cpus that run at breakneck speeds without any kind of
pause, if properly handled these emulations could gain a major speed boost
out of such a CPU.
> >> you definitely should focus on graphics speed and keep the z80 engine
> >> simple.
> >
> >Should I heck! :) Graphics on a Speccy is dead simple. I couldn't
squeeze
> >all that much more speed out of the current asm implementation really.
>
> assumed that 80% of the emulation time is screen refresh, this is
> definitely the best point for optimisation. ok, if you have already done
> it, the z80 is a good target for further optimisation.
Nah... on the older machines the CPU is king. The graphics chips were too
simple to be given any real boost by cleverly optimised assembler routines.
And on the new machines? Let the hardware on your 2d/3d card do the hard
work. The CPU is where optimisation it at! ;P
> finish:
>
> if you make screen refresh 10% faster it has the same effect as if you
> make your z80 engine 80% faster. that's basically what i wanted to point
> out.
Not if you want to overclock it doesn't. Or what if you want to emulate a
number of 4mhz Z80As simultaneously? And that's the main reason I wrote the
code. (Well, that and it's a decent challenge, of course).
> p.s.: as you may have noticed, i tried to write in British English ...
You've done a better job of it than me :) I nearly spelt favourite without
the 'u' a while ago... *shiver* ;)
Andrew
Andrew Davidson
AndyC <a.p.c...@uea.ac.uk> wrote in message
news:373F7007...@uea.ac.uk...
> kio wrote:
[snip some stuff about self-modifying code problems]
>
> What about instructions that modify the actual opcode itself eg:
>
> res 0,sm_instr
> jp c,useBreg
> loop ld hl,sm_instr
> set 0,(hl) ; change xor b to xor c
>
> ...
>
> sminstr xor b
Can't handle those at the moment. Potentially I could add support for such
simple modifications using self-modifying x86 code but I don't know if it's
worth it really... how often does this type of code get used? I was also
thinking of conditional jump/call/ret modifications but again, how often
does this happen.
Ideally I want to avoid having to recompile code every time something is
written to a previously compiled area of memory.
> Or instructions which modify the parameter:
>
> ld (branch_to+1),hl
>
> ...
>
> branch_to
> jp NNNN
>
> (I have to admit I do this quite a lot)
Yup. That's catered for. EVERYONE seems to do this a lot, the bastards! ;)
It's especially gitish when you think that there are perfectly good jp
hl/ix/iy opcodes just waiting to be used... *sigh*
> or LDDRs that overwrite the LDDR mid instruction? (rare, but sometimes
> useful)
Ack... don't do that... why on earth would someone do something that nasty?
> Also do you have to flush the Pentium's cache if you modify the x86
> code? I'd have thought that would cause problems in very tight
> self-modifying code.
>
> AndyC
Apparently not (according to the peeps on comp.lang.asm.x86 anyway). I
assume the Pentium keeps a track of memory writes/memory cache reads to sync
the memory correctly. Mind, what's the INVD (invalidate cache) opcode for
then? Mind, looking at that in the intel docs it's best left alone and would
probably screw up my code far far worse by not writing the memory
modification that caused the need for the cache flush. Erk. Think I'll leave
that well alone then... I'm feeling rather confused.
Andrew
Philip Kendall
> > It's called ZMhT (Zx eMulator high Technology) by Ergon developments of
> > Spain.
>
> Oops :)
>
> Have you got a homepage or anything? I'd be dead interested to take a look
> at that...
The shareware version is available on WoS (as is mentioned in the FAQ :-) )
Phil
--
/ Philip Kendall (pa...@cam.ac.uk pa...@kendalls.demon.co.uk) \
| New? Read the FAQ: http://www.kendalls.demon.co.uk/cssfaq/ |
| The Threat to Spectrum Emulation: |
\ http://www.kendalls.demon.co.uk/pak21/spectrum/threat.html /
Ian Collier
>Ack... don't do that... why on earth would someone do something that nasty?
Game protection systems do everything under the sun, so you really have a lot
to do if you want it to be able to run original copies of games. A favourite
trick is
LD HL,$4000
LD DE,$4001
LD BC,0
LDIR
which will stop after overwriting the ED-byte of the LDIR instruction,
because it no longer says LDIR. Also, I think the +3 disk version of
speedlock contains code which modifies itself depending on whether the
operation is a read or a write (for example, store the opcode for INIR if
it's a read and OTIR if it's a write). Not that you want to emulate the +3,
but it wouldn't surprise me if similar tricks were used elsewhere.
And self-relocating programs modify the target of all sorts of instructions
like JP, CALL and even LD.
BTW how on earth are you going to make the interrupts work? It might need to
interrupt in the middle of one of your compiled routines, and, worse, the
interrupt might change the values of any registers or flags.
A few years ago Arnt Gulbrandsen claimed on this group that he was planning
to write an emulator based on incremental compilation which would run Manic
Miner at 50x speed (or, more accurately, at the correct speed but using only
2% cpu). This emulator was to be called QAOP. I haven't heard much about
it - have you? :-)
--
---- Ian Collier : i...@comlab.ox.ac.uk : WWW page (including Spectrum section):
------ http://www.comlab.ox.ac.uk/oucl/users/ian.collier/imc.html
New to this group? Answers to frequently-asked questions can be had from
http://www.kendalls.demon.co.uk/cssfaq/index.html .
Sam Coupé FAQ: http://www.mono.org/~unc/Coupe/FAQ.txt
AndyC
Andrew Davidson wrote in message <7ho2c5$cdl$1...@news8.svr.pol.co.uk>...
>
>AndyC <a.p.c...@uea.ac.uk> wrote in message
>news:373F7007...@uea.ac.uk...
>> kio wrote:
>[snip some stuff about self-modifying code problems]
>>
>> What about instructions that modify the actual opcode itself eg:
>Can't handle those at the moment. Potentially I could add support for such
>simple modifications using self-modifying x86 code but I don't know if it's
>worth it really... how often does this type of code get used? I was also
>thinking of conditional jump/call/ret modifications but again, how often
>does this happen.
Depends. I've seen quite a few sprite scaling routines use these kind of
tricks. And protection systems do all manner of Wierd Shit(tm) :-)
>Ideally I want to avoid having to recompile code every time something is
>written to a previously compiled area of memory.
>> Or instructions which modify the parameter:
>Yup. That's catered for. EVERYONE seems to do this a lot, the bastards! ;)
>It's especially gitish when you think that there are perfectly good jp
>hl/ix/iy opcodes just waiting to be used... *sigh*
Yes but it's usually because there aren't enough registers to store things
in. Gawd knows how 6502 programmers get by...
>> or LDDRs that overwrite the LDDR mid instruction? (rare, but sometimes
>> useful)
>
>Ack... don't do that... why on earth would someone do something that nasty?
Evil code. 3:->
AndyC
Andrew Davidson
news:11079-hea...@comlab.ox.ac.uk...
> Andrew Davidson entertained comp.sys.sinclair with the following story:
nasty?
>
lot
> to do if you want it to be able to run original copies of games. A
favourite
> trick is
>
> LD HL,$4000
> LD DE,$4001
> LD BC,0
> LDIR
>
> which will stop after overwriting the ED-byte of the LDIR instruction,
> because it no longer says LDIR. Also, I think the +3 disk version of
> speedlock contains code which modifies itself depending on whether the
> operation is a read or a write (for example, store the opcode for INIR if
> it's a read and OTIR if it's a write). Not that you want to emulate the
+3,
> but it wouldn't surprise me if similar tricks were used elsewhere.
How does all that protect a game? Where are you running the example code
from? That speedlock thing is a bit sucky mind... I'll have to add in/out
modification then (I can cope with that with a few bytes of self-modifying
x86 code as the in/out op implementations are similar).
> And self-relocating programs modify the target of all sorts of
instructions
> like JP, CALL and even LD.
Hmm... how many times are they planning on relocating themselves? Once
should be copeable with (as long as they don't write over previous code, of
course). Or do you mean they modify immediate values for jp/call/ld (ie.
they are relocatable programs)? That's already taken care of.
> BTW how on earth are you going to make the interrupts work? It might need
to
> interrupt in the middle of one of your compiled routines, and, worse, the
> interrupt might change the values of any registers or flags.
Already done. It keeps a track of the tstate count within the compiled code,
jumping out when it reaches a given value. Then interrupts are handled
within the C code before jumping back into the machine code.
> A few years ago Arnt Gulbrandsen claimed on this group that he was
planning
> to write an emulator based on incremental compilation which would run
Manic
> Miner at 50x speed (or, more accurately, at the correct speed but using
only
> 2% cpu). This emulator was to be called QAOP. I haven't heard much about
> it - have you? :-)
Manic Miner is easy-peasy to run. Hardly does any self modifying stuff (just
the odd immediate value I think which is very very easy to trap). Mind, 50x
speed?!? Was that a real claim or are you just plucking a number out of the
air?. It might *just* be possible if you could maintain around 2% cpu usage,
maybe. On my P2 at any rate. Even on whatever passed for a stonking PC
(P200MMX? Probably a lot less than that really. I can't remember that far
back.) 'a few years ago' I can't see how that would be feasable. I mean,
really you'd need somewhat less than 2% because 2*50 is 100 and 100% means
no time for screen draw (Which uses maybe 10% with HEAVILY optimised
assembler, drawing to a Vesa2 screen, maybe...). So you're looking at a
steady 1.8% with ridiculously fast (probably MMX based) screen redraw. It's
an interesting idea mind... a 175mhz Z80a emulation... It might be possible
in the right circumstances (With a program which spends 95% of its time
running a halt op, for instance. Some of the demos available do this.).
Anyway, there's no real need to support some of the truely obscure programs.
I don't want to have to sacrifice half the emulators speed merely to ensure
that the odd wierd speedlock routine or whatever works properly. If you're
optimising for speed then you're optimising for speed, regardless.
Incidentally, I just added an overclock feature to SpectrEm-Dr so I'll stick
it on my website tonight. Look for version 0.7 if you're interested.
Oh yeah, totally unrelated and off-topic other than its sort of about
emulation and will definately require a dynamic recompiler... has anyone
seen the mpeg of the Dreamcast version of Ecco the Dolphin that's on Dave's
Classics? Do you know if that's just running pre-rendered video or if it's
doing it in real time? If it is it's absolutely incredible. Should be a
laugh trying to get that emulated on a modern PC... :)
Andrew
Philip Kendall
> Ian Collier <i...@comlab.ox.ac.uk> wrote in message
> news:11079-hea...@comlab.ox.ac.uk...
>
> > And self-relocating programs modify the target of all sorts of
> > instructions like JP, CALL and even LD.
>
> Or do you mean they modify immediate values for jp/call/ld (ie.
> they are relocatable programs)? That's already taken care of.
No, he means the self-modifying CALL instruction. Consider what
happens if PC=x and SP=x+3 and (PC)=CALL nnnn. Where do you think the
jump goes to? (The answer's in the Warajevo docs, BTW)
> > A few years ago Arnt Gulbrandsen claimed on this group that he was
> > planning to write an emulator based on incremental compilation
> > which would run Manic Miner at 50x speed (or, more accurately, at
> > the correct speed but using only 2% cpu). This emulator was to be
> > called QAOP. I haven't heard much about it - have you? :-)
>
> Manic Miner is easy-peasy to run. Hardly does any self modifying stuff (just
> the odd immediate value I think which is very very easy to trap). Mind, 50x
> speed?!? Was that a real claim or are you just plucking a number out of the
> air?.
Remember that this is Arnt "JPP" Gulbrandsen which can get to full
speed on a *386-25*. He's a pretty good coder...
:-)
john pickford
definately capable of those type of images.
Those shots are eerily similar to a game I worked on for N64 ( The original
Creator project, which became Mario Artist)
JP
Ian Collier
>"Andrew Davidson" <and...@lemure.freeserve.co.uk> writes:
>> Ian Collier <i...@comlab.ox.ac.uk> wrote in message
>> > instructions like JP, CALL and even LD.
>> Or do you mean they modify immediate values for jp/call/ld (ie.
>> they are relocatable programs)? That's already taken care of.
>No, he means the self-modifying CALL instruction. Consider what
>happens if PC=x and SP=x+3 and (PC)=CALL nnnn. Where do you think the
>jump goes to? (The answer's in the Warajevo docs, BTW)
Er, no I didn't mean that. I did in fact mean modifying the immediate values
in the JP, CALL and LD instructions. I have lots of programs that do this
(also HiSoft GENS and MONS do it when you start them up) but with most of
them it only happens once just before the program is started. I have one
program that is capable of moving itself in memory while it is running.
kio
>
> "Andrew Davidson" <and...@lemure.freeserve.co.uk> writes:
[snip]
> No, he means the self-modifying CALL instruction. Consider what
> happens if PC=x and SP=x+3 and (PC)=CALL nnnn. Where do you think the
> jump goes to? (The answer's in the Warajevo docs, BTW)
ahh .. that's good.
since the z80 pushes the exact address of the instruction after CALL i
assume it reads the entire address before pushing the PC. if not, i do
handle it wrong.
do i?
Erik Kunze
> since the z80 pushes the exact address of the instruction after CALL i
> assume it reads the entire address before pushing the PC. if not, i do
> handle it wrong.
You handle it right.
Maintainer of XZX, *THE* Spectrum 48/128/+3 emulator for UNIX/X11.
XZX home page: http://www.philosys.de/~kunze/xzx/
Arnt Gulbrandsen
> > > planning to write an emulator based on incremental compilation
> > > which would run Manic Miner at 50x speed (or, more accurately, at
> > > the correct speed but using only 2% cpu). This emulator was to be
> > > called QAOP. I haven't heard much about it - have you? :-)
Yes... I started writing that the winter around February or March
1993, and then in May my hands broke down. A terrible time - all that
summer I couldn't type for more than 2-3 twenty-minute sessions a day.
I stopped all my hobby programming, stopped even work for a few months
and lived off savings, and am now pretty much well again. There are
periods of weeks now, when my hands don't hurt at all.
But it meant no more hobby programming for years. Sorry. And I'm not
taking things like that up again - these days, my interest lies in
user interfaces. Some of you may have seen my latest hack, the
installation program for Caldera Openlinux 2.2?
> > Manic Miner is easy-peasy to run. Hardly does any self modifying stuff (just
> > the odd immediate value I think which is very very easy to trap). Mind, 50x
> > speed?!? Was that a real claim or are you just plucking a number out of the
> > air?.
Self-modifying code isn't particularly hard - it just means that some
JIT-compiled basic blocks need to be invalidated and recompiled more
often than for other code.
> Remember that this is Arnt "JPP" Gulbrandsen which can get to full
> speed on a *386-25*. He's a pretty good coder...
The numbers I had were based on real analysis. I can probably find
the old code I had... IIRC, I treated every jump target as the start
of a basic block, compiled that when I needed it (using a very quick
longest-match z80 to native translation table) into native code until
I found a return, or a jump to another basic block, or something.
It really wasn't that different from JPP... JPP used a simple dynamic
p-code approach, but it mapped one instruction at a time, not one
basic block at a time, so there was a lot of overhead.
And just for the record JPP got most games to full speed on my old
386sx-16. Sixteen megahertz, not twenty-five.
--Arnt
Arnt Gulbrandsen
I modified JPP to count the times when it needed to reinterpret an
instruction: Approximately none.
Lots of programs modified ARGUMENTS, but both JPP and my aborted
basic-block-based emulator looked at the Spectrom RAM location for
arguments. "LD A, 42" would be handles as... oh, shit, I've forgotten
Intel assembly syntax :) "mov ah, (bp+1)" where BP held the Z80 PC, I
think.
Very few programs modified CODE after the first time that code had
been interpreted. Thus, self-modifying code should not mean much of a
slowdown.
--Arnt
kio
Hi,
ehrm .. basically i think it slows down the z80 engine because it has to
check every instruction whether the code was modified just because you
can't trust anybody. even if no code is ever overwritten at all. you
just have to do it for the same reason for which you implement
undocumented opcodes and undocumented flag behaviour or the r register.
most time you don't need it, may be even never. but you check it because
it might be used any time. and these precautions make the z80 emulation
slower.
and, i personally have written at least two routines which created code
on the spot (thus overwriting previously executed code) though not on
the Specci but on the Atari ST (under an almost unknow OS called RTOS):
a line drawing routine and a text drawing routine which both supported a
number of different styles. i copied a few bytes into the inner loop
which depended on the style and then executed it. a nice hack, rarely
used because noone knew this damn OS... and probably every compiler and
assembler does this and it is also true after loading a program from
tape and (-: cut stop mercy :-)
... kio !
n.b.: caldera 2.2? is the time near when we'll get easy and reliable
Linux administration tools?
--
k...@odn.de (private)
Andrew Davidson
news:m33e0js...@lupinella.troll.no...
> One more thing.
>
> I modified JPP to count the times when it needed to reinterpret an
> instruction: Approximately none.
>
> Lots of programs modified ARGUMENTS, but both JPP and my aborted
> basic-block-based emulator looked at the Spectrom RAM location for
> arguments. "LD A, 42" would be handles as... oh, shit, I've forgotten
> Intel assembly syntax :) "mov ah, (bp+1)" where BP held the Z80 PC, I
> think.
>
> Very few programs modified CODE after the first time that code had
> been interpreted. Thus, self-modifying code should not mean much of a
> slowdown.
>
Erm... I don't mean to be rude but you really ought to think about having
another look at those programs... Spectrem-Dr fully supports immediate byte
modification yet fails to run the majority of the test snapshots I have in
full pelt mode. In interpreted mode these programs run fine (and as both
modes use exactly the same code there's no chance it's due to a bug). The
majority of modifications are relatively simple. For example, Contact Sam
Cruise changes the second byte of the ldir instruction used in the scroll
routine to an lddr when it tries to scroll left. Skool Daze modfies the
second byte of a SET 0,(HL), either modifying the bit set, or switching the
SET to a RES (I didn't bother checking to see exactly what it was doing
here). It would be possible to provide a quick hack to support these changes
(lddr/ldir changes could be implemented by changing two x86 bytes to flip
between increments and decrements, for example) but JetPac is a little more
vicious, peridoically replacing a JP 699Dh to LD A,0224h. And these are not
isolated cases. Bak2Skool and ChaseHQ for instance, rely on self-modifying
code. I originally began writing a dynamically recompiled emulator thinking
the same as you, that supporting immediate data modification would provide a
high level of compatability with the existing Spectrum software available
but it simply isn't true. People coding for a Z80 aren't likely to be coding
in C or another higher level language. Thus you don't get the sort of simple
self-modifying code a C compiler would produce (if it produces any at all).
These people were writing programs on a machine with a three and a half
megahertz CPU and needed all the speed they could possibly squeeze out of
the thing and, as I'm sure you're aware, self-modifying code is an excellent
way to achieve this. Such simple self-modification support just isn't enough
for all the clever tricks that were used in the good old days... believe me,
I've spent the last week trying to implement a fast self-modifyer to support
these damn games and, after two abortive attempts, it's driving me up the
wall... ahh well... I've had a day off to recuperate and I'm going to try a
third (and hideously more complicated) approach tomorrow... :(
Andrew
Arnt Gulbrandsen
2. An instruction that has been executed at least once, is
changed in such a way that its interpretation must change.
3. The instruction is executed.
I counted the number of times step 2 occured.
There are several interesting questions, such as "how many programs
use this sort of thing", and "how many places in each program are
there such instructions". I counted neither. I simply counted the
number of times an instruction was modified and then executed.
If an instruction was modified before being executed for the first
time, my counter didn't tick. If it was modified, then executed
twice, my counter ticked once.
And my counter showed "a few dozen per second or less". That's easy
to handle. I mean, we're talking about invalidating and recompiling
short blocks for code a few times per second. Nothing.
If I understand you correctly, you tried to self-modify compiled
native code when the Z80 code is modified? That approach sounds
hard... my approach was very different.
Each time I came to a jump or call, I'd find a compiled block of code
for the target, or else make one. Then I'd call my compiled code.
Each time anything was written to Z80 memory, I'd check a table and
see if changing that byte would change the meaning of anything I'd
compiled, and if so I'd throw away my compiled block.
(This is very similar to JPP, except dealing with basic blocks instead
of single instructions.)
The result is that when LDIR is changed to LDDR, the compiled version
of the entire routine is thrown away, and then the routine is
recompiled the next time it's used.
--Arnt
Matthew Westcott
Arnt Gulbrandsen wrote:
> Lots of programs modified ARGUMENTS, but both JPP and my aborted
> basic-block-based emulator looked at the Spectrom RAM location for
> arguments. "LD A, 42" would be handles as... oh, shit, I've forgotten
> Intel assembly syntax :) "mov ah, (bp+1)" where BP held the Z80 PC, I
> think.
>
> Very few programs modified CODE after the first time that code had
> been interpreted. Thus, self-modifying code should not mean much of a
> slowdown.
I'd tend to agree; 99% of the self-modifying routines I've written work
in the way you mention. However, there is one exception (and I probably
ought to have mentioned this earlier): suppose you had an interrupt
routine containing a CALL nnnn to (say) a music player routine. The
easiest way to turn off the music within the program would be to change
the instruction to LD BC,nnnn.
I'm not exactly the most advanced coder in the world, and I'd be
surprised if I was the only person to do this - although to be fair, all
a dynamically recompiled emulator would do in that situation is keep the
music on all the time.
Matthew Westcott
Andrew Davidson
> I simply counted. I didn't look at all.
>
> 2. An instruction that has been executed at least once, is
> changed in such a way that its interpretation must change.
> 3. The instruction is executed.
>
> I counted the number of times step 2 occured.
>
> There are several interesting questions, such as "how many programs
> use this sort of thing", and "how many places in each program are
> there such instructions". I counted neither. I simply counted the
> number of times an instruction was modified and then executed.
If the few snapshots I've got together are anything to go by about 10-15% of
programs perform step 2. However, if you tried adding tzx/slow tap style
loading into the emulator I'd imagine that number would increase as most
custom loading/protection routines almost certainly use heaps of
self-modified code.
> If an instruction was modified before being executed for the first
> time, my counter didn't tick. If it was modified, then executed
> twice, my counter ticked once.
>
> And my counter showed "a few dozen per second or less". That's easy
> to handle. I mean, we're talking about invalidating and recompiling
> short blocks for code a few times per second. Nothing.
Depends to the program. Contact Sam Cruise does one modification every
scroll, which occurs once every few seconds in the worst case senario.
Bak2Skool changes code all over the place and really fairly often. The main
problem is that we are not always talking about invalidating and recompiling
short blocks. Recompilation time for ity bity little blocks can account for
a fair bit of cpu time every second and, if the blocks end up being on the
large size (a few hundred ops or so), recompilation time would seriously
slow down the emulation.
> If I understand you correctly, you tried to self-modify compiled
> native code when the Z80 code is modified? That approach sounds
> hard... my approach was very different.
That's not what I was doing, it's what I'm trying to do now as it seems to
promise much more speed in the majority of cases. One instruction recompile
every frame, for example, is obviously going to be a lot faster than doing a
block of 100 50 times a second... However, it has its down side too. If the
new instruction takes more memory than the old instruction I'm going to have
to start copying memory all over the place which could really slow things
down.
> Each time I came to a jump or call, I'd find a compiled block of code
> for the target, or else make one. Then I'd call my compiled code.
Yup. Any unconditional modification of the Z80 program counter is used to
mark the end of a given block.
> Each time anything was written to Z80 memory, I'd check a table and
> see if changing that byte would change the meaning of anything I'd
> compiled, and if so I'd throw away my compiled block.
So far I only do this correctly for immediate bytes, which I compile as x86
immediate bytes, not x86 memory accesses like JPP (I'm not sure but I think
this will be faster than reading them out of memory in most cases, I'm going
to try and get everything else working and then test the two approaches to
see which wins).
> (This is very similar to JPP, except dealing with basic blocks instead
> of single instructions.)
>
> The result is that when LDIR is changed to LDDR, the compiled version
> of the entire routine is thrown away, and then the routine is
> recompiled the next time it's used.
Shouldn't be much of a problem with an ldir/lddr type block as they are
likely to be fairly tight, short, loops. However, I still think there's
plenty of processor usage to be saved by minimising the amount of recompiled
code.
Andrew