serial port checkpoint message delay xemu

54 views
Skip to first unread message

Ben-401

unread,
Feb 5, 2017, 10:25:26 PM2/5/17
to C65GS Development
it has been mentioned here:
https://groups.google.com/d/msg/c65gs-development/UuszXy0soL0/O_J7yHNxAQAJ
that an improvement to the design would be to:
- remove the hardcoded delay of approx 2000 clock cycles while the serial port outputs a char

This improvement is now under consideration.

The proposed solution is to implement a busy/ready flag so that the CPU knows that the char has been sent.
This proposed solution will also allow the xmega65 to run much faster by not having to emulate the 2k-cycles delay.

I want this thread to discuss the proposed solution.

Paul Gardner-Stephen

unread,
Feb 5, 2017, 10:27:30 PM2/5/17
to Ben-401, C65GS Development
Hello,

This is something we proposed previously, and has my support.  As you indicate, it allows us to remove the busy-wait that slows down the real machine slightly, and the emulator, substantially. 

Paul.

--
You received this message because you are subscribed to the Google Groups "C65GS Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to c65gs-development+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Ben-401

unread,
Feb 5, 2017, 11:33:38 PM2/5/17
to C65GS Development
My understanding of the current design is that to output to the serial port in assembly(ASM) you can use the "checkpoint:" function in "kickstart.a65".
This function does more than just output a single char, but wraps your "message/String" within other debug output like A,X,Y,P-register values, etc.

At the label "cp5" within "kickstart.a65", there is the following code:

CODE-START
    ; output checkpoint message to serial monitor
    ldx #0
cp5:lda msg_checkpoint,x
    sta hypervisor_write_char_to_serial_monitor
    inx

    ; delay at least 2,000 cycles to allow character to be written
    ldy #2
    ldz #0
cp6:    inz
    bne cp6
    dey
    bpl cp6

    cmp #10
    bne cp5

CODE-END

I dont know too much about the register "hypervisor_write_char_to_serial_monitor", maybe Paul can elaborate on this process.
I have found the following:
kickstart_machine.a65:    .alias hypervisor_write_char_to_serial_monitor $d67c

So maybe register $D67C has a trap associated with it? Ie if that register is written to, the uart loads the char-written-in-$D67C and sends that char?

I will need to dive into the vhdl to find out more of how this works.

LGB Gábor Lénárt

unread,
Feb 6, 2017, 5:53:34 AM2/6/17
to C65GS Development
Hi,

As far as the remember, the proposed idea was not using a busy loop for waiting a given time on serial port, but using an I/O register which tells if enough time already passed. Instead of the fixed "timing" loop, that flag should be checked and wait for being "ready". For the board, for sure, it would cause to only be changed to "ready" when enough time spent. On the emulator side however I can "fake" this "ready" bit as "always ready" since for the emulator there is no need to wait at all. In this way, the very same KS would work perfectly on both of xemu and the board normally + better performance (this latter one on the emulator). So the problem here that this flag should be implemented in VHDL and then to be used in the KS. Then I need only to have this flag wired to "always ready" in xemu.

A comment however: as it's probably noticed, the "fixed timing loop" (for waiting the desired time just burn tons of CPU cycles) should provide the same result on Xemu too. The only problem here, that Xemu not yet enough fast, and only runs on ~3.5MHz emulated CPU clock. That causes that the same amount of "CPU cycles burner" kind of timing is much slower on Xemu than with the board.

Ben-401

unread,
Feb 6, 2017, 7:23:09 AM2/6/17
to C65GS Development
Hi,
I have discussed this briefly with Paul, but we did not come to agreement on what is the best approach.

Some points of consideration are:
- implement a first-in-first-out (FIFO) buffer
- blocking or non-blocking call
- interrupt driven
- offloading a CHAR at a time, or a STRING at a time.

My approach would be to implement a FIFO to allow the CPU to quickly offload the string and continue processing.
The ASM-code (at about the "cp5:" label) would check to see if the FIFO was full, if not full then write the CHAR to a designated FIFO port. Continue until all CHARs of the string are offloaded.
The uart state-machine(SM) is required to perform some timing-delay to meet the baud rate. This is unavoidable.
The uart SM could continue to send chars, one-at-a-time, until the FIFO is empty.

The above design/implementation would not hold-up the CPU unless the FIFO was full, ie too many messages being sent in a short amount of time. This is an acceptable consequence at the hardware level.
The xemu would simply indicate that the FIFO is never full.

LGB: yes, we are considering the use of an I/O register to give the processor some information about the status of the serial port. This way the processor can choose to block and wait, or to go and do something else. Similar to a polling technique.

An alternative may be to use a hypervisor call, ie LDA or STA or BIT the ADDR=$D67C, which would cause the hypervisor to implement a function, a return code like CARRY=CLEAR would indicate that the message was offloaded to the output FIFO. CARRY=SET would indicate some error.

This gives rise to the concept of a transaction. How are other transactions performed within the VHDL?

LGB Gábor Lénárt

unread,
Feb 6, 2017, 8:34:29 AM2/6/17
to C65GS Development
Hi Ben,

What I can see here, that from my view point _only_ it's not important how it's done, ie a single "ready" bit can be used to implement per character "ok to send" or even some signal of FIFO (ie, FIFO is not full), the exact details does not count too much _i think_ for the KS. Of course there are differences how fast the stuff then (ie, what makes waiting after every char, or having a FIFO quickly filled by KS, then have time "slowly" to be emptied over the USB/serial link) from some viewpoint. I just want to say, that whatever the exact VHDL implementation will be, IMHO there can be a single bit stuff for that, and the exact implementation may even change, with keeping the "interface" ie the single ready bit, so KS wouldn't be needed to be changed for the new behaviour every time. So because of this, I think for the first solution there can be even a single char transmit capability at time with a ready bit, which should be implemented in VHDL some way, and KS should be modified to take account that bit. Later, the VHDL level implementation can decide to introduce a FIFO for example, but even then, the same KS works: even if it checks ready bit after each char to send, it will work with a FIFO-scheme as well, as it will found "ready" for many characters thus not causing more delay than the instruction takes for testing that bit. Another advantage of this, that Xemu just needs to keep ready bit always ready, and that's all (or not ... technically, even Xemu needs time when internal buffer is full and a connected m65dbg hasn't started to empty it with some reading, but it's more flexible stuff than a serial protocol). Ok, so in nutshell, from view point of KS, I can imagine a solution where you only need to check a single bit in an I/O register before sending a single byte info over the serial link, and waiting for being ready if it's not that already. That would work with FIFO or single byte transfer scheme as well then. What I also suggest this: it's often the cause that development is slow if we need the "perfect" and "final" solution in a single step, what I learnt a lot from my work too :D :D

Paul Gardner-Stephen

unread,
Feb 6, 2017, 2:19:26 PM2/6/17
to Ben-401, C65GS Development
Hello,

It can't be a KS trap, because it is used from within KS.

Whether we add a FIFO is secondary, as it doesn't change the interface. It will, however, consume quite a bit more FPGA space, for limited return (since the number of chars per second doesn't increase, and it already is at the limit when booting.

Making the IO port blocking is a real pain from the VHDL side. Also, it stops the program deciding if it has something more useful to do while waiting.

Thus, I suggest keep it simple for now: We make the status bit that indicates ready, and then can switch to using that to solve most of the problems.

Start simple, optimise when there is a compelling need.

Paul

--

Ben-401

unread,
Feb 9, 2017, 8:31:38 PM2/9/17
to C65GS Development, ben.0...@gmail.com, pa...@servalproject.org
I have implemented the status signal from UART to CPU.
I have tested on actual hardware and it works great. Much quicker even on fpga.

LGB: can you please implement the required changes in xemu?
i suggest returning either "01" or "00" whenever the "D67C" register is READ.
The returned value needs to be opposite of its previous returned value, ie toggled.

When xemu is updated, and the pair confirmed to work together, i suggest then merging branch "serial-port-delay" into "development".

Ben-401

unread,
Feb 9, 2017, 10:45:40 PM2/9/17
to C65GS Development, ben.0...@gmail.com, pa...@servalproject.org
stop-press, i have found a problem, sorry.
seems the cpu locks up on first boot, and to overcome need to deassert sw-12 at PROG, and when KS is running, then assert sw-12.
I will look into this and try find a better solution.

The emulator should not experience this same issue, so LGB please continue to update the emulator to return "00", then return "01", then "00", then "01, 00, 01, 00, 01, etc.

Ben-401

unread,
Feb 9, 2017, 11:45:10 PM2/9/17
to C65GS Development, ben.0...@gmail.com, pa...@servalproject.org
Paul has made a suggestion that results in no changes to the xemu.
The suggestion does not require the toggling of the bit.
Currently I am testing out Pauls suggestion.

LGB Gábor Lénárt

unread,
Feb 11, 2017, 7:54:23 PM2/11/17
to C65GS Development, ben.0...@gmail.com, pa...@servalproject.org
Hi,

OK, honestly I've just read the messages here. okey then, please give me information about the progress if need to implement anything in this area then, according to the latest information :) :) Thanks.

Ben-401

unread,
Feb 12, 2017, 9:57:12 PM2/12/17
to C65GS Development, ben.0...@gmail.com, pa...@servalproject.org
Hi LGB,
Sorry for the confusion on this one.
Just now I have uploaded a new version of kickstart(KS), it is on the "spdelay2" branch at:
https://github.com/MEGA65/mega65-core/blob/spdelay2/src/kickstart.a65
because we now have others using the xemu with this new version of KS.

Others:
go into the $GIT_ROOT dir, cd mega65-core, git checkout spdelay2, ./compile, CTRL-C
then copy the KICKUP.M65 file into the $GIT_ROOT/xemu directory so the emulator can read the new version of KS.

I have tested the KS in the "spdelay2" branch, and it works well in the xemu/xmega65.

This is because currently the KS just writes chars to the $D67C, where the emulator then grabs these chars to create its output string.

At some stage in the future, we will have to update the vhdl to provide actual feedback, and then update KS to check this feedback.
Currently I am stuck on implementing the 'Pauls suggestion' and am having trouble getting the ASM code to read correctly the feedback signal from the serial-monitor.
Once I have it working in hardware, I will then need to talk to LGB about any changes required (if any) to the xemu.

LGB Gábor Lénárt

unread,
Mar 6, 2017, 5:25:08 AM3/6/17
to C65GS Development, ben.0...@gmail.com, pa...@servalproject.org
Hi Ben,

I'm sorry, I am not the fastest recently to respond :( I've tested it, and looks fine, and indeed it's quite speedy now compared to the original KS :-) Now, I am thinking to replace the built-in KS of Xemu/M65 binary for people don't want to use external one at all, which in case xemu defaults to the built in. Just I don't feel a too good idea to use a branch of the mega65-core repo too much, since if the change isn't committed back to the main (or at least the devel?) branch, soon I will miss features introduced in other branches :( Well, to be honest, I am also a bit confused about branches, ie master or development is the best to be used to build the default KS for Xemu's built-in one, what do you think?

Ben-401

unread,
Mar 6, 2017, 10:46:34 PM3/6/17
to C65GS Development, ben.0...@gmail.com, pa...@servalproject.org
Hi LGB,

Firstly, I like to watch the "Network Graph" as it graphically shows the changes to branches and etc.
https://github.com/MEGA65/mega65-core/network

Secondly, the only change between "development" and "spdelay2" is shown by clicking the HEAD of "spdelay2" on "the Network Graph".
https://github.com/MEGA65/mega65-core/commit/dec4f30370a6dcc3d5bcf50de29e195647080ccd
As the above link shows, I have just removed the 'delay'-code and modified the comment to reflect this.

So, until the above change in 'spdelay2' is merged back into an upstream-branch, yes, there is fear that it will never be merged and the EMU will be using a stale/special KS from an unactive branch.

One option is that I could merge the 'spdelay2' branch into the 'development' or 'master', but I am not ready to do that. This is because I have not updated the VHDL to reflect the change in delay signalling, more particular I have not correctly wired the serial-port-status-signal back to the cpu, my work is currently in the "serialportdelay" branch. Paul has suggested an alternate mechanism, which I am yet to implement.

I have just quickly removed the delay in the 'spdelay2' branch because others are now using the XEMU for further development purposes. They do not need to compile the VHDL and use the Nexys board.

We need to keep the VHDL and ASM (ie KICKSTART) in sync, especially for the 'master' and 'development' branches. Currently all branches are in sync except for the 'serialportdelay' and 'spdelay2' branches. These later two are a work in progress.

Short answer, I would embed into XEMU the KS from the 'spdelay2' branch, for now.

What do you think about your XEMU compile process to check to see if there is a "./../mega65-core" directory, and if so, then your makefile calls "./../mega65-core/src/make firmware" to generate the KS of whatever branch the user has in their $GIT_ROOT$ directory?

Yes, the EMU is now quite speedy compared to when the EMU had to emulate the delay.
Ben.

LGB Gábor Lénárt

unread,
Mar 7, 2017, 5:30:20 AM3/7/17
to C65GS Development, ben.0...@gmail.com, pa...@servalproject.org
Hi Ben,


Firstly, I like to watch the "Network Graph" as it graphically shows the changes to branches and etc.

Surely, I know that since ages, but it does no answer about details the purpose and "usage" of branches, their goals, whatever :)

 
One option is that I could merge the 'spdelay2' branch into the 'development' or 'master', but I am not ready to do that. This is because I have not updated the VHDL to reflect the change in delay signalling, more particular I have not correctly wired the serial-port-status-signal back to the cpu, my work is currently in the "serialportdelay" branch. Paul has suggested an alternate mechanism, which I am yet to implement.

I have just quickly removed the delay in the 'spdelay2' branch because others are now using the XEMU for further development purposes. They do not need to compile the VHDL and use the Nexys board.

We need to keep the VHDL and ASM (ie KICKSTART) in sync, especially for the 'master' and 'development' branches. Currently all branches are in sync except for the 'serialportdelay' and 'spdelay2' branches. These later two are a work in progress.


Short answer, I would embed into XEMU the KS from the 'spdelay2' branch, for now.

I see. I've just updated the "built-in" KS in Xemu, btw.


What do you think about your XEMU compile process to check to see if there is a "./../mega65-core" directory, and if so, then your makefile calls "./../mega65-core/src/make firmware" to generate the KS of whatever branch the user has in their $GIT_ROOT$ directory?

That doesn't make too much sense for multiple reasons. First, that Xemu even know does not change the embedded KS ever, you need specially do it "manually" not the part of the normal build process, etc. That is, because I thought not everybody has a built KS to build the "C hex file" from it then and to be included, etc etc. So it's a pure manual process, and even the repository contains only the result only. The other problem with that approach that it's kinda complex even know it takes ages for Travis build infrastructure to install everything and compile for checking/building process, if you depends on other git repo, whatever, it's even more complicated. I think, this embedded KS stuff is something I don't want to do always anyway and it's even better that I need to do it manually because some can decide when to do. In longer term, I think, this embedded KS thing must go away, it's something too "fixed". Once I do the automatic download for needed files if user accepts, it can download KS too. And then there is no need to have strict separation between embedded built-in KS in Xemu, and the one it can load (I mean before the SD card init, not the KS upgrade ...).It's even more simple and logical, having only two KS sources then instead of three, as now. The only reason of built-in KS now, that user may don't have KS for Xemu and then Xemu cannot be used until the situation is fixed but it's more or less purely an organization issue. This Travis thing basically builds (automatically) Xemu every time on every github commit etc, also if the branch is "deploy" the result is uploaded to bintray, where you can download Xemu binary builds (for win32, win64, ubuntu linux 64 bit deb package, a converted rpm package from deb, and OSX) from. So it's both for build check and actual binary distributions. The other thing: not every user building Xemu from source is expected maybe to have Ophis, etc etc installed to also build mega65-core firmware, and so on, and I feel it's a bit out of scope of Xemu at this point already. For Xemu, KS should be treated as a "ROM" or "data" in the sense that it needs to be executed by the emulated machine, but not too much inner details it's need to know, only binary data needed.

 
Yes, the EMU is now quite speedy compared to when the EMU had to emulate the delay.

Yes, however surely Xemu is still slow, since it runs only at ~3.5MHz emulated clock speed (or ~1, in case of "slow" mode, for sure), inherited from my C65 emulator. I am just thinking that it's time soon to change on this, however I am a bit confused on speed control of M65, does anybody here have a good summary on this topic? Still, the 48MHz clock maybe too much now for Xemu/M65 on an average PC (should be optimized, etc) but at least I can see then where is the bottleneck and how, currently it cannot be seen too much, since an avg PC is enough usually for ~3.5MHz, for sure. I really think that I should work on the M65-compatible speed control (just I am a bit confused here with things like POKEing address zero, and things like that) at least if no other it helps see how to continue optimizing Xemu for hopefully M65 speed :)

Paul Gardner-Stephen

unread,
Mar 7, 2017, 5:48:26 PM3/7/17
to Ben-401, C65GS Development
Hello,

Just a quick note on the branches at the moment:

In addition to the branches where Ben is working, the other important branch is the cursorkeys branch, that despite its name (and this is something I need to sort out) is where Falk and I are committing our various important fixes to the M65, including those necessary for GEOS support, bitplanes etc.

Thus we need to commit the hypervisor serial port changes there as well.

At some point, we will hopefully be able to remerge these two foci of activity, but until then, it is the cursorkeys branch that is the one that is actually the one that will be used for bitstream releases.

Paul.

--

LGB Gábor Lénárt

unread,
Mar 8, 2017, 3:05:40 PM3/8/17
to C65GS Development, ben.0...@gmail.com, pa...@servalproject.org
Hi,

Thanks, Paul, exactly something similar information I was interested in, since just gazing the commit log of different branches, I felt totally confused, especially compared with the branch names :-P Surely, I am more focused currently on the KS source, not the VHDL, but it's also a great information thanks, I will try build a bitstream from cursorkeys branch then - now that I have basic ISE knowledge with some working random Z80 based computer implemented myself, btw :)
To unsubscribe from this group and stop receiving emails from it, send an email to c65gs-developm...@googlegroups.com.

Ben-401

unread,
Mar 15, 2017, 7:12:32 PM3/15/17
to C65GS Development, ben.0...@gmail.com, pa...@servalproject.org
Hi LGB,

Firstly, sorry if I seemed arrogant by using firstly/secondly etc.
Secondly, see above.
:)

As for the branch goals/usage details, I suggest that this forum be used to advertise to others what each branch-of-significance is used for.
And then, if it is not spoken about, then I think we can all assume that it is a private-branch under development and not expected to be recognized by the wider group of developers.

Thanks for your explanation above, here is mine:
As for the xemu compiling the KS, I was thinking in rom/Makefile, modify the "kickup.cdata: KICKUP.M65" to call a script that:
if "../../../mega65-core/src/Makefile" exists
  then compile the ../../../mega65-core/src/KICKUP.M65 and copy into xemu/rom as "kickup.cdata"
else
  ../build/bin2values KICKUP..M65(your supplied version) "kickup.cdata"
end

this way, the travis-build should not be affected as it will do the else-branch in the "if" above.

I respect this is your project, i just want to offer different ideas.
Those working on the xemu at the moment know how to recompile the xemu with a user supplied KICKUP.M65 to be embedded within xemu (similar to the proposed script above), so we can just "manually" recompile as you suggested.

Ben-401

unread,
Mar 15, 2017, 7:22:57 PM3/15/17
to C65GS Development, ben.0...@gmail.com, pa...@servalproject.org
Hi Paul,


On Wednesday, March 8, 2017 at 9:18:26 AM UTC+10:30, Paul Gardner-Stephen wrote:

Thus we need to commit the hypervisor serial port changes there as well.


Currently i think the serial-port change to the KS needs to go into its own branch, or as a patch. NOTE that it is only used by those working on the xemu.
Firstly, I need to rework the VHDL to be in sync with the KS-ASM, (for now only xemu users need the serial-port fix)
Secondly, merge that branch into 'dev' so that both the hardware-users and the xemu-users get the change together.
 
At some point, we will hopefully be able to remerge these two foci of activity, but until then, it is the cursorkeys branch that is the one that is actually the one that will be used for bitstream releases.


Yes, we are working on merging the two foci now, as per
https://groups.google.com/d/msg/c65gs-development/YsDcjTlaDhQ/JbbX2k1nBQAJ

Ben-401

unread,
Mar 21, 2017, 2:10:08 AM3/21/17
to C65GS Development, ben.0...@gmail.com
Hi LGB,

The serial-port-delay changes have now been incorporated, and seem to work well on the actual hardware.
I think you have already found what needs to be done in the XEMU from your discussion in another thread (MASTER branch...).
Can you please confirm 'in this thread' that you have updated the XEMU to support the serial-port-changes affective from KSv10 onwards?

For those reading out of interest,
Within the KS we now have to check a flag before sending chars to the serial-port-UART.
Refer to commit:https://github.com/MEGA65/mega65-core/commit/f93ca46938096cad0387ae78d5274b881e9a51d3
for details on how the KS ASM neds to be written.,

Ben.

LGB Gábor Lénárt

unread,
Mar 21, 2017, 7:38:45 AM3/21/17
to C65GS Development, ben.0...@gmail.com
Hi,

If you mean about the method to read $D67C to see, if UART is ready, then yes. Here is my Xemu commit about this to be supported:

https://github.com/lgblgblgb/xemu/commit/bcfa5336485bfd861b63d9d252f35e0197a1a4ea

Though I am a bi confused, since I don't see v10 KS only v11. Maybe because I've checked only development branch out :) Anyway it must be the same I guess, right?
Reply all
Reply to author
Forward
0 new messages