Firmware 1.0.0 Progress, this is the biggie!

[Julian Skidmore]

Hi folks,

Hot on the heels of Firmware 0.9.9 is the development of 1.0.0. Firmware 0.9.9 was mostly about improving FIGnition's programming environment: editing large amounts of text and being able to trap execution (Break keys and stack checking). As a side-effect of trapping execution, interrupt support was added and to provide the code space for all this the Amic Flash chip driver was converted from 'C' to Forth.

Firmware 1.0.0 is the biggie, a real milestone in development! The big improvements are:

1. A working audio bootloader. The new bootloader will be versatile, capable of (a) Upgrading the FIGnition firmware and (b) Transferring source code to and from the external Amic Flash as whole programs at a time. The audio In routines can also be hooked by user code (or at least AVR code) to extend audio input for other reasons.

2. Floating point arithmetic :-) ! I'll use the standard AVR floating point library as a basis to provide f+ , f-, fneg , f* , f/ . I'll also add fpoly and fvec routines which will make it easier to implement transcendental functions via Chebyshev polynominals (and other polynominal functions) as well as matrix operations.

There are two smaller projects in 1.0.0 which are:

3. A Better AVR interrupt subroutine/procedure callling mechanism as part of a space-grabbing exercise. This is actually quite valuable to the AVR community as a whole. The issue was that AVR interrupts in 'C' would push every AVR register if the interrupt routine called another function and it cripples interrupt performance. This means that AVR interrupt writers avoid doing that if possible as you can see from how FIGnition's video interrupt works. However, when scanning the keyboard I really do need to call a separate function so I had a kludgy bit of code which pushes all the other registers; then grabs an execution address; calls it; then pops all the registers again. It's slow and bad for stack usage, so I've implemented the far more elegant mechanism I've wanted to for ages. This will be covered in a oneweekwonder blog.

4. Faster searching. FIGnition's Forth rom uses a variety of related search algorithms in the compiler, the editor and the Flash Driver. I think it would benefit from a search byte code primitive; particularly for the Flash Driver which is many times slower than it should be because of a poor search algorithm. This is today's job (perhaps it'll take longer).

As before I'll provide regular updates of the progress for all of these things!

-cheers from Julz

--
                             
                  The DIY 8-bit computer from nichemachines™

[Julian Skidmore]

Hi folks,

I've completed the essential part of step 4. It turned out that the Flash Driver searching algorithm and other searching methods didn't really have much in common so I implemented the most important one, which was a re-write of the Flash Disk block search algorithm so that it had a simple inner core and then converted that part to assembler. That now works and the Flash Disk driver is now at least as fast is it was originally when it was written entirely in 'C' (because I used a slower search algorithm in 'C').

So, the next step is to start working on the Bootloader itself! Also I missed a few routines that are needed to support floating point, namely fint and float as well as stack operations to better manage floats: dvar, dconst, d>r, dr>, dswap, dover, 2dover, drot, 2drop, d@ and d! ( i'm using d in a number of situations to avoid ambiguity with the number 2, e.g. it's easy to write 2var instead of 2 var so I'll be using dvar); I'll probably also benefit from d, and darr to create double-length arrays and comma in double length values.

Added support for double-length values complicates stack operations quite a bit, because of course they are liable to contain a mixture of double and single length values which implies a whole set of operations for those combinations, clearly inappropriate.

-cheers from Julz

[Julian Skidmore]

Hi folks,

I didn't manage to send the Friday update last week, so here's a combined update for last week and this week's work on the new bootloader.

Last week I did quite a bit of conceptual work for the audio loader. I considered implementing the routines as actual device drivers, but later rejected that because the user syntax would look more complex than I wanted for relative newcomers (who would be using these commands very early on). That is, commands of the form:

tape> >ram cp

But new users, and most of us, I think would prefer something equivalent in simplicity to load"" . In the end I settled on the following commands and what I'd like to know from you guys is whether you think they make sense.

ear ( loads an external ram image from audio and needs no parameters)
mic” name” ( saves a ram image as audio, and gives it the name "prog" [ audio data contains a header at the start which can contain a name])
blk len save” prog” ( saves from external flash to audio)
blk load” prog” ( loads to external flash)

The underlying code would still be accessible, the core routines for audio transfer would have Forth vector addresses so it would be possible, e.g. to write code to copy eeprom to audio (or vice versa) or data from another device such as I2C. This will be properly documented.

Last week I also did quite a bit of work on the actual coding for the audio bootloader. The core bootloader must be in 'C'/Assembler because you need to be able to upgrade even if Forth has been bricked. However, it turns out that Forth will be fast enough to handle the callbacks for audio transfer in the other cases given a byte transfer routine in 'C'. This makes it much shorter than it might be.

However my estimates show that audio support will go over 1Kb. I did some work to simplify it, but it'll still be around 1200 bytes long. And because that puts constraints on the floating point code, I needed to do some work to estimate how long that would be, which meant basically writing that code too!

Floating-point libraries are amorphous beasts. For a start, in most embedded applications you don't want to use floating-point code as its slow so there's no support for it in low-end microcontrollers like the AVR (and even many higher-end ones such as Cortex M0). However, GCC for example will insert its own general-purpose IEEE 754 code if you just try to use floats in an AVR program. It's really big and there are quite a lot of forum posts from users shocked at what it's done to their programs.

The normal solution is to use the libm avr floating point library. It's nice, fast, IEEE 754 compliant and small, weighing in at 672b for the basic f+, f-, f* and f/ routines. However, when I added my Forth support routines it took it above 1142b which meant I was now around 350 bytes over budget :-(

So I went back to the drawing board. I remembered a long time ago I'd worked on my own single-precision 32-bit floating-point library, which I'd written for compactness. So, I converted it from Thumb code to AVR and found I could squeeze it down to 354 bytes :-) And then because of the way I'd written it, some of my other Forth code could use routines in it to save some more space and now I have an (untested) full floating-point library with the addition of 32-bit stack operations; comparisons; float<->int conversion and float<-> string conversion all in less than 800b :-)

So, with the saving of about 340b it's now looking OK again!

I hope to publish my FPcode as an alternative FPLibrary. It's not IEEE754 compliant as the FP format is slightly different ( exp8:sign1:mantissa23) and it doesn't handle NaNs, infinities or underflow properly (it just has an 'overflow' value and treats any operation on overflows as returning overflow). On the other hand, it's not much slower than the normal one; computes mantissas to 32-bits internally; supports operations on denormalised values and most importantly, only 53% of the length! That's got to be worth something to someone ;-)

It also means I need to work on and debug the float code first as it defines how much space I have available for audio. Meanwhile here's some info about it:

Firstly, number ranges:

The reason I support denormalized values is because 0 is then handled automatically. Note: both 0.0 and -0.0 are currently possible, e.g. -3.5e-46 / 10 => -0.0.

The floating point format is 8-bits of exponent (with a bias of 127, like IEE754). Then a 1-bit sign bit. Finally there's a 23-bit mantissa+ a hidden msb in bit 23. Normalised mantissas are in the range 0.5 to 0.9999998808.

Second, accuracy and performance:

Floating point arithmetic is accurate to 6.92 decimal places. This is better than a Jupiter Ace, which only has (exactly!) 6 dps of accuracy. The FIGgyFP library can calculate internally to 9.3 dps of accuracy (though this is rounded out when results are written back to the stack).

FP performance on FIGnition should be significantly better than 20x Jupiter Ace performance, because (a) the AVR simply has much better computational resources than a Z80, more registers, more efficient instructions, 2-cycle 8-bit multiplies. (b) The Jupiter-Ace BCD representation for floats is computationally awkward.

Thirdly Forth commands:

The following commands are provided:

fp numbers can be entered directly in the format: 12.3456789e-20 . You must use a small letter e to denote exponents, as an upper-case 'E' can be interpreted as a hexadecimal value. In theory: $12.E5e10 is a valid number ( it means $12E5 x 10^(16-2)). The underlying 'number' command in the main ROM will be extended to recognize floating point strings so you can use it that way too.

You can display floating point numbers using f. ; there's no formatting options, i.e. no f.r for example. However, the underlying conversion routine #f is also provided so you can construct formatted floating point output.

The basic routines: f+, fneg, f-, f*, f/ are provided of course.

Conversion routines: fint ( fp -- d)  and float ( d -- fp) convert between doubles and floats.

Comparisons: f0< and f< are implemented. -0.0 0.0 f< is true.

Finally, stack operations: 2drop d>r dr> 2over and 2swap have been added. The last two are in assembler, so they're as fast as can be. 2rot isn't there ( it'd be d>r 2swap dr> 2swap).

With this, even though I won't be able to handle vectors, polynominals nor transcendental functions; floating point support on FIGnition will be finally available and pretty comprehensive :-) !

Testing all this comes next week!

-cheers from Julz

[Julian Skidmore]

Hi everyone!

Just a minor update. As I was manually checking the floating point code I kept finding subtleties I needed to sort out. Now that's done I've managed to set up a simulation test environment and linked it all into one binary & I'm ready to start testing.

So what kind of subtleties arise? Well here's a delicious sample:

1. FP negation just involves inverting the sign bit, but inverting 0xffffffff (the special "overflow" value) would result in a valid number - I had to add code to check overflow first!

2. Dividing by denormalized numbers could lead to mantissas containing nonsense.

3. Exponents can go out of range in the middle of a calculation so I ended up needing 16-bit exponents temporarily.

4. mantissas during need to be shifted right twice , because a carry from its most significant bit mustn't be allowed to affect the sign.

Etc etc Stuff like that :-)

-cheers julz

[Julian Skidmore]

4. Should have said "mantissas during Addition...."

-cheers julz

[Julian Skidmore]

Hi folks,

Floating-point arithmetic is notoriously tricky - ask anyone who used an original Pentium processor in 1994!

After quite a lot of debugging, I now have FNeg and f+ working to my satisfaction. I've had to scale back my ambitions for FIGnition floats, they no longer handle denormalized values. On the other hand, the format is largely like IEEE (except the bias is 127 instead of 128; though I might change that).

Last night I managed to start testing f* and with a bit of work this morning it also works to my satisfaction - though it might need a bit more testing. The only remaining core routine is f/ .

Following that I will still need to test the remaining definitions:

2over, 2swap, d0<, f<, fint, float, (fnum) [ which parses floating-point strings into floats] fscale, #fDigit, #f and f. [ which convert floating-point representations into strings]. That's still quite a bit of work.

-cheers from Julz

On Tue, Apr 1, 2014 at 7:35 PM, Julian Skidmore <theorigi...@gmail.com> wrote:

4. Should have said "mantissas during Addition...."

-cheers julz

[Julian Skidmore]

Hi folks,

Today's update: f/ now works, that means the core routines for floating point arithmetic work and I'm on to testing the higher-level stuff ( f- is pretty trivial, being : f- fneg f+ ; )

-cheers from Julz

[Julian Skidmore]

Hi folks,

Well, amazingly I managed to define f- incorrectly! Can you spot the error in this?

: f- fneg f- ;

:-) ?

It just shows that no matter how trivial you think the function is, it's still possible to get it wrong :-)

I've now tested the remaining assembler routines: 2swap, 2over, fint and float. This means I'm on to the higher-level routines written in Forth itself. (fnum), (fscale), #fDigit, #f, f. , . As I keep saying, I don't need to test f- , it's trivial ;-)

For the non-trivial routines I have written (what I believe) to be relatively comprehensive test sets, with automatic checking. For example, I have 8 float tests and 15 fint tests as follows:

    0x00000001, 0x40000000, // 1d => 1.0.
    0x00000017, 0x42380000, // $17d => 17.0
    0x00010000, 0x48000000, // 65536d => 65536.0
    0x12345670, 0x4e11a2b4, // (roundup) 0001:0010:0011:0100:0101:0110:0111:1000 =>
                            // Mantissa: 1001:0001: 1010:0010: 1011:0011: 1100:0
    0x7fffffff, 0x4f800000, // Max +ve int.
    0xffffffff, 0xc0000000, // -1d => -1.0
    0xEDCBA990, 0xce11a2b4, // (-roundup) 0001:0010:0011:0100:0101:0110:0111:1000 =>
                            // Mantissa: 1001:0001: 1010:0010: 1011:0011: 1100:0
    0x00000000, 0x00000000, // zero.
   
    0x40000000, 0x00000001, // 1.0 => 1d
    0x42380000, 0x00000017, // 17.0 => 17d
    0x4a11a2b0, 0x00123456, // 6-digit correct expansion.
    0x4bffffff, 0x00ffffff, // maximum +ve correct value.
    0x4f7fffff, 0x7fffff80, // maximum +ve value.
    0x4e11a2b4, 0x12345680, // Previous float conversion reconverted.
                            // Mantissa: 1001:0001: 1010:0010: 1011:0100: 0000:0
                            // (rounds to) 0001:0010: 0011:0100 0101:0110: 1000:0000
    0x4fffffff, 0xffffff00, // maximum unsigned value.
    0xc0000000, 0xffffffff, // -1.0 => -1d.
    0xca11a2b0, 0xffedcbaa, // 6-digit correct expansion.
    0xcbffffff, 0xff000001, // maximum -ve correct value.
    0xcf7fffff, 0x80000080, // maximum -ve value.
    0x423fffff, 0x00000017, // 17.99999 => 17d.
    0xc23fffff, 0xffffffe9, // -17.99999 => -17d.
    0x00000000, 0x00000000, // +0.0 => 0
    0x80000000, 0x00000000, // -0.0 => 0

It might look a bit cryptic, but essentially all these tests are written in 'C' - the Floating point library and its test harness is currently a 'C' program run in simulavr on a simulated AtMega8 (because my simulavr won't handle a proper AtMega168). The first number in each pair is a double that's placed on the simulated Forth stack; then I execute an FP operation (e.g. _Float or _FInt) and check to see that the number on the top of the stack is the second number in the pair. If it isn't, the test failed. In the float tests, the numbers on the left are integers and the ones on the right are the hexadecimal pattens for floating point numbers. In the Int tests the ones on the left are hex patterns for floats and the numbers on the right are long integers.

-cheers from Julz

[Julian Skidmore]

Hi folks,

As of this evening, the floating point code all works :-) my recent efforts to read and display floating point numbers have been completed which means I can type things like

1.09e-10 99.6e4 f* f.

And it will all be processed correctly :-)

Audio data transfer next :-)

[Julian Skidmore]

Hi folks,

I now have some benchmark timings for floating point calculations. This simple benchmark:

: bm1f 10000 0 do 10.9 9.8 f+ 7.6 f- 5.4 f* 3.2 f/ 2drop loop ;

Takes 3.3s with FIGnition's FP code, which amounts to about 10K floating-point operations per second.

On a Jupiter Ace (with 1000 loops instead of 10000 , it takes 14.18s). This means that for this code, FIGnition is about 42.9 times faster!

FIGnition's FP code is 45.3 x faster than a ZX Spectrum in BASIC.

All rather excellent I'd say :-)

-cheers from Julz

[Julian Skidmore]

Hi folks,

I've also been comparing floating point calculations with other 8-bit computers (and a Mac Plus). On the 8-bit machines I do 1000 loops and the program is written in BASIC. Even for BASIC, floating point calculations will dominate the timings:

FIGnition: 3.3s, or 2.7s for the calculations themselves.

Jupiter-Ace: 14.18, -0.6s =  for calculations themselves, 13.58. FIGnition is 50.2x faster.

ZX Spectrum: 14.94s, really 10.14s without the loop overhead. FIGnition is 37.6x faster.

Acorn Electron: 8.06s, about 6.56s without the loop overhead (assumed to be 1.5s). FIGnition is 24.3x faster.

VIC-20: 10.91s, about 7.91s without the loop overhead (loops are 3s). FIGnition is 29.3x faster.

Oric-1: 15.75s, or 9.75 without the loop overhead, FIGnition is 36x faster.

Dragon 32: 14.72s, or 11.72? without loops. FIGnition is around 43x faster.

I also tested the program against a classic Macintosh running Think Pascal under vMac. Here, the run-time was 30.75 or 27.2s without the calculations, making FIGnition about 10x faster.

-cheers from Julz

[Jeff]

Do you actually own all these machines?!

On Wednesday, 16 April 2014 16:58:34 UTC+1, Julz wrote:

Hi folks,

I've also been comparing floating point calculations with other 8-bit computers (and a Mac Plus). On the 8-bit machines I do 1000 loops and the program is written in BASIC. Even for BASIC, floating point calculations will dominate the timings:

FIGnition: 3.3s, or 2.7s for the calculations themselves.

Jupiter-Ace: 14.18, -0.6s =  for calculations themselves, 13.58. FIGnition is 50.2x faster.

ZX Spectrum: 14.94s, really 10.14s without the loop overhead. FIGnition is 37.6x faster.

Acorn Electron: 8.06s, about 6.56s without the loop overhead (assumed to be 1.5s). FIGnition is 24.3x faster.

VIC-20: 10.91s, about 7.91s without the loop overhead (loops are 3s). FIGnition is 29.3x faster.

Oric-1: 15.75s, or 9.75 without the loop overhead, FIGnition is 36x faster.

Dragon 32: 14.72s, or 11.72? without loops. FIGnition is around 43x faster.

...

[JESSON SEAN]

And do you still have the PDP-11?

--
You received this message because you are subscribed to the Google Groups "FIGnition" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fignition+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Julian Skidmore]

Hi Jeff,

Great to see people reading the updates!

I'm afraid all the other machines were timed in emulation, but using the emulated machines' internal clocks; e.g. locations 274, 275 on the dragon 32 and the TI command on a VIC-20.

Nevertheless, I have a ZX Spectrum (natch), C64, Oric 1, Apple II, Acorn Atom (not tested) and a real, working (well, when last tested) Mac Plus.

Hi Sean,

Yes I still have the micro-pdp11/73 (and micro-Vax II) at my parents' house :-) . Software floating-point on the pdp-11 would be somewhat slower than a Mac Plus, but both the pdp-11 and Vax have FPUs which are much faster than FIGnition's code. For example, the pdp11/34 FPU single-precision ops managed 8.9µs (minimum) for add/sub and 16.2µs for mul/div. So FIGgy's f* will be about the same; f/ about half the speed; f+ takes an extra 1µs per denormalization / normalisation bit shift compared with 0.24µs on the pdp11/34. So sometimes f+ will be OK, but often FIGnition will be much slower.

We can also compare FIGnition floating-point arithmetic to, say, an original 8087 hardware unit. The 8087 took 70 to 100c for fadd ( 14µs to 20µs); 90 to 105 for fmul ( about 20µs) and 193-203 for fdiv ( about 40µs). For my bench test, this would be an average of 40KFlops, about 2.7x faster than FIGnition.

-cheers from Julz

[Si Brindley]

Julz do you have a sense of how close to the theoretical limit for the chip you are?

—
Sent from Mailbox for iPhone

<FIG - black on whiteMini.jpg><NmLogoMini.jpg>

[Julian Skidmore]

Hi Si,

Good to hear from you! Do you mean are we near the theoretical limit for capability of the firmware or floating point performance?

-cheers julz

[Si Brindley]

I mean floating point performance (at the current clock speed).  I’ve been following along, by the way; just had my interest irresistibly piqued by the comparison to the old computers :)

[Rob Fielding]

Hi

Jules,

Apologies, I know too well that non-response not being a signal no one is reading, but isn't a terrifically warm confirmation of the excellent work you've been doing. I've been following along too, and had a reply drafted but didn't work it into a reasonable question.

First thing: awesome work tuning up.  This level of craftsmanship is the quality so lacking in many IT projects.

I was telling someone recently about educational computing, and how RPI and Arduino have their position in this, but if you want a true electronics to computer experience, Fignition offers so much more. 

We've just started sleep training for our 5mnth old, but Ive got a plan to incorporate Fignition as an education tool when he's old enough. 

So how is your roadmap? We all want to see an awesome success for you, and there's so much which is part of that. 

- The board and the soldering experience

- The plug in and programme experience

- The "official book" of the design story, much of which could be taken from you wonderfully in depth posts on this very mailing list.

- Supporting those big ticket items: audio loading

So what else is on the roadmap for 1.0.x  ?

How long before you have to handle a digital video out ?

--- 

Robert Fielding

twitter: @_rpf

[Julian Skidmore]

Hi Si,

With FIGnition's floating-point code there's now three (maybe four) Floating-Point libraries: the GCC general-purpose one which is clumsy, slow and large; there's the libm library which is part of the libc libraries for GCC, but you have to remember to link it in. There's possibly (I'm guessing) the fp library that comes with e.g. IAR or MikroC compilers.

I can only compare FIGnition with libm. FIGnition's FP is optimised for size rather than speed, it's about 60% the size of libm for the same functions. Having said that, FIGnition's f* and f/ core routines will be roughly as fast as each other (since we need the same number of mul, add, adc instruction for f* and they dominate the timing; and the algorithm for f/ will be an almost identical bitwise algorithm). f+ will be somewhat slower since my packing and unpacking isn't so fast. However, both libm and FIGgyFP use bit-wise normalisation and denormalisation for f+. This is the biggest area for improvement, because you could substitute an algorithm that handles shifts of 8 bits by copying bytes and a shifts of 0..7 bits using mul and that'd give you much faster f+ (actually f+ is probably the slowest routine in both libm and FIGgyFP).

If there's space in the firmware for that improvement at the end I might put it in (faster F+ vs dedicated sqrt? what would you choose)?

Hi Rob,

Thanks a lot for the message - some great points there - I'll try and reply soon :-)

-cheers from Julz

[Julian Skidmore]

Hi folks,

Yet more on FIGgyFP! So why did I pick sqrt as the alternate function? Because sqrt is almost as fast as division in binary, and has a similar algorithm. Consider x=25, sqrt(x) in integer arithmetic. In each step we have a 'root' which starts at 1 . we shift the next two most significant even digits of x into m and then calculate root=2r-1. If m>=root, then m-=root and root+=2.

M=01 x=[01]1001 r=2*1-1 => m=01-01, r+=2 => 11.

M=10 x=[0110]01 r=2r-1=101 ( no subtraction, since m<101)
M=1001 x=0 r=2r-1=1001 => m=m-(2r-1), r+=2, so m=0 and r=11. 

Final result is (r-1)/2 = 5. This is really just an optimised way of adding a sequence of odd numbers (which gives you the sequence of square numbers). Perhaps surprisingly, this method works for floating point values as well as whole numbers (surprising, because (a) square roots of values <1.0 get larger; i.e. towards 1.0 and (b) the evaluation leads to irrational results for non-square numbers).

-cheers from Julz

[Julian Skidmore]

Hi folks,

Another update, on maths; then the audio code.

fsqrt will take in the region of 66µs if it's done in assembler. However, there are other ways of doing it. One way is to compute is as exp(ln(x)/2), which is simple, but slow. In Forth this would be:

: fsqrt

  ln 2.0 f/ exp

;

This weighs in at 13b for the actual code. It's slow because it involves two transcendental functions (both of which, if implemented as Chebyshev polynomials would result in dozens of floating point operations. One ZX81 variant uses a Newton-Raphson approximation to compute square roots. Newton Raphson is a simple method which extrapolates a straight line from a point on a curve (x',y') to y=0, with the slope = the slope of the curve at (x',y'). For square roots, calculating the slope is easy since it will be 2*the current x coordinate. (d/dx of x^2 = 2x).

Let's say we want to find fsqrt(0.7). The full calculation is x=sqrt(0.7). If we square both sides we get x^2=0.7 and then by subtracting 0.7 on both sides we get x^2-0.7 =0; which we can solve by extrapolating a line on the curve to y=0. This is shown below:

If we start with our first guess at 0.7, then 0.7^2-0.7 = -0.21, the slope will be 2x0.7 = 1.4 and when we project the line from the slope it hits y=0 when x=0.85. So our second guess will be 0.85, which is much closer to where the y= x^2-0.7 graph actually meets 0. If we repeat this 3 or 4 times we'll get to the right answer. Approximating like this works well here because straight line slopes hug the curve quite well. It wouldn't work very well near x=0, because the slope of the graph there is almost horizontal, so a guess near 0 would lead to a line that hits 0 a long way from the correct result..

A FIGnition Forth Newton Raphson routine therefore has two stages. We first modify the input's exponent so that the input is in the range 0.5 to 1.9999 . This is because if a floating-point number is 2^(exponent)*mantissa then the square root will be: 2^(exponent/2)*sqrt(mantissa). Then we repeat the slope calculation until the adjustment is 0 and finally add back half of the original exponent. This code looks like this:

: fsqrt

dup 0< if

2drop 1d ;s ( overflow if <0)

then

dup 1 >> $7F80 and >r ( x : exp(x)/2)

$FF and fneg 2swap ( -m m : exp(x)/2)

0d ( -m m dx : exp(x)/2)

begin

f- ( -m m’-dx=>m’ : exp(x)/2)

2over 2over ( -m m’ -m m’: exp(x)/2)

2dup f* f+ ( -m m’ m’^2-m: exp(x)/2)

2over $80 + f/ ( -m m’ (m’^2-m)/(2x)=>dx : exp(x)/2)

dup until

2drop 2swap 2drop ( m : exp(x)/2)

r> +

; ( 59b+header = 68b)

It's much faster than the ln.. exp method, but still takes about 1.2ms per square root.

The final method is the assembler one, which will take about 66µs, about 20x faster!

Audio In/Out

***************

At the moment I'm back on the audio code, writing the audio out routines. Audio in had largely been written, but needs some tweaking.

-cheers from Julz

[Julz]

There should be a graph in the post above!

[Julian Skidmore]

Let's try again!

-cheers from Julz

On Wed, Apr 30, 2014 at 3:19 PM, Julz <theorigi...@gmail.com> wrote:

There should be a graph in the post above!

--
You received this message because you are subscribed to the Google Groups "FIGnition" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fignition+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Julian Skidmore]

Hi folks,

Yet more Maths. I recently came across the algorithm for binary logarithms! Let's say we want ln(y). It can be calculated as log2(y)/log2(2.7182818..). So binary logs are pretty handy. You can calculate log2(y) pretty easily too. The format for a floating point number y will be 2^exponent(y)*mantissa(y). Log2(2^exponent(y)) is simply exponent(y). To calculate log2(mantissa(y)), where mantissa(y) is in the range 0.5 to 1.0 is:

: log2mantissa ( m)

  0d ( m log2m)

  25 0 do

    2dup d+ 2swap ( log2m m)

    2dup f* 2dup 0.5 f< if

      2.0 f* 2swap 1d d+ 2swap

    then

  loop

  2drop >r 7 u* r> 7 u* d+

;

Basically, you square the mantissa and if it's <0.5 you double the mantissa, then double the log and add 1 else you just double the log. So, each pass through the loop generates a bit of the result. You can see how it'd work with 0.5 and 0.999999: 0.5^2 is always < 0.5 so the next bit is always set and the final result will be 0.99999... and the true log2 is -0.999999... On the other hand, 0.99999... when doubled is always >=0.5 so the final results will be -0.000000..., which is correct.

It's nice and neat, the only downside is that it requires from 25 to 50 floating point operations. The entire ln algorithm would then be:

: ln ( y)

  dup 0< if

    2drop 1d ;s ( overflow if <0)

  then

  dup 7 >> >r $7F and ( m : exponent)

  log2mantissa fneg f> float f+ ( log2[y] )

  log2e f/

;

However, I think I'll be using Chebyshev for ln and exp.

-cheers from Julz

[Julian Skidmore]

Hi folks,

Here's a small weekend update. I realized there's 25 interrupts (excluding Reset) on the AVR which meant that if anyone tries to use SPM Ready as a user-int (unlikely) FIGnition wouldn't be able to handle it. I spent a bit of time reducing the number of RAM bytes used by the FIGgypad driver and also moving a couple of them to sysvars ( helpCount is now at sysvars+28). In the process I also saved 64 bytes in the FIGgypad driver itself :-)

With the RAM bytes saved I added one to the userInts in sysvars. This means there's room for 32 user ints. The top 7 can be used to trigger events so if you do:

128 $FF 26 sysvars + >port>

It will generate a user-interrupt from user-code; handy for simple event schedulers (such as one that's likely to be in the audio data transfer code) :-)

-cheers from Julz

[Julian Skidmore]

Hi folks,

Monday update, a bit of a milestone! I've managed to write all the audio data transfer code :-) Woohoo, that's a great step forward!

I did quite a lot of work on compacting it, though I may still have more to go, but the code is written for the audio upgrade bootloader; audio transfer to and from RAM as well as audio transfer to and from flash.

At the end of the development, as is fairly usual there are always a number of additional, minor items that need to be supported, it feels like you've completed it, but then you discover more things need to be done!

These were the last lot of changes I identified (a copy/paste from my journal):

√ I/O Register setup for the upgrader code.

√ I/O Register setup in forth for ear and load”

√ Visual feedback. At the moment we don’t have any, but I want a simple output routine on the LED to say if loading is OK. How? Originally I wanted to feedback at 1Hz or 2Hz etc. Loading rate LED is OK. Works for upgrade mode and in Forth as it’s part of TapeIn.

√ Capture lead is poor, it only expects one lead bit. Capture lead now expects  >= 16 bits, and will be reset anytime there’s a bit that’s out of range or a 1 bit and capture lead<16bits. LED flickers at 15Hz during capture lead.

√ Scatter loading support. Using a bitmap of load packets in internal RAM we can set each bit when its block has been loaded correctly and when all the blocks have been loaded OK, loading is complete. Works in upgrader, not yet in Forth.

√ Need to report via LED a loading failure, it just goes back to the beginning, waiting for a header, this will cause the LEDs to flicker. Works in upgrader, and Forth (in text) √ .

√ Upgrader returns to FIGnition, resetting vectors when done.

√ Being able to change the loading/saving bit rate from Forth.

√ Reclaim back to block in Forth.

As you can see from the ticks, it's now all done :-)

I think I'm now ready to start the next stage, first - getting all the code compiled (which mostly means getting the forth code compiled since the 'C' bootloader code compiled fairly recently and there haven't been many changes since); then starting the testing and debugging process itself. Judging by the rest of the size of the code I'm not sure if both Flash and RAM loading can be supported, though there may be ways around that (e.g. flash loading could be supported in terms of a RAM load, whose first block is the code needed to support Flash loading or by supporting only Flash loading/saving and RAM audio I/O would be additional code you'd load in from Flash). Anyway, we'll see!

-cheers from Julz

[Rob Fielding]

Woo!!!

> √ Visual feedback. At the moment we don’t have any, but I want a simple output routine on the LED to say if loading is OK. How? Originally I wanted to feedback at 1Hz or 2Hz etc. Loading rate LED is OK. Works for upgrade mode and in Forth as it’s part of TapeIn.

For my 2c, a slow and discrete patterns of LED flash is easily documented, and importantly communicate back by the user to the mailling list. Imagine a simple cut sheet listing the half-dozen different states, with "what to do next".

A rapidly flickering LED (imagine the LED representation of the Spectrum loading "noise" (which I like btw)), might tell an engineer more about what's happening, but a non-engineer may be unable to communicate the important points.

Just an idea, to exercise the point,

quick double pulse for loading (or flicker)

slow blink for error

solid for done, or ok

[Julz]

Hi Rob,

Yes, and I also want the LED to show what's actually happening, rather than a fake representation.

For example, at the moment the LED flips state every 1Kb that's loaded when loading packets and it flips state at 17Hz when waiting for the leader tone before every packet (there's also a long leader tone at the start). This has the following effects:

1. If the tone level is too low or too high, the LED will stay in its current state.

2. If the loading gets stuck in the leader state then it'll just flicker quickly forever.

3. If loading is correct, you'll see it invert approximately every 0.5s.

Using this you should be able to adjust volume levels and monitor progress.

(which is probably a bit too fast, I should slow to 8Hz)

On Tuesday, May 13, 2014 12:23:25 AM UTC+1, Rob Fielding wrote:

Woo!!!

> √ Visual feedback. At the moment we don’t have any, but I want a simple output routine on the LED to say if loading is OK. How? Originally I wanted to feedback at 1Hz or 2Hz etc. Loading rate LED is OK. Works for upgrade mode and in Forth as it’s part of TapeIn.

For my 2c, a slow and discrete patterns of LED flash is easily documented, and importantly communicate back by the user to the mailling list. Imagine a simple cut sheet listing the half-dozen different states, with "what to do next".

A rapidly flickering LED (imagine the LED representation of the Spectrum loading "noise" (which I like btw)), might tell an engineer more about what's happening, but a non-engineer may be unable to communicate the important points.

Just an idea, to exercise the point,

quick double pulse for loading (or flicker)

slow blink for error

solid for done, or ok

-cheers from Julz 

[Julian Skidmore]

Hi guys,

My audio firmware compiles so I'm on to testing :-)

To do this, I've started writing a desktop java app to convert forth and hex files into audio. This is also my testing app, but it'll have a tape-deck based GUI. Stay tuned for progress :-)

-cheers julz

--

[Romilly Cocking]

Hi julz,

I love the idea of a tape-deck based GUI for the converter/tester.

It is inspiring to watch your development in near real-time :)

Thanks for the effort you are putting in to this.

cheers,  Romilly

[Julian Skidmore]

Hi folks,

Thanks for the comment Romilly!

Today I've managed to do some more work on the converter/tester. I can get it to open and play both raw .wav files and generated audio (tested). I can also change the sample rate for the audio and I have a slider to control the volume (though as yet it only acts the part, it doesn't really control volume :-D ).

I've written and compiled the code to open and convert text files (e.g. Forth files) into the right audio format for FIGnition.

In theory, this means I'd be able to open a Forth text file (or binary file) and play it in FIGnition's audio protocol and load it into FIGnition at this point. Except of course this new code hasn't been tested at all so there's bound to be quite a lot of bugs in both ends of the software.

Good progress though :-)

-cheers from Julz

[Julian Skidmore]

Addendum: one of the really interesting things about loading into external ram is that it should be possible to stick up to about 8kb of source text into the data and by overwriting the TIB get the computer to auto-compile it (and auto-run it ;-) ). To an extent this gets around the problem of binary RAM programs being specific to a firmware version (because the firmware compilation addresses change with new firmware).

-cheers julz

[Julian Skidmore]

Hi folks,

Today's progress: the desktop java app now appears to be encoding .fth files correctly with a correct header; lead tones for each 64b packet and CRCs as well as outputting it as audio. It still doesn't allow me to change the volume level, but that shouldn't be too hard.

The next stage then is to see if I can start to pick up the audio and process the data correctly with a FIGnition!

-cheers from Julz

[Julian Skidmore]

Hi folks,

I got the volume control working and then I decided to produce a complete checklist to be able to deliver firmware 1.0.0. The summary is:

Stage 1, loading RAM images: 7 steps.

Stage 2, saving RAM images: 8 steps.

Stage 3, loading to External Flash: 5 steps.

Stage 4, saving from External Flash: 3 steps.

Stage 5, upgrading FIGnition firmware via audio: 5 steps.

Stage 6, generating final 1.0.0 ROM images (i.e. PAL16, PAL32, NTSC16, NTSC32): 6 steps.

Stage 7, developing the FIGnition firmware and bootloader upgrader firmware: 7 steps.

Stage 8, Final release procedures: 4 steps.

After this, Firmware 1.0.0 will be finally out there. Most of these steps are testing steps and some of them are fairly 'small' (for example the first step in stage 1 is to make sure I can activate the audio interrupt without it going off if there's no audio signal). Some of the steps involve additional development (for example the Desktop Java app can currently convert text and .fth files into audio, but it can't convert .hex files and can't input audio and convert it). I also of course need to write the USB-loaded FIGnition firmware image that will load firmware 1.0.0 via audio, to external flash before finally overwriting the bootstrap and run the upgrader [ it will work a bit like the FIGkeys upgrader; fully checking the image and asking you before you finally overwrite the bootloader!].

Finally, finally I'll need to write the documentation and package it all together for the 1.0.0 release!

So, there's still a bit of work to go, but I'll be able to give you the progress as I go along!

-cheers from Julz

[Julian Skidmore]

Hi folks,

After doing the checklist I started on some debugging for the first step. Guess how far I got?

Answer: Nowhere, I spent 3 hours debugging another error without even getting to the first step.

This is what happened. I compiled a new firmware image (at the moment I'm adding all the new code to a 32Kb image so I can support USB and the new audio code as I debug it). And when I tested it, it booted but would hang if I tried to run any commands.

I'd had a similar problem a while back, I'd add a command to the Forth ROM, but when I booted and ran vlist it wouldn't appear and other words also might not appear. It turned out to be a problem with my Forth Firmware Compiler, it could get the linkage of commands out of order if some other commands were conditionally compiled out. I'd fixed that problem a while back so it wasn't that (though I spent a bit of time checking).

In the end it turned out to be a problem with .align 1 in AVR assembly code embedded in a Forth command that was conditionally compiled out, FFC would always include the .align 1 in the relocation calculations.

It took me quite a while to find that. But again, all this goes back to problems with the gcc assembler I encountered at the beginning of FIGnition's development. At the time, the whole of the Forth ROM part of the code was written in assembler code, though really it consisted entirely of .byte for byte codes and .word for addresses statements.

AVR assembler is little-endian so normally you'd want addresses to be stored in a little-endian format. But of course I wanted most addresses to be stored in a big-endian format, because that's how it works with FIGnition. And despite the "everything and the kitchen sink" approach to the GCC assembler there's no way of telling the assembler if any addresses are supposed to have a different endianness.

Which is amazing - it means no-one in the decades-long history of using GCC ever wanted to stick a value in assembly code with a different endian order than the native order! Is that likely? I don't think so, I think it's more likely that quite a number of people wanted to, but modifying the set of GCC tools to support a means of swapping byte orders was too daunting for the hacks they needed to do. So it was never done.

And I didn't do it either. In the early FIGnition versions I compiled the Firmware twice but with the Forth ROM offset by 256 bytes and then I wrote a hacky 'C' program to search for the inconsistencies and swap words where they were found.

When I moved to the Forth ROM in (by and large) Forth I added code to directly work out where to swap bytes. Hence the need for relocation information; hence 3 hours of debugging today fixing a problem that should never have been needed in the first place.

Rant over :-)

-cheers from Julz

[Julian Skidmore]

Hi folks,

I've had a fairly unfruitful few days. I've been trying to get my FIGnition to recognize a leader tone, but I couldn't get it to work consistently: it would keep jumping and missing transitions every so often. I found another way of turning off the video without stopping Timer1, instead I turn off the timer1 interrupts.

I traced it to what appeared to be an issue with Java, problems with buffering the tone data. The waveform started off OK, but after a few seconds, millisecond glitches would start to appear every 50ms or so. I thought, maybe it's core audio on my Mac mini or something or another app that outputs Audio conflicting with it somehow. So, I retried it on different platforms.

After a while, I was able to get my iMac G5 under Mac OS 10.5 to output a solid waveform with the java app. However, on my Mac mini using the same app it still generated errors.

Yet the curious thing was that if I sampled the output using a jack-to-jack lead from my Mac mini into the iMac G5 the glitches on the waveform disappeared!

In the end, it turned out to be the cable. If I plug in a mono jack the glitches appear. Plugging in a stereo jack reveals that each of the channels correctly output the waveform! I think the Mac mini must be dynamically testing for stereo/mono connections or something!

But at least now I might be able to make some progress!

-cheers from Julz

[dla...@gmail.com]

Julz:
  Many thanks for your time and effort. I have to remember why it is so easy to use FIGnition -- your hard work!

While reading your posts on the audio interface being developed I recalled the Kansas City Standard for storing data on audio tape.
Here's the Wikipedia article: http://en.wikipedia.org/wiki/Kansas_City_standard but I found something you may have already seen
by Ed (?) at http://www.netbay.com.au/~dxforth/ -- "KCS v0.8 - Kansas City Standard and CUTS tape decoder/encoder".
It's some forth code for encoding and decoding forth programs.

Do you think it will be faster than 300 baud? ;)

Donald

[Si Brindley]

Julz, if you haven't already, have a play with "Audio MIDI Setup" in your Mac's

Utility folder. 

Also, in the "Accessibility" System Preference pane you might find an audio setting "Play stereo audio as mono". 

Disclaimer: I'm on my iPhone right now so I haven't checked these.

Anyway, I can't wait for my first "R Tape loading error, 0:1"

[carl]

Hi Julz,Si and Donald.

Another way of converting Stereo to mono is by using Audacity, you can turn the whole track into a mono recording and resave it.

It could be that the sound is biased to the redundant section on your stereo Jack lead.

Remember to put tippex on the volume control when you get the levels right :-)

Regards.

[Julian Skidmore]

Hi guys,

Firstly, Donald. CUTS was used in a number of early PCs, but later computers shifted towards a simple zero-crossing technique as it allows both higher speeds and simpler hardware support.

http://www.worldofspectrum.org/tapsamp.html

I use a threshold technique, a capacitor and pair of resistors hold the normal voltage of an audio input at 2.5v and when an input voltage of +/-1v is put through the circuit it should generate a swing of up to +/-2v, enough to shift the input from between 4.5v down to 0.5v and this can be detected digitally.

My testing baud rate is effectively about 8268.75 for an equal number of 0s (5.5KHz) and 1s (11KHz). I need 8 bits + 2*'1' or 1*'0' bits of data per byte (there is a 'phase' bit at the beginning of each byte). If I can get it to work at that rate I'll see how high I can go, though I expect 22.5KHz to be the highest (about 2Kb per second). I can take the system down to lower baud rates if we wanted to use real tapes! (or Vinyl ;-) ).

Secondly, Simon: I haven't tried that, but I will. At the moment it means I'll have to take a stereo to stereo phono splitter and then just use one phono. On the iMac G5 I wouldn't need to do this, it's only a problem on the mac Mini. At the moment the basic tape error message is " Tape not"

Which when executed will cause "Tape not OK" to be displayed :-D

Thirdly, Carl: Yes I was using Audacity to generate example tones in some of my tests when I was thinking it was a Java issue. The Java app should be able to generate audio directly (it currently does) and also audio files (.wav/.aiff). The hard bit in Java is the garbage collector, even adding a few local variables in the processing loop was enough to cause glitches in the audio output!

-cheers Julz

--
You received this message because you are subscribed to the Google Groups "FIGnition" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fignition+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Julian Skidmore]

Correction: CUTS was used in a number of early 70s microcomputers (which people called Personal Computers at the time, even before the IBM PC).

-cheers from Julz

[Stuart Taylor]

Hi,

Correct me if I'm wrong, but don't apple laptops etc allow you to plug in the apple iPhone headset, which you can use for Skype etc. If I'm right, then that audio out, isn't an audio out, its a little bit more.

Stuart

[Julian Skidmore]

Could be :-) !

[Julian Skidmore]

Hi folks,

A little update for Thursday. With a stereo 3.5mm jack to dual mono phono I was able to get FIGnition to recognize a lead signal; a start pulse and now a single demo byte of data. Next up, a whole packet of data!

I'm still having issues with the non-realtime nature of Java so I'm having to remodel the Java code that converts text files to audiowaveforms. I'll have to pre-generate the bitstream. However, if I simply created a 44.1KHz x 16 bit stereo bitstream then I'd end up using 44100*4*32 = 5.6Mb files for a 21s file; which sounds big to me. Instead I'll do an intermediate conversion which I can output in realtime.

-cheers from Julz

 

[Romilly Cocking]

Julz,

Forgive me if I'm missing the point, but are you sure it's Java that's the barrier? I'm guessing that you are using a standard (i.e. non-real time) OS on your host, in which case any userspace code could get pre-empted when it's running.

If Java is the problem, ISTR there is stuff you can do to minimise or eliminate the risk of triggering garbage collection while your code is running. It's a long while since I had to wrestle with Java performance and I may be way out of date.

Let me know if you'd like me to try to restore the relevant bits of knowledge from my brain archive :)

Romilly

Romilly - @rareblog

[Stuart Taylor]

Hi Julz,

Can you expand on that a little?

44.1khz is your sampling frequency?
32 is your two (for stereo)16 bit channels?
What is the 4? 4bit word?

I don't know which tones you are using, or which modulation technique you are using, but reducing the sampling frequency would reduce the file size.

Stuart

[Julian Skidmore]

Hi Stuart,

Yep, I was getting my figures muddled :-)

The 32 should be 21 for 21s. On average, 21s=21/1.5 = 14s worth of bits if they were all 1s. I'm currently planning to send 5.5K '1' sample bits per second so that would be 14*5.5 KBits/9 = 8Kbytes worth of source code; about 16 blocks, which would be a typical FIGnition program.

-cheers from Julz

[Si Brindley]

What, no error checking?

<FIG - black on whiteMini.jpg><NmLogoMini.jpg>

[Julian Skidmore]

Hi Si,

aaaah... details, details ;-)

Here's a few more details on the format.

Sample Periods: These will always 44.1KHz/(2^n). For testing, n=3, so a series of 1s have a 5.5ishKHz frequency and 0s are at 2.75625KHz.

Bit Level: Bits are defined by signal transitions, a '1' bit is a single sample-time transition and a '0' is a double-sample time transition. So, e.g. if _ is low and # is high and we start low then 10101101 would be _##_##_#__# .

Byte Level: A byte is transmitted as [0 or 11] followed by bit7..bit0. I wanted to keep the average voltage as close as possible to 0, so every 8 signal transitions is preceded by one or two polarisation bits. For example, 10101101 has more #'s than _'s, but if we precede it by a __ then the same byte becomes #__#__#_##_ . If we'd preceded it with _# (which would have taken the same amount of time and would correspond to a pair of 11s) then the data itself would be back to: _##_##_#__# . The polarisation bit allows us to re-bias the voltage in the direction we want and it happens as a natural consequence of just inserting the one or two polarisation bits.

Packet Level: A packet is consists of a leader, which is 48 0's followed by a [11] polarisation bit pair and then byte0 to byte63 then a two-byte CRC for error checking. The clever thing though is the choice of transitions for a leader and polarisation. That's because a series of 48 0s are a series of __##__##__##__## ... and because they don't change the overall bias of the signal if it was data then every 8 bits would need to be followed by a [11] polarisation pair to denote a bias that doesn't change, a continual sequence of __##__##__##... would mean you keep changing the polarisation every byte. This means that a series of __## longer than 3 byte's worth can never occur as part of a data sequence and it can be used as a leader.

A Header packet: struct { byte packetType; ushort packetCount; byte name[61]; } . packetType=1 is a RAM data, 2 is external Flash, 3 is an upgrade file type. The maximum file size is therefore 4Mb.

File Level: A 'file' is transmitted as a 2 second lead tone followed by the header packet followed by all the data packets and terminated by another 2s lead tone.

Hope this helps :-)

-cheers from Julz

[Si Brindley]

Can those first 64 bytes contain the program name then?

[Julian Skidmore]

Hi Si,

The first 64 bytes contain the program name (or file name). So if you open the file from the java app the name part of the header will be the name of the file (except for the suffix). If you type:

mic MyProg

The header will contain the name "MyProg"

Does that help?

-cheers from Julz

[Si Brindley]

Like this? :)

[Julian Skidmore]

Ha! Excellent!

I don't have a border effect (which is actually done by copying the ear input to the border port). And most of the reason for this is that video scanning takes too much time. However, by turning off proper video scanning apart from HSYNC and then converting the timing between each transition into an interrupt period on a scan line I can represent audio input onscreen.

So yes, a little like that: more informative than a ZX81, but less than a Spectrum :-) I'll post a bit of a video in due course!

-cheers from Julz

[Julian Skidmore]

Hi folks,

I've managed to get the firmware to load in a single packet - so that's a bit more progress :-)

-cheers from Julz

[Mark Wills]

You're properly coding down-on-the-metal now ;-)

Cups of strong coffee (try it ice cold with milk/cream - it's good) and your favourite music in the background helps!  I'm quite partial to some Dire Straits personally. I can recommend the Love Over Gold and Brothers In Arms albums :-)

Good luck. Let us know if you need us to send in supplies. I have baked beans and toilet rolls a plenty ;-)

Mark

[Julian Skidmore]

Hi folks,

Correction: It's managed to load in a single packet CORRECTLY, including the CRC!

To help you decode the image. Firstly "VIGeition" - no I haven't renamed the product ;-) I've just been using the first and third characters for debugging information :-)

The real packet contents start on the first line, and there's 64 bytes. The first character, which looks like a space is actually a byte value of 2. It's then followed by the number of blocks (0 for an empty program). Then the name of the file = EmptyForthProgram.fth and then there's a load of 0s (except the packet info is placed in UDG0 and UDG1 so it looks funny). Finally there's the crc which is $8619 or the half-checkerboard and 'L' graphic together.

The scrambled data below is just a previous attempt to download a packet - I'd accidentally added a pair of '1's at the beginning in the Java program so everything was picked up wrongly :-)

Oh the joys of serial transmission!

The screen displays "Tape not OK" at the end, and that's because it's returning 'failed' by default at the moment.

-cheers from Julz

[Julian Skidmore]

Hi Mark,

LOL - excellent! :-)

-cheers from Julz

[Julian Skidmore]

Hi folks,

And what you're about to see is the best crc16 algorithm in AVR assembler I know of.

movw rTmpLo,rCrcLo

mov rCrcLo,rTmpHi

mov rCrcHi, rTmpLo ; crc  = (unsigned char)(crc >> 8) | (crc << 8);

eor rCrcLo, rData ; crc ^= ser_data;

ldi rTmpHi,16 ; These 2 instructions are faster than executing 4 times 'lsr 4'

mul rTmpHi,rCrcLo

eor  rCrcLo, r1 ; crc ^= (unsigned char)(crc & 0xff) >> 4;

mul rTmpHi,rCrcLo ;r0= (crc&0xff)<<4.

eor rCrcHi,r0 ; crc ^= (crc << 4) << 8;

ldi rTmpLo,32

mul rTmpLo,rCrcLo ; (crc&0xff)<<5

eor rCrcLo,r0

eor rCrcHi,r1

clr r1 ;  crc ^= ((crc & 0xff) << 4) << 1;

ret

Most byte-wide CRC16 algorithms you see use 2x 256 x 2byte tables = 1Kb of tables and most other algorithms are bit-wide. However, there was a space efficient byte-wide computational algorithm that was developed in the 1970s and instituted as a BSI standard. It was so old, it was targeted at Motorola's first 8-bit CPU, the 6800 (not 68000!). I came across it in the mid-90s when working on the Heathrow express, because a previous generation engineer who had worked on it shoved the BSI book in front of my face and wouldn't back off until I'd promised to use it :-)

This algorithm is making a comeback. For example, you can find an AVR version here:

http://embdev.net/articles/CRC-16-CCITT_in_AVR_Assembler

That's the version mine's based on, except that my inputs are passed as parameters. His (hers?) is 18cycles and 18words not including the initial Crc swapping. Mine's 14c and 11 words. His/hers is 26cycles and 26 words if you include the swap and parameter return, mine's 17cycles and 14 words with the same criteria.

So, there you have it, a CRC16 algorithm that uses an astounding 2+1/16 cycles per bit.

In forth, the algorithm on FIGnition is:

: crc ( data crc)

  256 u* or xor ( swap crc hi and lo bytes)

  xor ( xor in the data)

  dup 255 and 4 >> xor

  dup 255 and 12 << xor

  dup 255 and 32 * xor

;

It's a bit slower in Forth ( 33b about 60us per byte is my guess)

-cheers from Julz

[Carl Attrill]

But does it play games? 

[Mark Wills]

Wanna see mine, baby? :-)

IN FORTH!

Check out the code-poetry herewith:

: >HASH ( c-addr len -- u)
  \ hashes a string using the CRC-16 algorithm
  $FFFF             \ intial CRC16
  -ROT              \ move it out of the way
  OVER + SWAP DO    \ for each byte in the string
    I C@ XOR        \ xor with CRC16
    8 0 DO          \ for 8 bits in the byte
        DUP 1 AND   \ note the LSB prior to shift
        SWAP 1 >>   \ shift the CRC16
        SWAP IF
            $A001 XOR \ if LSB was 1 then apply polynomial
        THEN 
    LOOP
  LOOP ;

Oh yeah baby!  That's what I'm talking 'bout!

That's actually the Modicon MODBUS CRC-16 algorithm. Used for serial communications with PLC's and lots of other industrial type controllers. It's quite possible your algorithm is the same one!

Ace!

Mark

[Julian Skidmore]

Shouty shouty forth ;-)

: >HASH ( c-addr len -- u)
  \ hashes a string using the CRC-16 algorithm
  $FFFF             \ intial CRC16

 -ROT

  OVER + SWAP DO    \ for each byte in the string

    I C@ SWAP
    DUP 8 LSHIFT SWAP 8 RSHIFT

    XOR ( swap crc hi and lo bytes)

    XOR ( xor in the data)

   255 SWAP

    2DUP AND 4 RSHIFT XOR

    2DUP AND 12 LSHIFT XOR

    2DUP AND 5 LSHIFT XOR

 LOOP ;

Would be a standard forth version of the byte-parallel version of the algorithm, assuming shifts are less costly than u* or *.

-cheers from Julz

[Julian Skidmore]

Hi folks,

A good bit of progress now: I'm able to load in small programs directly into RAM. I added a Command Line Interpreter feature to the java app and using it can type a small program, e.g:

: hello ." Hello World."

load it into a FIGnition via audio with the FIGnition command

6 ear

and when it loads, it'll compile the hello command which I can then run.

The same technique (in theory) can load programs as text: the first part loads in a short Command line which contains:

packet 7 + ic@ ear

addr tib !

packet 7 + ic@ fetches the original loading rate and uses it for the loading rate for ear again. When it's finished loading (which loads text to the end of RAM), it then sets the input buffer to the given address and so FIGnition will just start compiling the program from text. In this way it's possible to load source programs directly into RAM and compile them on the spot (but the programs can't be too big as most FIGnitions only have 8Kb of RAM).

Wacky huh :-) ?

I haven't tested anything longer than a single block of 3 packets yet, so I don't anticipate this next bit working until tomorrow. I have however, tested this process working 10/10x in succession, which means loading confidence is up to 1920bytes without error.

-cheers from Julz

[Julian Skidmore]

Hi folks,

I've been able to get to that next stage: I can now load entire programs as text into RAM and compile them :-)

Wow!!!

-cheers from Julz

[Mark Wills]

This is a mega achievement Julz ' congratulations!

Mark

[Romilly Cocking]

What Mark said.

@rareblog http://romillys-robots.blogspot.co.uk/

[Julian Skidmore]

Hi guys,

Thanks very much :-) It's been a major goal since RevE boards were first produced and finally I'm getting there! My next set of steps is to be able to save programs from RAM (in this case they'll be saved as binary programs).

-cheers from Julz

[Julian Skidmore]

Hi folks,

So, yesterday and this morning's progress is that I've written the audio input code for the Java app and I've tested that it can sample raw audio.

I've also written the audio decoder java software too, so in theory I can take .wav file containing a program in a FIGnition audio format, sample it into the Java app and then decode it into header and data blocks. This isn't tested, yet - that's the next job.

One of the handy things now though is that since I have the FIGnition 'C' and forth code which can correctly decode audio, it makes it easier to write a workable decoder in Java. Similarly, the debugged audio output code in Java will help me with the audio output code on the FIGnition side. The audio development process is getting a little faster :-)

-cheers from Julz

[Julian Skidmore]

Hi folks,

So, the java program is now able to decode the bits from an audio wav file, and correctly decodes the first 64-byte frame, with I think a correct checksum. However, it doesn't yet process subsequent frames correctly (it carries on, incorrectly decoding data into the header frame).

-cheers from Julz

[Julian Skidmore]

Hi folks,

Last evening I was able to get the Java program to properly decode an entire wav file (played out through the line out using Audacity and sampled back into the Mac's mic input by the Java app). This morning I've added the code to translate the decoded wav file (which may consist of more than one header frame + body frames) back into a text file.

This means I can now take an original Forth program as text (e.g. Blitz.fth); convert it to audio (here sampled by Audacity and saved); then play the audio back and convert it into exactly the same text file (Blitz.fth).

I'm now in a position where I can go back to the FIGnition and get it to generate the appropriate audio data to sample into the Java app. That's the next step :-)

-cheers from Julz

[Carl Attrill]

Keep it up Julz, your work is a real education as to how computer work. There is a good book in you.

[Julian Skidmore]

Hi Carl!

Thanks for that, encouragement always welcome :-)

-cheers from Julz

[Mark Wills]

You're officially allowed the evening off to watch Uruguay V England ;-)

Mark

[Julian Skidmore]

Hi Mark,

England didn't do too well did it :-(

Anyway, on a more positive note: I've made quite a bit of progress today. I've managed to get through a number of the steps on the FIGnition side: starting with generating an automatic lead tone; and progressing through sending single bytes and even a whole (header) packet!

That's great progress from my viewpoint. The code is even capable of correctly sending a CRC at the end of it.

Deluded by my progress I had a go at saving an entire program (not Blitz, but the simple ChrSetDemo that's on the Google site); it kinda saves it (actually, it saved an extra 32Kb on top, but I've fixed that ;-) ) unfortunately, not all of it was saved correctly so I have a bit of work to do to fix a few errors there.

And the saving side of stuff seems to have broken the loading code, rats... aaah, but as I type, I think I know why... hang on...

-cheers from Julz

[Julian Skidmore]

Hi folks,

Yes, that was the problem with the loading side! I had stuck a bit of debugging info on the mic output routine which clashed with the input side of the code. Shifting the debugging code fixed it.

I'm now able to save a program from FIGnition and attempt to load it from the Mac. The FIGnition seems to save it OK, but the Java app seems to think everything but the first packet is corrupted (which it might be). I need a bit of a break though so I'll give you an update later!

Still, on the saving side, I think I'm almost there (oh the folly of presumption ;-) )

-cheers from Julz

[Julian Skidmore]

Hey guys,

WOW! WOW! WOW! I've Done it!!!!! This is truly Amazing!!!!

I've managed to complete a whole audio save/load sequence:

1. I loaded a program (ChrSetDemo.fth) as text into FIGnition using my Java app.

2. I then saved the program to my computer using $8000 here mic ChrSetDemoSave.bin"

3. I then loaded back the same binary (mic saves raw bytes) into FIGnition using my Java app.

I was then able to run ChrSetDemo!!!!!

What an amazing end to Friday :-) :-) :-) :-) !!!!!

It doesn't prove I can do this with all programs, ChrSetDemo is pretty minimal, but it's really exciting!

I had a few problems on the way...

Mostly, I'd forgotten that the Java app glitches at the start of playback (and probably at the start of recording too). This meant although I'd been able to load in a single header packet I kept finding that an entire binary program contained mucked up packets with junk data. By waiting a few seconds before actually sending the data from FIGnition the Java app was able to load it correctly - which means FIGnition was sending it well enough :-)

The java app saves binary images with its own 5 byte header containing the file type, packet length and start address, but I'd forgotten to take that out when loading binary images back into FIGnition. Hence the image was mucked up and FIGnition hung.

Once I'd fixed those two items FIGnition was able to load the binary image and run the program :-)

Fantastic!!!! I'll say this again FANTASTIC!!!

So, you're thinking "a release must be around the corner!!!" Well, not quite. There are still a few stages:

1. I need to test it with larger programs to prove it all works as it should!

2. I need to add support for saving and loading text programs to external Flash (this is non-trivial).

3. I need to add support for upgrading FIGnition's firmware via audio.

4. I need to generate correct 1.0.0 candidate firmware images for PAL/NTSC and AtMega168/AtMega328 FIGnitions.

5. I need to develop a custom firmware image that can load the 1.0.0 firmware images (that's because the 1.0.0 ROM overwrites the existing bootloader and you really, really, don't want to be left with a bricked FIGnition :-) )

6. Finally I need to package up the java app and add the necessary documentation for the 1.0.0 version of FIGnition.

Gosh, now it seems like I've hardly started!!?!?!

But really, this is a massive breakthrough :-)

Have a great weekend!

-cheers from Julz

[Julian Skidmore]

Hi folks,

I've done a little bit more work over the weekend and after finding a bug or 2 I'm able to save binaries at least up to 1287bytes long (the size of Blitz as compiled code, it's 2695 bytes as source).

An interesting discovery was that loading the binary image for Blitz doesn't fully restore the game, because I only saved RAM and not internal AVR RAM, and the UDGs get copied to internal RAM as part of the compilation - I'd have to copy them by hand (or add the udg cmove to the initialisation).

But this is really good! It means that from the beginning of next week I'll be on to loading and saving external Flash :-) !

-cheers from Julz.

FYI: The bug was a packet leader bug. Every byte is preceded by a phase signal: a pair of 1s or a single 0. A stream of 0 bytes longer than 2 of them would always end up having "11" phase signals, because the phase never needs to be inverted.  Therefore we can use such a stream of 0s as a lead tone as it'll never appear as part of the data and we can then start a packet with a pair of 1s.

This is one of the key requirements for reliable data transfer: when all you can read is a stream of 1s and 0s how can you tell the difference between data that happens to be 0s and the signal before any data has arrived (which might look just like 0s). Moreover, if the system misses some data, how can it resync? How can it tell whether you're in the middle of a packet (that contains 0s) or in the setup for a new packet?

FIGnition's system (like many others) enables us to tell the difference by defining the protocol so that the pattern for lead tones can never occur in a packet of data.

The mic bug was that although the first lead tone was being set up correctly, the lead tones would only randomly contain 11s to begin the next packet, so the Java program would miss the beginning of some packets.

[Julian Skidmore]

Hi folks,

Yay!! Another breakthrough!!!!

Today I'm able to load in a text file directly to external Flash storage :-) Wooohooo!!!!! I've tried it with both my little ChrSetDemo.fth program and also Blitz.fth (Blitz - it's the standard program :-D ). I was able to compile it back in again directly from flash.

So, this is really, really amazing - it means that for the first time FIGnition is able to load in the entire source code for a program in one go :-) :-) :-)

There's a bug I need to fix though :-( the loader loaded the code correctly (because all the checksums matched), but it didn't match the number of correct sectors - don't know why yet and therefore reported a tape error when there shouldn't have been. Hmmm, why was that?

Aaaah, I know, because it calculated the count based on the packets loaded, not the sectors loaded... I can fix that!

-cheers from Julz

[Julian Skidmore]

Hi folks,

I found the (well, it turned out to be a few minor interrelated bugs) and when I fixed them, I was able to complete an entire Flash load sequence on FIGnition without errors! This is totally amazing, flash loading at last! Entire programs copied to flash... in a flash :-)

The next stage is to implement external flash saving!

-cheers from Julz

[Julian Skidmore]

Hi folks,

And in a very surprising turn of events I've been able to code and debug the external Flash saving code!

This means I've now completed another couple of major steps. To recap:

1. FIGnition can now save and load from RAM.

2. It can save and load from external Flash.

The next stage is being able to do upgrades over audio! One thing that's worth mentioning is that so far I've found I've had amazing levels of reliability when transferring audio. The audio data transfer system should be capable of 'scatter' loading: if some of the packets don't match crcs then it doesn't copy them, but will load the rest and if you re-load the entire file new OK packets will be included until the entire file has been loaded.

However I'm finding I just haven't had enough data errors to try this out (my data errors so far have been when my java app had an audio glitch due to under-run).

At this point I need to explain something about External Flash loading.

To get the Firmware image to fit on FIGnition, the firmware only directly supports loading and saving blocks of RAM. However, so far I've implemented 4 types of audio loading:

1. Loading a binary directly into RAM - this is the simplest of all.

2. Loading a command line into the tib. The java app has an editable text field for typing in a command line, when you send this to a FIGnition the java app fills in the correct file type, start address and length (with the name "cli") and when ear finishes, FIGnition tries to interpret the next command in the tib, which has been reset and overwritten by the loaded command line.

3. Loading a source program into RAM and then compiling it.

4. Loading a source (or binary) into external Flash storage.

So, how is 3 and 4 done? The technique is to use the command line to correctly load the rest. To load a source program into RAM a command line is set up which contains "packet 7 + ic@ ear top @ ramFileLength - tib !" The actual source program is set to load at address 0 (which FIGnition interprets to be top @ - the File's length) and then at the end of the command line - once the source program has been loaded, the tib is set to point to the beginning of the source program which causes it to be compiled. Because it's placed near the end of RAM the compilation process shouldn't overwrite the text :-) That's the two-stage loader.

To load into Flash storage we first load a source program into RAM as described above; the source program contains the Flash loader itself (attached). Then the Flash loader is compiled automatically and made to auto-run. The first thing is does is tell you to pause the tape until it's pre-allocated all the flash blocks and when it's finished, it asks you to press a key and then press play. The flash loader loads in the tape packets and copies them to Flash. This is the three-stage loader.

Good grief, it's like bootstrapping a 1960s minicomputer ;-)

To save programs from Flash you need to load a program to do that (attached), you can of course upload it to Flash (via audio) and then you'll have it there all the time.

The upshot is that audio data transfer is more of a job than I'd anticipated. It's turning into an entire tape operating system as I've had to support data transfer across different media, rather than just being able to upgrade the internal Flash image on a FIGnition. One of the good things though is that the audio data transfer routines are accessible (well, mostly via kern vectors) and this means you can write your own custom data transfer code e.g. to transfer to EEProm or log data to the audio system.

Hope you're finding this interesting. After upgrading I have 3 more major steps before release:

* Producing a proper 1.0.0 ROM image (I'm working with the 32Kb development firmware, which contains quite a bit of debug).

* Producing the firmware upgrader image you'll need as a stepping stone to get to 1.0.0 (which involves converting Forth loader routines into 'C').

* Packaging it all up and adding documentation.

That's all for today!

-cheers from Julz

[Stuart Taylor]

FigTOS? my head asplode!

[Julian Skidmore]

Hi folks,

Progress for Friday (and Thursday). The current step is to be able to upgrade a FIGnition firmware image via audio. Unfortunately, up until now I've only been working with a debug image. There are some subtle changes to the final image, particularly for 16Kb FIGnitions, of which the most important is that the floating point and tape code goes in the bootloader section (i.e. replaces the bootloader). And also I needed to know that I can actually fit the final version of the firmware image into 16Kb.

I spent much of Thursday getting the image down to size without sacrificing any of the functionality - I did it a step at a time so that I didn't break any of the code either. Today I spent wrestling with the gcc toolchain creating the right "sections" (a.k.a. segments) and relocations so that I could build a proper 16Kb image. Which I've now managed to do :-)

In the process I decided that the mini-bootloader for FIGnition is going to be 128b rather than the 64b I had originally designed. This means it'll be a stand-alone one which works by holding SW1 and then pressing SW3 on boot-up, and this will cause a firmware image to be downloaded from external Flash (which must reside between 8Kb and 32Kb on the chip) via SPI at 1MHz. That way, having a suitable image on Flash will always mean you can reprogram the firmware. The 1MHz speed means that it shouldn't be too hard to recover a bricked FIGnition using, say an arduino.

The upgrade routines will download new firmware to external flash. I'm now in a position where I know I have a viable 16K firmware image: I need to write the new mini-bootloader and after that, the audio-based upgrader.

I have... about 15b free below the bootloader and another 30b free in the bootloader section. Wow, that'll be another 2 Forth commands at least ;-)

-cheers from Julz

[Julian Skidmore]

Hi folks,

Monday's update: A 16Kb FIGnition (with 1.0.0 firmware) can now upload a new firmware image to external Flash and verify it!

This means that I only have to be able to debug my External Flash bootloader to be able to complete the upgrade process. I think it may be the case that I can leave 32Kb FIGnitions with the USB bootloader in place, which will mean that 32Kb FIGnitions would still upgrade via USB (all the audio code will still work though) and 16Kb FIGnitions would use the audio upgrader.

Then the last-but-one final job is to create the Version 1.0.0 migration custom image :-)

Finally, I'll write the documentation and release the firmware (this will be a Candidate version of the firmware).

-cheers from Julz

[Julian Skidmore]

Hi folks,

In some late news: I've been able to make my microbootloader work, so at this point I have a FIGnition 16Kb image that can be upgraded properly via audio :-)

-cheers from Julz

[Julian Skidmore]

Hi folks,

Early morning update: I've managed to demonstrate my (nearly) candidate FIGnition 1.0.0 ROM working on NTSC in both 16Kb and 32Kb AVRs :-)

I'm now on to implementing the 1.0.0 Migration program.

-cheers from Julz

[Julian Skidmore]

Hi folks,

Probably the last update for the day - the migration custom firmware has now been written and compiles, but I haven't started debugging it yet.

-cheers from Julz

[Carl Attrill]

This is great news Julz,

Just a quick question about the lead, I have many 3.5mm jack stereo to dual RCA leads laying about ( I get them left over from customer installs for CCTV & other uses)  ; Did you say that the bug in loading was down to Stereo or was that eliminated, if so I would use this lead with one RCA plug. 

If it were a problem I guess I would have to obtain or make a 3.5mm mono to RCA, no great shakes as they must be cheaper to buy than to make living so near to a Maplin or Rapid electronics ;-)

Looking forward to you candidate.

Carl.

[Julian Skidmore]

Hi Carl,

The big appeared to be restricted to my Mac mini. If I plugged a 3.5mm mono jack into it, then there would be glitches on the audio out. This didn't happen with my G5 iMac and it didn't happen if I plugged a stereo jack in.

So I don't know if that kind of issue is likely on other platforms. The most likely issue is with the java program in that java isn't brilliant at real-time audio. I have tried to minimize it by precalculating all the samples (sort-of) so that no memory allocation of any kind happens while it's playing, but I won't be able to take into account anything else people have running in the background nor their java implementations. So I expect a bit of community debugging there on release.

Hope this helps, exciting time ahead :-)

Cheers Julz

[Mark Wills]

Yeah Yeah.... What everyone wants to know is: have you tried it on a cassette deck yet? ;-)

Mark

[Julian Skidmore]

Hi Mark,

I'm not sure I even have a cassette deck in the house, but I'll have a look ;-)

It'd be exciting & authentic though :-)

The ear and mic commands take a 'div' parameter, which is normally 6. It governs the prescalar (bits 2..4) and '1' period shift ( bits 0..1), so by changing it we can slow the bit rate enough to make it work with tape (though in current builds mic just drops it and uses 6 anyway :-S ). I may fix that by release.

I haven't checked any speed apart from 6.

Cheers Julz

[Mark Wills]

Of course we need to see a YouTube video,  otherwise it never happened!  :-)

[Julian Skidmore]

Hi Mark,

Of course we need to see a YouTube video,  otherwise it never happened!  :-)

Ha! Yes indeed! If someone downloads from tape in a wood and no-one hears it, did it really happen :-) ?

-cheers from Julz

[Carl Attrill]

I have some tapes, and a tape deck.

and 32637 bytes free.

I am Ready...

:-)

[Julian Skidmore]

Hi folks,

Well this is awkward.

I've managed to get quite a long way into supporting the migration firmware: the Java program sends the correct migration audio; the firmware can read it from tape, write it to flash and finally check the page CRCs. All this works correctly.

But I can't complete the final stage: rewriting the boot vector and copying the external Flash bootloader across. I thought it might be a subtle programming error, such as wrong "Boot-lock-bits", but they're OK and my code is exactly what I thought it should be (it's a copy of the bootloader's firmware flash writing code). And then I came across this in the ATMega328 manual on page 280:

27.3.1 Application Section:

The Application section is the section of the Flash that is used for storing the application code. The protection level for the Application section can be selected by the application Boot Lock bits (Boot Lock bits 0), see Table 27-2 on page 284. The Application section can never store any Boot Loader code since the SPM instruction is disabled when executed from the Application section.

And of course the migration program is running from the Application Section.

Show-stopper?

Well, not quite. Of course your FIGnition already has a USB Bootloader containing SPM instructions that do work (because you've used it to upload new FIGnition firmware). So, perhaps I could use the SPM routines there?

But it turns out that's not easy by any means. Firstly, the V-USB bootloader I'm using turns out not to have a neat little SPM routine I can call from 'C' - instead there's just a bunch of cases where SPM is used in a long routine that's called usbFunctionWrite in the source code, but has instead been inlined into main in the object code. That means I can't even just do a subroutine call to usbFunctionWrite, because it only returns at the end of the USB bootloader's main.

The joys of modern development and clever compilers - it just gets better!

So, I can't execute spm from the migrator code and I can't call a routine that uses spm in the bootloader code either and I can't modify the bootloader code itself for the same reasons. It looks like I'm stuffed! And of course, not everyone's FIGnition might have the same bootloader code (although they will still all have spm instructions).

Except I don't think so. I know it looks like it'll take a miracle, but I think I can work out a way to get it to work (though I'm not 100% sure). How do you think I'll do it? Answers on the forum please ;-)

-cheers from Julz

[Rob Fielding]

I'm guessing but I'll play.

'i'm thinking: if youre hinting you can exec code from the current bootloader, but the spm code is buried in an unsuitable function (for usb write),  then is the solution to prepare a new firmware with the function call in a more plain form. Maybe you need to bury the code offset by a fixed memory location or find it via a magic number at the beginning or end of the memory region?

Could another way be a firstboot routine to detect a pending state and finalise the update, then reboot.

--

You received this message because you are subscribed to the Google Groups "FIGnition" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fignition+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Robert Fielding

[Julian Skidmore]

Well, looking for the spm sequence is an obvious first step. And that should be easy, because there's very few choices when executing spm because it has to be done within 4 cycles of writing to the SPMCSR register. In assembler you'd do:

out SPMCSR,regContainingEraseOrWriteOrLoadOperation

spm

(which is how I do it in my bootloader) or

sts SPMCSR, regContainingEraseOrWriteOrLoadOperation

spm

(which is the case actually generated by the compiler). So, there's only a couple of choices (ignoring the actual register used in the out/sts command). The Migrator firmware already contains code for writing to internal flash, but of course I can't use it, because it's in the application section. But if I call the code in the bootloader where the spm instruction is, it'll just crash because it'll just run into the rest of the USB bootloader after executing the spm.

And the key thing is that I need to overwrite the existing bootloader with my own, since on the 16Kb FIGnition Firmware, the USB Bootloader is replaced by a Floating point library;  the audio loading code and a rudimentary SPI bootloader (in 128b). But the USB bootloader won't allow code to be loaded over itself*, so I must provide firmware which can overwrite the bootloader firmware, from the Application section. And spm will only work from the existing bootloader, but calling that code from the bootloader won't return properly.

A conundrum!

-cheers from Julz

* Or rather loading code via USB over the bootloader will cause it to brick the FIGnition by overwriting the jump vectors (if placed at the beginning) or preventing it from returning to the application section (if placed at the end).

[Julian Skidmore]

I think we're back in business ;-)

-cheers from Julz

[Julian Skidmore]

Hi guys,

Wow!!!!! I've done it!!!!

It's been done!!!!

The impossible!!! It's been done!!!!

This is incredible :-)

Let's recap:

In the AVR manual it says:

27.3.1 Application Section:

The Application section is the section of the Flash that is used for storing the application code. The protection level for the Application section can be selected by the application Boot Lock bits (Boot Lock bits 0), see Table 27-2 on page 284. TheApplication section can never store any Boot Loader code since the SPM instruction is disabled when executed from the Application section.

I should tell Atmel to delete that section ;-)

OK. A couple of details: Basically I've managed to complete the last section of the bootloader transfer, where (part of) the Bootloader itself gets reprogrammed with the microbootloader, using spm commands in the bootloader itself :-) Then at the end of it, the migration program transfers control to the new microbootloader, which reads the new firmware from external Flash and then reboots FIGnition.

That's been done :-) !!!

However, the current code is full of debug messages so I need to eliminate those; tidy up the Migrator and then the development will all be done. I might do that this evening, or tomorrow. I'll certainly write up a blog about it as it's

!!!!SO MONUMENTAL!!!!

But in any case: crisis over :-)

-cheers from Julz

[Stuart Taylor]

Wow, first up congrats!! That is some achievement!

second,, whoa, like, ok, i'm gonna need way more detail. Because i know what your saying, but i dont know how, it does not compute.

I had everything crossed, but I thought it was going to be a no go.

My head is spinning.

stuart

[Julian Skidmore]

Hi folks,

Well, this is good news. I've now completed the migration program correctly :-) As I wrote yesterday, I managed to get it to reprogram, but I would need to remove debug.

That sounds easy, but it turns out (as normal) that there are side-effects, subtle bugs that were in that final stage of copying over the bootloader that took a while to debug; especially as removing the debug means you're programming blind! (Well, not quite, there's always the all-purpose LED :-) ). Anyway, you can read about that in the blog.

So, the development for V1.0.0 of FIGnition is now complete and it does all I wanted it to do. The only remaining task is to complete the documentation, release notes and package it up into a V1.0.0 candidate release. The development time for that should be more predictable; I imagine that it'll only take a couple of days.

-cheers from Julz

[Benjamin Brown]

Looking forward to the new build - I want to display Gameboy camera photos (just because I can and it's similar resolution) and separately may have a cut down port of this little app - I'm hoping loading to/from tape is going to be possible :) 

Benjamin