Announcement: FIGnition Firmware 0.9.7!

[Julian Skidmore]

Hi folks,

It's been a LONG time, but I'm finally happy to release FIGnition
0.9.7! It's a rewrite of the editor and involves changes to the
compiler as well as a few other niceties. I've done the regression
testing I thought I needed to and I've included the NTSC version here
too!

You should expect to see the new code properly up on the google site
and github in the next few days. You'll like all of this!

The New Editor
******************

I've long-promised a ragged line editor; it's finally here and it's
good :-) The new editor is written entirely in Forth, it's clever and
as nippy as you could wish for with the core editor a mere 923 bytes
long - amazing! Here's a few more details:

The new editor is far more like a standard editor, typing always
inserts characters and line lengths are variable, not fixed at 25
characters. Text will also scroll if it exceeds the editing region.
Mark and Copy is supported, but the mark cursor is no longer
displayed. The normal way to quit the editor is with the sequence:
Shift then Enter (i.e. individual keys in sequence, not by pressing
them together).

The Block Editor
*******************

The core editor is used in the flash block editor to edit a block at a
time. Because it's no longer constrained by screen dimensions, we can
now edit up to 511 characters (the last character must be '\0'). The
block editor supports commands as before and thankfully they're mostly
lower-case now: 'w' writes the block back and quits; 'z' clears the
text for the current block; 'E' (with caps) erases the flash memory
and only after the confirmation key 'Y'; Cmd, digit 0..7 allows you to
enter trigraph characters, by modifiying the character under the
cursor, multiplying its character code by 8 and adding the digit. This
also means that any character code can be entered by generating a code
in the range, e.g. 64 to 95 followed by a Cmd, digit. However, you
mustn't generate a '\0' as it'll interpret it as an end of text!!!

The block editor has a nicer status line, showing the currently edited
page in decimal(!) and the character count, which updates if you press
Cmd, Enter.

The Command Line Editor
******************************

This is the most useful new change! From firmware version 0.9.7 the
command line is now at the bottom of the screen, a bit like a Jupiter
Ace / Sinclair computer. Actually, the command line is a proper
editing window, which just uses the bottom 4 lines. It means you can
now enter multiple lines of text in it and edit your command line with
full cursor control, insert and delete, <cr> characters and use mark /
copy. This will make FIGnition much better for beginners and school
kids because it means that they're far less likely to mess up the text
before they're ready to run it.

Like editing in general you now have to type Shift, Enter to exit,
which in this case, executes a command. When you do, the results
appear on the top half of the window, again a bit like a Sinclair
computer. You can't edit what's there though, it's output only. The
screen handling is fairly neat: output can occupy the entire screen as
easily as before; it's only when the system reenters the command
prompt that it'll scroll text if output had gone into the command line
area. It's really cool, but most importantly, makes FIGnition much
easier to use!

Compiler Changes
*********************

I plan to do some more compiler changes in the near future, but at the
moment I've just done the minimal amount needed for the editor. The
big change is that enclose now treats all characters from '\1' to ' '
as whitespace which means that it doesn't get confused by the <cr>s in
edited blocks or command lines.

Because we use the new editor, we no longer need the rather rubbish
'expect' command to input text, which leads nicely onto the next great
change:

Text Handling
****************

To my mind standard Forth is pretty lousy for interactive text
handling. Expect isn't what I'd expect ;-) and it's jolly awkward to
support string copying, slicing and concatenating with
[length][text...] type strings. For FIGnition Forth I'm going to be
moving towards 'C' style '\0' terminated strings. They're a bit slower
to process, but much easier to make use of.

The editor itself uses '\0' terminated strings. I've added "len ( str
-- len) which takes as input a zero-terminated string and returns its
length. Thus:

str1 str2 over "len cmove will copy a string, as a command it's:

: "! ( str1 str2 -- ) over "len cmove ;

str "len + returns the end of a string, thus:

: "+ ( str1 str2 -- ) "len + "! ;

Concatenates strings. Other string functions are similarly simple,
truncating a string to n characters is:

: "cut ( str1 n --) + 0 swap c! ;

Note, this will work even if the original string was shorter than n chars.

A string starting at the nth character is: simply str n + (which is
also a string). But since that won't work if n > the length of the
string it's better to do:

: "from ( str1 n -- str ) over "len min + ;

You can (with a bit of thought combine them):

str 5 "from 3 "cut str "! (which will extract the 5th to 8th
characters of str and copy it back to the string).

Displaying a zero-terminated string can be done with:

: ". ( str --) "len type ;

This is OK, but until now it was hard to actually input strings from
FIGnition Forth, thus with games it was hard to have high scores and
do other kinds of input, for example generating forms that involved
text entry. Now it's EASY!!

Firstly, we can input strings directly into the terminal input buffer
using query, since query now just uses the new editor. For example:

query tib 1+ "len .

Allows you to input some text and when you've finished (using Shift,
Enter) it'll return the length.

Query is simple, you can easily use it to input text and copy it to
your own strings:

0 var myStr 80 allot
query tib 1+ myStr "!

(assuming you've defined "!). Of course, you can treat the tib as a
temp 79 char string pretty much as you like too from within your own
code.

Although query is simple, it's a bit restrictive; text input is always
at the bottom of the screen and is always copied to the 79 character
tib. So, boxed has been defined which allows you to define a text edit
box anywhere on the screen and input text into any string of your own:

0 var str2 20 allot
0 str2 ! ( init)
5 5 at ( set the origin for the text box)
str2 18 8 2 boxed ( buff maxLen width height --)

The initial 0 str2 ! is used to initialize the text and the text
proper starts at str2 1+ . If you don't initialize the text it'll use
the previous text already in the string; thus you can use it to modify
a string with user input as well as input new strings (which is better
than BASIC generally allowed).

One caveat: edited text always starts at str 1+ , that's because the
text editor uses '\0' to terminate strings at both ends - the
paragraph searching algorithm expects this.

I'm really pleased with this, although most FIGnition programs are
simple games, this opens up a whole new set of programs that can be
feasibly written (and enhances compliance with the new GCSE computer
studies).

FIGnition Forth now also supports a primitive string searching
function cIn" ( dir buff ch -- buff' ) which searches for ch in the
direction dir (-1 or +1) and returns the address where it was found,
or where '\0' was found. "len is defined in terms of cIn", it's : "len
1 swap 0 cIn" ;

Finally, edit, query and boxed use ed as the core editor - thus it's
possible to use ed to create your own editor with command extensions
and features, though it's a bit more involved. I'll explain this in a
later post.

Minor Changes
*****************

There are a couple of minor, but handy changes you need to know about.

List has been removed, you'll need to use edit instead. Edit used to
show the physical page in Flash where the edited block will go, it was
helpful to me for debugging purposes. You can find out this
information by executing block blk> . if you ever have a need for it.

Vram has been moved to the end of internal RAM and future firmware
updates will always make sure it stays there. In some recent firmware
updates that wasn't the case which made FIGnition particularly
vulnerable to stack underflow, one extra . or drop and the machine
would usually crash!

Load of course works with the new ragged-line block format, but it'll
also work with the old one, old style of blocks can be still be edited
and since the line lengths were 25 chars they look pretty readable
(except that there's wasted space there).

New: The FIGnition ROM version can be displayed using 32768 1+ kern
.hex ( which means return the little-endian value of the first entry
in the kern vector list ). It displays 0x97.



FFC: The FIGnition Forth Compiler
*****************************************

The FIGnition Forth Compiler is pretty robust these days, it means the
entire Forth ROM part of the firmware can now be written in Forth,
with kludges to handle interfacing to Assembler / C. I've included the
entire editor as it currently stands in this post, so you can check
out what a practical piece of firmware code actually looks like.

FFC handles language changes that will be part of future updates to
FIGnition's compiler, in particular, it supports go> .. >while ..
until and local variables (using the locs, l>, >l and loc;) which are
FIGnition opcodes, but not generally available to the compiler). case
... of ... endof ... endcase is support too.

In addition you can see a whole host of other features, (:) allows me
to type in headerless definitions (which allow me to structure the
code without wasting space on unwanted dictionary headers); (;)
allows me to exit a definition without inserting a return opcode; #asm
and #forth allow me to switch between assembler and Forth (I can also
create native AVR code definitions this way too); #lit and #litc allow
me to insert literals whose symbols are known to the assembler and
linker, but not to the Forth system (an FFC kludge, because it doesn't
properly handle gcc symbol tables); #compile compiles words directly
into firmware rather than inserting a call to 'compile' or treating
the word differently according to what would be the current interpret
mode (because FFC doesn't really have an interpret mode).

#inline allows me to insert lines ready for the assembler's preprocessor stage.

'chr' generates litc's for characters (syntactic sugar). #deflag
provides for conditional compilation - I can create debugging code
which can be easily stripped out for release firmware.

All of this is a consequence of me wanting to be able to generate
practical firmware code in Forth for FIGnition - it's designed to
actually work, which it does - I've used it to produce the whole of
the Forth ROM as efficiently as before. There are a number of
compromises due to wanting to keep FFC fairly small etc.

Conclusion
************

It's often more involved than you think to get new firmware revisions
out. In this case, firmware revision 0.9.7 took quite a lot of
background work so that I could produce a development environment that
was good enough to produce substantial amounts of Forth ROM code in
Forth rather than assembler-ized Forth. The upshot though is that I'll
be able to improve the Forth ROM more easily in the future and that
thanks to the FFC compiler I'm now able to generate ROM images (and
multiple ROM images) automatically, which means that NTSC and PAL
versions can be updated with a single run of:

make FIGVER=0xVVvr Distro (where VV is the major version, v, the minor
version and r the revision, in this case 0097).

For users, Rev 0.9.7 is a usability improvement centred around an
editor which can be used for source code editing; command lines and
text IO. It's more suited to programming; being used by children and
because text IO is now much easier, it makes a new class of programs
feasible on FIGnition.

Enjoy!

-cheers from julz

--

The DIY 8-bit computer from nichemachines™

[fig8r]

I am very excited about the new firmware. All of the new features sound very useful. I uploaded it today, I wanted to share some first impressions about

The Command Line Editor

 - Now the command line input area is at the bottom of the screen and execution discards the input and only outputs the result to the session log (which is top of the screen at the beginning).

What has been very attractive about FIGnition is the ability to enter programs on the device, which created the same experience as the home computers of the past. FORTH modularity allows building large complex programs incrementally one word at a time. Working out FORTH constructs can be facilitated by interactive experimenting in the session, where the inputs produce certain results, and we arrive at a sought construct interactively.

The previous firmware provided continuous session log where input and the result are visible for a number of recent commands. Even if that was different from ZX or Ace, it could be considered as FIGnition advantage. In the new firmware, there is no way to tell which input produced which result, so the interactive and iterative aspects of computing are missing.

 - Shift-Enter vs Enter. From experience with shells (even the younger generation in the form of texting), we expect each entry to be executed with the Enter key. It is quite hard to recondition oneself to use Shift+Enter for shell execution; you end up hitting Enter all the time.

At the same time, in a text editor, we expect Enter key to make new lines. And it's perfectly fine to have a related command (Shift+Enter) for Save and Quit. It is very nice and valuable to have multi-line command text editing, however to some, the above interactive aspect may still be of more importance.

A possible alternative is to swap the meaning of Enter and Shift+Enter in the command line. For quick and sure commands, it will act as before executing with Enter, and when a multi-line definition is needed (less often too), Shift+Enter will create new lines.

- Placement at the bottom: this is related to preserving the input when executing. It may exceed the complexity, but if were possible to keep the command line at it's natural position below the last result and extend up to 4 lines as needed for multi-line editing, it would probably be the best combination of interactivity and multi-line editing.

- Possible new feature: Line recall. After execution, the old line may still be stored somewhere in memory. If Up is hit before any other character, the old entry is reincarnated as the current entry.
 

On Monday, September 3, 2012 11:56:23 AM UTC-4, Julz wrote:

Hi folks,
...

The Command Line Editor
******************************

This is the most useful new change! From firmware version 0.9.7 the
command line is now at the bottom of the screen, a bit like a Jupiter
Ace / Sinclair computer. Actually, the command line is a proper
editing window, which just uses the bottom 4 lines. It means you can
now enter multiple lines of text in it and edit your command line with
full cursor control, insert and delete, <cr> characters and use mark /
copy. This will make FIGnition much better for beginners and school
kids because it means that they're far less likely to mess up the text
before they're ready to run it.

Like editing in general you now have to type Shift, Enter to exit,
which in this case, executes a command. When you do, the results
appear on the top half of the window, again a bit like a Sinclair
computer. You can't edit what's there though, it's output only. The
screen handling is fairly neat: output can occupy the entire screen as
easily as before; it's only when the system reenters the command
prompt that it'll scroll text if output had gone into the command line
area. It's really cool, but most importantly, makes FIGnition much
easier to use!

...

[Julian Skidmore]

Hi Oleg,

Thanks for the feedback,

> I am very excited about the new firmware.

Thanks Oleg!

> - Now the command line input area is at the bottom of the screen and
> execution discards the input and only outputs the result to the session log
> (which is top of the screen at the beginning).

Your solution to the command-line logging I think sounds good and is
possible, so I'll probably try to introduce it after I've done the
blitter (as it will add a little to the size of the editor, so I need
to think about how to fit it in). The update will use the 4 lines
after the current text position for the text edit area. That approach
will also fix the inconsistency with scrolling - for example, vlist
uses more>> to paginate the dictionary, but if the last page is more
than 20 lines long, definitions will be lost when the screen scrolls
to make way for the new command editor.

> - Shift-Enter vs Enter. From experience with shells (even the younger
> generation in the form of texting), we expect each entry to be executed with
> the Enter key. It is quite hard to recondition oneself to use Shift+Enter
> for shell execution; you end up hitting Enter all the time.

Admittedly, I sometimes still type Enter instead of Shift, Enter too
:-) However, I think I'll keep it for the moment for two reasons.
Firstly, it's safer: if we press enter, expecting it to execute a
command and it just inserts a <cr> then we'll simply think 'D'oh!' and
type Shift, Enter and there's no harm done. But if it was the other
way round, we'd be bound to accidentally enter half-finished commands
(simply because sometimes we'd mistype or neglect the Shift).

Eventually we'll condition ourselves - and this will be less of a
problem for those new to FIGnition than us. The shift-enter vs enter
inconsistency isn't just a problem for FIGnition, note ZX computers
and the Ace fixed it by not allowing multiple 'paragraph' commands, so
long lines are quite clumsy to edit; and the problem propagates to
modern systems too, e.g. Facebook statuses expect Shift+Enter to
insert paragraphs (Enter enters the status) and yet messages use Enter
for <cr>.

Secondly, I think it's a better UI principle to be consistent between
the block editor and command editor. Also, it's the rest of the world
that's wrong, so we need to run a campaign to get them to swap over,
Shift+Enter for execute, Enter for <cr> wherever that makes sense ;-)
It's in their best interest long-term - they'll thank us for it :-D

> - Possible new feature: Line recall. After execution, the old line may still
> be stored somewhere in memory. If Up is hit before any other character, the
> old entry is reincarnated as the current entry.

Line recall would be surprisingly easy. When the editor is entered,
the initial text had been present, but is deleted by storing 0 in the
first word of the buffer. Preserving the original characters and being
able to restore them after a suitable initial key sequence (though I
might use Command, Up rather than Up) is fairly trivial.

-cheers from julz


--

[Julian Skidmore]

Hi folks,

I thought I'd be able to fix the associated tools for 0.9.7 and
release in short notice, but getting the new version of T2E2 has taken
longer than expected. However, I think it's finally there. The source
code for the new T2E2 is attached and is written in 'C'; you'll need
to compile it for your computer. I'm thinking, perhaps it should be
converted to Java and then I'll be able to distribute cross-platform
binaries. Hey ho.

This version of T2E2 has the following changes:

* The default mode converts files to ragged-line blocks for the new
FIGnition firmware, you'll need to add a -p option to make it create
padded blocks.
* The padded block conversion uses Oleg's enhancements to handle
end-of-line comments that are removed during the conversion process.
* The ragged-line version makes sure that word definitions aren't
split across blocks.
* You don't need to split programs into 25 line chunks any more and
you don't need to confine the right margin to 25 characters (however,
having a 25 character margin will still help because it'll make it
look nicer on the FIGnition screen).
* The ragged-line version handles special end-of-line comments like
Oleg's if they're separated from the previous text with a tab
character.
* Blocks up to 511 characters are now generated.

This version of T2E2 has one major quirk, multi-line comments have
<cr> replaced with <space> because there's a bug in the 0.9.7 version
of enclose which treats all characters <32 as being the end of a
token. I plan to fix this by 0.9.8.

I've checked the new version of T2E2 on the Brikky game.

Now that T2E2 is ready I can check it all into GitHub, which I hope to do today.

-cheers from julz