Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Forth webserver ported to VFX & ciforth

427 views
Skip to first unread message

Julian Fondren

unread,
Dec 13, 2016, 12:13:39 AM12/13/16
to
Hello clf,

The forth webserver I talked about before now runs on VFX and ciforth
(lina) as well. lina only tested in its 32-bit configuration.

From the following ApacheBench numbers you can conclude... that
SwiftForth's FILE-SIZE is quite slow (it seeks to the end of the file
instead of statting it). Otherwise I'm not sure what to say about
performance, except that lina's more competitive than I expected for
this workload. These numbers come from a Dell XPS 12 with CPUs in
'performance' mode as it wasn't running on battery.

The 'hello' columns are served by hello-website, which just responds
to any HTTP request with 'hello'. The 'index' columns are served by
simple-website, and are responding to the front-page query with the
contents of a brief index.html. There's also a game-website but it
just mixes 'hello' performance with 'simple' performance, depending on
the query.

ab -c 1 -n 10000 -r http://127.0.0.1:4000/

Requests per second:
-c 1 -c 2 -c 10
hello index | hello index | hello index
sf 10312.51 6139.10 | 14027.51 13474.13 | 15286.13 15819.26
lina 15493.14 8971.72 | 13985.37 15104.36 | 14934.04 14921.36
vfx 9756.34 9052.46 | 13856.13 16168.67 | 13198.67 13526.11

In the process of expanding the number of Forths that can run the
webserver, I've broken (temporarily) the epoll engine. The threaded
engine's still broken. Although some of the changes would seem to
have made things worse for multithreading, I don't think it's that bad
off. It just needs attention.

epoll was working fine with VFX for a while. In the previous commit
actually.

As part of this process I removed SWOOP from everything but some
SF-only code, and replaced it with packages (for namespacing) and
chibi tables (for data structures). SWOOP was already less pleasant
for VFX, which didn't have [OBJECTS ... OBJECTS] locals syntax.
The time I put into trying to get even PortableSWOOP to run under
ciforth was pretty wasted.

For the most part removing SWOOP improved the code. Not to fault
SWOOP. That's how it worked out.

Routing, initially:

routing begin:

route :: index-rule
if: client request request-uri s" /" compare 0=
if s" /index.html" client request 'request-uri 2! then false ;

route :: google-rule
if: client request ua s" Googlebot" search nip nip ;
do: client kill ; \ close connection without logging

simple :: time-rule
if: m" /time" ;
do: client created @ .now ;
...
routing end;

But that request stuff used CLIENT as an injected local object
variable. I couldn't get that working with PortableSwoop, so this code
briefly became:

routing begin:

route :: index-rule
:noname [ over -> match-xt ! ] >r
r@ USING client request request-uri s" /" compare 0=
if s" /index.html" r@ USING client request 'request-uri 2! then false r> drop ;

route :: google-rule
:noname [ over -> match-xt ! ]
USING client request ua s" Googlebot" search nip nip ;
:noname [ over -> fulfill-xt ! ]
USING client kill ; \ close connection without logging

simple :: time-rule
:noname [ over -> match-xt ! ] drop m" /time" ;
:noname [ over -> fulfill-xt ! ] USING client created @ .now ;
...
routing end;

Not *quite* as pretty.

It now looks like this:

routing
rewrite / /index.html

:noname request. ua s" Googlebot" search nip nip ;
:noname client. kill ;

when /time
:noname [OK client. created @ .now OK] ;
...
end-routing

Before I had 'route' objects with match and fulfill methods, and
subclasses of them, and they were put into a linked list with its own
methods. Now I just have normal definitions and helpers that create
definitions, and it all gets compiled into a single definition:

: routing ( -- )
depth route-depth !
routes >order ;

PRIVATE

: end-routing ( xt xt ... xt -- )
depth route-depth @ - n>r
:NONAME
r> begin dup while 1-
r> COMPILE,
dup if 1-
r@ ['] noop = if r> drop else
POSTPONE if r> COMPILE, POSTPONE exit POSTPONE then
then
then
repeat drop
POSTPONE ; is route previous ;

...

So this:

routing
' A
' B

' C
' NOOP

' D
' E

' F
end-routing

Gets compiled into this:

:NONAME A IF B EXIT THEN C D IF E EXIT THEN F ;


Meanwhile a lot of former objects have former methods that are
actually completely unchanged (this was a lot of the point of chibi
tables' design), and former OBJ METHOD invocations now mostly look
like PACKAGE. WORD

So this:

: connected ( -- )
request clear reading on now created ! fd on written off
status off peer off buffer off
socket @ socket. fd peername if drop peer ! then ;

Just became this:

: connected ( -- )
request. clear reading on now created ! fd on written off
status off peer off buffer off
socket. fd peername if drop peer ! then ;


On porting, getting it run with VFX was very easy. I just had to
remove some advanced SWOOP stuff, mostly. VFX does seem to completely
silently fail in cases where SF has verbose and nice error messages,
OTOH debugging VFX is much smoother when it drops you right into an
editor on the line with the error. But I had some nice-with-SF
constructs that were wasted on VFX:

( at the beginning of poll/pollserver.fs )
[DEFINED] clients 0= [IF] abort
package clients
package-peeker client.
max-clients 1+ table client
end-table
: connected ( -- ) pollfd now has a live socket ;
: reap ( -- ) clean up, log death, etc. pollfd will be killed afterwards ;
: update ( -- ) you have input! (or may now write). deal with it ;
end-package
[THEN]

If you tried to load this with SF it'll fail and show you the context
of the failure -- oh, right, this code assumes a clients package and
table.

Porting the webserver to ciforth required:
1. giving up on an advanced OOP system altogether
2. reimplementing core words like GET-ORDER and S\"
3. making mistakes in #2 which were extremely confusing because I'd
made them a long time ago and didn't think of core words as 'in scope'
for errors.
4. asking C for a lot of extra syscall numbers and some data offstes
5. dealing with C *lying* about #4. I have no idea why my ->st_size
calculation is wrong, I just know (experimentally) that it should be
some other number that I've hardcoded.
6. Adding "don't do this on ciforth" conditional code:

$ grep -P '(?<!: )\[CI\]' *.fs */*.fs|wc -l
11

But it was still something I only worked on for two days, not counting
stuff like the package+chibi replacement plan.

If there's a gforth release that has a C interface that doesn't demand
a lot from the system (like a C compiler) or a syscall interface
(something like ciforth's XOS and XOS5 primitives is all that's
*really* needed), then it'll of course work easily, too.

I have some iForth code but it needs some extra build system work if
I'm not to say "Step 1. provide the following words somehow", so I've
put it aside for now.

That is all. This is all still a work in progress.

The code can be found at https://bitbucket.org/demonview/forth-httpd/

I am using fossil for development and am just pushing with git, so
don't expect much of interest in git metadata.


-- Julian

Albert van der Horst

unread,
Dec 13, 2016, 1:05:41 PM12/13/16
to
In article <b77004c0-2fe1-4116...@googlegroups.com>,
Julian Fondren <julian....@gmail.com> wrote:
>Hello clf,
>
>The forth webserver I talked about before now runs on VFX and ciforth
>(lina) as well. lina only tested in its 32-bit configuration.
<SNIP>

>Porting the webserver to ciforth required:
>1. giving up on an advanced OOP system altogether
>2. reimplementing core words like GET-ORDER and S\"

I'm frugal but I'll add those to the library.

>3. making mistakes in #2 which were extremely confusing because I'd
>made them a long time ago and didn't think of core words as 'in scope'
>for errors.
>4. asking C for a lot of extra syscall numbers and some data offstes

I'm a bit of a stickup with this. I think that essential syscall
numbers of a Linux system and crucial data structures must be documented
without ressorting to c, even though it is obviously impossible.
So one should be able to routinely add a system call.
I could add all syscall numbers whithout giving a clue about how
to use them, that I don't want.
So I'll use your code as a hint to what should be added to make
ciforth more practical.


>5. dealing with C *lying* about #4. I have no idea why my ->st_size
>calculation is wrong, I just know (experimentally) that it should be
>some other number that I've hardcoded.

Exactly the reason why I hate this situation.

>6. Adding "don't do this on ciforth" conditional code:
>
> $ grep -P '(?<!: )\[CI\]' *.fs */*.fs|wc -l
> 11

I've no clue what this means.

>
>But it was still something I only worked on for two days, not counting
>stuff like the package+chibi replacement plan.
>
>If there's a gforth release that has a C interface that doesn't demand
>a lot from the system (like a C compiler) or a syscall interface
>(something like ciforth's XOS and XOS5 primitives is all that's
>*really* needed), then it'll of course work easily, too.
>
>I have some iForth code but it needs some extra build system work if
>I'm not to say "Step 1. provide the following words somehow", so I've
>put it aside for now.
>
>That is all. This is all still a work in progress.
>
>The code can be found at https://bitbucket.org/demonview/forth-httpd/

I'll certainly look into it for things to add to the library.

Is there original code to look at to see why my object system
didn't measure up?


>
>I am using fossil for development and am just pushing with git, so
>don't expect much of interest in git metadata.
>
>
>-- Julian

Groetjes Albert
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

Julian Fondren

unread,
Dec 13, 2016, 2:55:29 PM12/13/16
to
On Tuesday, December 13, 2016 at 12:05:41 PM UTC-6, Albert van der Horst wrote:
> In article <b77004c0-2fe1-4116...@googlegroups.com>,
> Julian Fondren <julian....@gmail.com> wrote:
> >
> >6. Adding "don't do this on ciforth" conditional code:
> >
> > $ grep -P '(?<!: )\[CI\]' *.fs */*.fs|wc -l
> > 11
> >
> I've no clue what this means.

That command finds eleven instances of this construct:

[CI] do in case of ciforth [ELSE] otherwise [THEN]

Also: [SF] [VFX] [GF]

And then [IFORTH] due to the unfortunate abbreviation.

This doesn't include completely forked code: every Forth has its own
net.fs and buffio.fs in separate directories.

> I'll certainly look into it for things to add to the library.

The more troublesome code was here:

http://theforth.net/package/package/current-view/compat-ciforth.fs

>
> Is there original code to look at to see why my object system
> didn't measure up?
>

What do you mean by 'original code'?

I spoke a little bit about this the other day in the 'chibi tables'
thread. I liked a per-class current object which I drew from your
object system, because it completely removes object management from
most code, but I had single instances of arrays of nameless objects.

The way I went also resulted in code being almost completely
unchanged. Compare the SWOOP http-request class with what it is now:

https://bitbucket.org/demonview/forth-httpd/src/3c02dfda761b7aa91c2eb7e1e32c657853fd8a34/http-request.fs?at=master&fileviewer=file-view-default

https://bitbucket.org/demonview/forth-httpd/src/5d3d4fc12b4294eff167adad4fe96cb7bc6d164d/http-request.fs?at=master&fileviewer=file-view-default

I don't mean to suggest that I came up with an 'advanced OOP system'.
I mean that once I gave up on getting any of those to work, that I
came up with something satisfactory. By such a system I mean SWOOP,
FMS, oof.fs, objects.fs to an extent - something with all the bells
and whistles.

Also, going with your object system would've meant reimplementing it
for the other Forths. Right now I have just three definitions, and
then some (not enough) helpers to account for features like SWOOP's
default ADDR method.

: table ( n-count "name" -- n a )
create here 0 , 0 , 0 , \ current address, 0th address, record width
does> ( n-index -- )
tuck cell+ 2@ -rot * + swap ! ;

: end-table ( n-count a -- )
here over cell+ !
2 cells + @ * allot ;

: rec ( n a record-size "name" -- n a )
over 2 cells + dup @ >r +! dup >r
: r> POSTPONE literal POSTPONE @ r> POSTPONE literal POSTPONE + POSTPONE ; ;


-- Julian

Doug Hoffman

unread,
Dec 14, 2016, 5:14:38 PM12/14/16
to
On 12/13/16 2:55 PM, Julian Fondren wrote:

> Also, going with your object system would've meant reimplementing it
> for the other Forths. Right now I have just three definitions, and
> then some (not enough) helpers to account for features like SWOOP's
> default ADDR method.
>
> : table ( n-count "name" -- n a )
>
> : end-table ( n-count a -- )
>
> : rec ( n a record-size "name" -- n a )

Without knowing exactly what your program needs it appears that
speed is the highest priority and oop was always inappropriate.
Apparently you only need fixed arrays of data structures and nothing
like late binding, polymorphic messaging, or namespace protection (or
you handle namespace some other more manual way).

You could achieve a performance gain with your chibi
tables using the following:

15 constant #points
create points #points 2 cells * allot
variable cur-point
: point ( idx -- ) 2 cells * points + cur-point ! ;
: x ( -- addr) cur-point @ ;
: y ( -- addr) cur-point @ 1 cells + ;

The rest of the code is the same.

Here : run 4 point ; takes only one instruction on VFX,
vs four instructions with a chibi table.

-Doug

hughag...@gmail.com

unread,
Dec 14, 2016, 8:53:06 PM12/14/16
to
I haven't been following this thread, so I don't know what all this code does. I will say that your code snippet above is some terrible Forth code. None of Forth Inc.'s books mention the struct and I have no evidence to indicate that anybody there knows what a struct is. Also, the only array definer they provide is limited to cell-sized elements and I have no evidence to indicate that anybody there knows how to write a general-purpose array definer. Also, they use global variables routinely and I have no evidence to indicate that anybody there knows what reentrancy is.

Anyway, using the novice package, I would write the above code snippet like this:

0
w field pt.x
w field pt.y
constant pt

15 constant #points

pt 0 #points ary points

Now to obtain the X and Y values given an index, use:
pt.x @
pt.y @

I almost never define a global variable such as your CUR-POINT --- that is also bad Forth.

If I want a 3-dimensional array of pts (possibly representing a 3-dimensional universe), you would define it like this:

pt 0 #points #points #points ary universe

The advantage of using a struct such as I have above, is that you get inheritance. For example:

pt
w field cir.r
constant cir

Now you have a circle derived from a point. When you hand-write custom code as you did above, you don't get inheritance. You would have to hand-write everything, even when the data-type is obviously derived from another data-type --- that is 1970s programming --- nobody has done that in almost a half century, because it is a bad idea.

I don't really understand why everybody on comp.lang.forth hates my novice package so much. Programming such I showed above is very basic computer-science that any high-school student would know. All of the ANS-Forth programmers insist on writing code in a grossly primitive manner that hasn't been seen since the 1970s. What is the purpose of this??? I suppose that by doing this you stay on Elizabeth Rather's good side, but what is the purpose of that??? ANS-Forth is a purely negative contribution to the Forth community --- this is just stupid.

hughag...@gmail.com

unread,
Dec 14, 2016, 9:04:04 PM12/14/16
to
On Wednesday, December 14, 2016 at 6:53:06 PM UTC-7, hughag...@gmail.com wrote:
> pt 0 #points ary points
>
> Now to obtain the X and Y values given an index, use:
> pt.x @
> pt.y @

That was a typo. I meant:
points pt.x @
points pt.y @

POINTS takes an index and provides an address in the array. Then the field name PT.X takes an address of a struct and provides an address of a field in the struct.

Julian Fondren

unread,
Dec 15, 2016, 3:28:48 AM12/15/16
to
On Wednesday, December 14, 2016 at 4:14:38 PM UTC-6, Doug Hoffman wrote:
>
> Without knowing exactly what your program needs it appears that
> speed is the highest priority and oop was always inappropriate.
> Apparently you only need fixed arrays of data structures and nothing
> like late binding, polymorphic messaging, or namespace protection (or
> you handle namespace some other more manual way).

I didn't remove SWOOP because it was slow, but because it was too much
trouble getting it to run on ciforth (in part due to my GET-ORDER
screwup), and because it wasn't as comfortable to use without
extensions not in PortableSWOOP.

I used late binding and a short class hierarchy in the routing code.
Instead of pairs of XTs, an object was placed on the stack; and XTs
were assigned to its members and different classes would execute their
XTs in different contexts, for example the 'simple' route subclass
would expect its match XT to only perform some matcher.fs operations,
and would go on to execute its fullfill XT only if the matcher.fs
operations completed without error.

routes.fs used to have those subclasses:

https://bitbucket.org/demonview/forth-httpd/src/973f4cdbad4902ee15356f620cfa88e71a74a289/routes.fs?at=master&fileviewer=file-view-default

I could have gone on to use, in pollserver et al., an array of object
addresses instead of a flat array of objects for the array of clients,
which would've permitted late binding method calls to arbitrary client
subclasses.

In the end I'm happy with the OO-less code, and you could argue back
from that result, but for me this wasn't some extreme plateau of 0.01%
programming from which OO must be excused. I was happy with the
original SWOOP code as well.

>
> You could achieve a performance gain with your chibi
> tables using the following:
>

Ah yeah. I'm dynamically fetching some constants. That can be avoided.

>
> Here : run 4 point ; takes only one instruction on VFX,
> vs four instructions with a chibi table.

--and should be! Thanks for working that out. If I ever do want to
ALLOCATE the space after the fact or do something weirder, I can
change the code at that time.


-- Julian

Julian Fondren

unread,
Dec 15, 2016, 3:32:02 AM12/15/16
to
On Wednesday, December 14, 2016 at 7:53:06 PM UTC-6, hughag...@gmail.com wrote:
>
> I haven't been following this thread, so I don't know what all this code does. I will say that your code snippet above is some terrible Forth code.

"I don't even know what's going on, but it sucks."

> ... Also, they use global variables routinely and I have no evidence to indicate that anybody there knows what reentrancy is.
> ...
> I almost never define a global variable such as your CUR-POINT --- that is also bad Forth.
> ...
> The advantage of using a struct such as I have above, is that you get inheritance. For example:
>
> pt
> w field cir.r
> constant cir
>

You can code in the darkness and you can code in the light. That's not
a moral distinction, it's just two contrasting potential conditions of
the environment in which you may find yourself writing or working with
some code. All of your ideas about modern and professional
programming, which you think to be cost-free progressions of the art,
are actually just darkness coping mechanisms: they are things you can
do, assuming perpetual night, to code while reducing your chances of
getting eaten by a grue -- or bumping your head against things you
can't see, having other people trip over you, etc.

Not that it *isn't* progress to be able to deal more effectively with
the dark, and not that the costs might genuinely be cheaper than
paying for the mistakes one makes in the dark. Just that, they *do*
have a cost, and *if* you aren't actually programming right at the
bottom of abyss, *then* you might want to consider the benefits of
alternate methods.

But speaking of 'considering', this is also a thing your modern and
professional programming seeks to avoid. If you resort to thinking
about things, and looking around, and evaluating your actual
circumstances, why, you might make a mistake. And of course it's quite
a hard thing to do, so it takes time and even after you've blown all
that time not writing code you can *still* make a mistake. It's much
easier and safer to just follow a set of rules that you know will help
with the worst-case condition, to smooth out your performance over
time.

INTERMISSION

On Wednesday, December 14, 2016 at 9:47:20 PM UTC-6, hughag...@gmail.com wrote:
>
> ANS-Forth is terrible at numerical programs. I'm very dubious that anybody will want to use my program. The user needs to use F+ F- F* F/ etc. rather than + - * / etc.. The user needs to put an 'e' in every number, so we would have 1e 0.142857e etc. rather than 1 0.142857 etc. (in Straight Forth: 1 1/7 etc.). All in all, ANS-Forth code is just ugly!

create vertices
| -1 -1
| 1 -1
| 0 1

eww. omg. just LOOK at those 32-bit floating point literals right
there. Why are they ugly? Why is Forth such an ugly language? Why did
the ANS Forth committee hate beauty so much? Why couldn't it look at
least *nice* like in Java?

float[] vertices = {
-1.0f, -1.0f,
1.0f, -1.0f,
0.0f, 1.0f,
};

It's just. I can't. I just can't. I can't even. I'm literally shaking
right now.

Get a grip, Hugh. A language gives you parsing words and you're going
to whine jealously about the rest of the languages of the world, where
data must either be A) ugly, or B) not inline with the code? Your
whinging about ANS Forth just keeps stretching out like you're falling
into a black hole.

END INTERMISSION

By darkness I mean lack of knowledge about other people, about the
future, about how code that you must integrate with will perform. In
some teams it's hard to communicate knowledge even when it technically
could be available--if skill levels vary enough, or if a team is
growing fast enough, it's just too much trouble to maintain a shared
understanding. c.f. the Mythical Man Month.

A light-level decision I made recently about this project went like
this:

* The client, http-request, and socket libraries all have their own
current object. It's costly (four instructions, as noted above) to
update them. In some cases it may not be strictly necessary to update
*all* of them for a sequence of code. Should I therefore:

A. define an update-everything word and adhere to the discipline of
using it instead of the individual updaters, or

B. freely use individual updaters when I know that the code doesn't
care, and may well change current object before it looks at the other
arrays.

I went with B.


-- Julian

Albert van der Horst

unread,
Dec 15, 2016, 6:52:58 AM12/15/16
to
In article <b77004c0-2fe1-4116...@googlegroups.com>,
Julian Fondren <julian....@gmail.com> wrote:
>Hello clf,
>
>The forth webserver I talked about before now runs on VFX and ciforth
>(lina) as well. lina only tested in its 32-bit configuration.
<SNIP>
>
>That is all. This is all still a work in progress.
>
>The code can be found at https://bitbucket.org/demonview/forth-httpd/
>
>I am using fossil for development and am just pushing with git, so
>don't expect much of interest in git metadata.

You can do
"
WANT TIME&DATE ALIAS
'SSE ALIAS now

Julian Fondren

unread,
Dec 15, 2016, 10:38:25 AM12/15/16
to
On Thursday, December 15, 2016 at 5:52:58 AM UTC-6, Albert van der Horst wrote:
>
> You can do
> "
> WANT TIME&DATE ALIAS
> 'SSE ALIAS now
>
> "
>

Heh. That alias is actually already in ciforth/compat.fs

I'd copied that file from another project, then didn't look over it
carefully, then punted on time.fs when the solution was already
available.

Well what I'd like to do next is try a few different designs of
threadserver.fs -- for ciforth, first. This involves creating an array
of a hundred or so THREAD-PETs, which is not convenient with the way
that word's currently defined. I'll implement time.fs properly when
there's something to show for that.

Thanks,
Julian

hughag...@gmail.com

unread,
Dec 15, 2016, 6:15:03 PM12/15/16
to
On Wednesday, December 14, 2016 at 6:53:06 PM UTC-7, hughag...@gmail.com wrote:
> On Wednesday, December 14, 2016 at 3:14:38 PM UTC-7, Doug Hoffman wrote:
> > You could achieve a performance gain with your chibi
> > tables using the following:
> >
> > 15 constant #points
> > create points #points 2 cells * allot
> > variable cur-point
> > : point ( idx -- ) 2 cells * points + cur-point ! ;
> > : x ( -- addr) cur-point @ ;
> > : y ( -- addr) cur-point @ 1 cells + ;
> ...
> Anyway, using the novice package, I would write the above code snippet like this:
>
> 0
> w field pt.x
> w field pt.y
> constant pt
>
> 15 constant #points
>
> pt 0 #points ary points

Doug --- my code is not only more readable, but it also avoids the need for multiplication in a non-optimizing Forth compiler so it should be faster too.

Your POINT looks like this under SwiftForth:

see point
49194F 8 # EAX MOV B808000000
491954 EBX MUL F7E3
491956 EAX EBX MOV 8BD8
491958 8D864 [EDI] [EBX] EBX LEA 8D9C1F64D80800
49195F EBX 8D904 [EDI] MOV 899F04D90800
491965 0 [EBP] EBX MOV 8B5D00
491968 4 # EBP ADD 83C504
49196B RET C3 ok

Here is POINT using my system:

: point ( index -- ) points cur-point ! ;
point isn't unique. ok
see point
491B5F 491AEF ( points ) CALL E88BFFFFFF
491B64 EBX 8D904 [EDI] MOV 899F04D90800
491B6A 0 [EBP] EBX MOV 8B5D00
491B6D 4 # EBP ADD 83C504
491B70 RET C3 ok
see points
491AEF 4 # EBP SUB 83ED04
491AF2 491A58 # 0 [EBP] MOV C74500581A4900
491AF9 EBX 3 # SHL C1E303
491AFC 0 [EBP] EBX ADD 035D00
491AFF 4 # EBP ADD 83C504
491B02 RET C3 ok

Under VFX however, yours and mine both optimize out the same:

This is yours:

see point
POINT
( 004E4420 C1E303 ) SHL EBX, 03
( 004E4423 81C360434E00 ) ADD EBX, 004E4360
( 004E4429 891D84054600 ) MOV [00460584], EBX
( 004E442F 8B5D00 ) MOV EBX, [EBP]
( 004E4432 8D6D04 ) LEA EBP, [EBP+04]
( 004E4435 C3 ) NEXT,
( 22 bytes, 6 instructions )
ok

This is mine:

: point ( index -- ) points cur-point ! ;
POINT is redefined ok
see point
POINT
( 004E4690 C1E303 ) SHL EBX, 03
( 004E4693 81C364454E00 ) ADD EBX, 004E4564
( 004E4699 891D84054600 ) MOV [00460584], EBX
( 004E469F 8B5D00 ) MOV EBX, [EBP]
( 004E46A2 8D6D04 ) LEA EBP, [EBP+04]
( 004E46A5 C3 ) NEXT,
( 22 bytes, 6 instructions )
ok

A lot of the code in the novice-package was written to support SwiftForth's lack of optimization. In this case, my ARY uses LIT*, internally which compiles a multiplication by a literal and uses a shift if possible. This is needed in SwiftForth because there is no optimization being done, but it is not needed in VFX where optimization is being done. This is the LIT*, meta-compiler:

: lg ( u -- val|-1 ) \ -1 if not a power of 2
dup 0= if drop -1 exit then
0 begin \ -- u val
over 1 and 0= while 1+ >r 1 rshift r> repeat
over 1 xor if 2drop -1 exit then \ not a power of 2
nip ;

: lit*, ( mult -- ) \ runtime: n -- n*mult
dup 0 = if drop drop, 0 lit, exit then
dup 1 = if drop exit then
dup lg \ -- mult lg(mult)
dup 0< if drop lit, *,
else nip lit, lshift, then ;

I actually went to a lot of effort to make the novice-package generate reasonably efficient code under SwiftForth despite SwiftForth's lack of optimization. At the time, I expected Forth Inc. to thank me for helping them overcome their lack of optimization. Instead Elizabeth Rather insults me and says that my novice-package was "written by a novice." I think this is because helping them overcome their lack of optimization tends to shine a spotlight on their lack of optimization. Elizabeth Rather just wants Forth programmers such as Julien Fondren who brown-nose shamelessly and who are too dumb to know anything about optimization.

I still don't know what any of this code is for --- I don't read Julien Fondren's posts --- I only responded to your post because I noticed that your code was quite unreadable --- it is really not necessary to manually implement arrays in order to get efficiency.

Julian Fondren

unread,
Dec 15, 2016, 6:40:18 PM12/15/16
to
On Thursday, December 15, 2016 at 5:15:03 PM UTC-6, hughag...@gmail.com wrote:
>
> I still don't know what any of this code is for

And you're still an idiot, lecturing people who aren't about things
they already know.

If you're this triggered by hostility milder, less personal, and less
constant than your own, maybe you should try civility for a while.

Doug Hoffman

unread,
Dec 16, 2016, 5:08:59 AM12/16/16
to
On 12/15/16 3:28 AM, Julian Fondren wrote:
> On Wednesday, December 14, 2016 at 4:14:38 PM UTC-6, Doug Hoffman wrote:
>>
>> Without knowing exactly what your program needs it appears that
>> speed is the highest priority and oop was always inappropriate.


> I didn't remove SWOOP because it was slow, but because it was too much
> trouble getting it to run on ciforth (in part due to my GET-ORDER
> screwup), and because it wasn't as comfortable to use without
> extensions not in PortableSWOOP.


> In the end I'm happy with the OO-less code, and you could argue back
> from that result, but for me this wasn't some extreme plateau of 0.01%
> programming from which OO must be excused. I was happy with the
> original SWOOP code as well.
>
>>
>> You could achieve a performance gain with your chibi
>> tables using the following:

Sorry. I incorrectly assumed the performance requirement.
Given that, I prefer your REC and so forth because it is obviously more
readable/maintainable.

-Doug

Julian Fondren

unread,
Dec 16, 2016, 10:48:13 AM12/16/16
to
On Friday, December 16, 2016 at 4:08:59 AM UTC-6, Doug Hoffman wrote:
>
> Sorry. I incorrectly assumed the performance requirement.
> Given that, I prefer your REC and so forth because it is obviously more
> readable/maintainable.
>

Eh? There's no reason for it be slower, though. The same words, used
the same way, can compile constants as your code does rather than
fetch from addresses. My original code is objectively slower than
necessary because I'm fetching constants from addresses instead of
inlining them, and you pointed that out just fine.

There's no readability vs. speed tradeoff.

So this for example

100 buffer: table-name
: table ( n-count "name" -- n a )
parse-word table-name place here 0 , 0 , 0 , ;

: end-table ( n-count a -- a )
>r here r@ cell+ !
r@ 2 cells + @ * allot
table-name count (:) ( n-index -- )
r@ 2 cells + @ POSTPONE literal POSTPONE *
r@ cell+ @ POSTPONE literal POSTPONE +
r@ POSTPONE literal POSTPONE ! POSTPONE ;
r> ; \ dropped or saved for table metadata

should result in the index-updating word costing one instruction:

Although it doesn't due to a quirk of VFX.

One thing...

create points create cur-point ok
: point: : 2 cells postpone literal postpone * points postpone literal postpone +
cur-point postpone literal postpone ! postpone ; ; ok
point: pp ok
: run 4 pp ; ok
see run
RUN
( 080C0B60 C705700A0C08600A0C08 ) MOV DWord Ptr [080C0A70], 080C0A60
( 080C0B6A C3 ) NEXT,
( 11 bytes, 2 instructions )

... is not like the other:

create points create cur-point ok
: :point (:) 2 cells postpone literal postpone * points postpone literal postpone +
cur-point postpone literal postpone ! postpone ; ; ok
s" pp" :point ok
: run 4 pp ; ok
see run
RUN
( 080C0B60 8D6DFC ) LEA EBP, [EBP+-04]
( 080C0B63 895D00 ) MOV [EBP], EBX
( 080C0B66 BB04000000 ) MOV EBX, 00000004
( 080C0B6B E8B0FFFFFF ) CALL 080C0B20 PP
( 080C0B70 C3 ) NEXT,
( 17 bytes, 5 instructions )
ok

So with that between that and the annoyance of saving the name, I may
as well just write:

table
cell rec x
cell rec y
end-table points


Cheers,
Julian

Albert van der Horst

unread,
Dec 16, 2016, 11:11:19 AM12/16/16
to
In article <1144d664-4e36-4788...@googlegroups.com>,
The parallel stuff is a bit low on documentation:
- if you output in a thread, you need some dictionary space, because
of the name collecting area. 400 bytes should be sufficient.
- if you don't use formatted output nor the PAD, you can do with
zero space.
- The per task area is huge 64 kbyte. It can be diminished.
The limiting factor is the user area that has now 216 (276) byte in use.
The user area is one quarter of the per task area, as is the data
stack, TIB and return stack.
If one can live with 256 bytes of data stack, the per task area
can be diminished to 0x400 (0x800 for 64 bits).
Best start with 0x1000, save a factor 16.

TAS = 0x10000 ;Size of unitialised area per task.

then
fasm lina32.fas -m256000

The resulting 2Mbyte forth can be size up to 1 Gbyte (example) by

lina32 -g 1000 lina1G

This would accommodate 1000 processes, without making a dent in the
main space.

In the parallel prime count as a prestudy for the colorforth program
I tried a few hundred PRE-EMPTIVE threads, which went wrong.
It may be interesting to retry this.

>
>Thanks,
>Julian

Julian Fondren

unread,
Dec 23, 2016, 1:45:08 PM12/23/16
to
Incidentally, yesterday, on a computer with a stale dev environment
and a different Linux distribution, I needed to serve files out of a
directory to use an application. So I updated this project and

1. I couldn't build lina at all, because constants32.fs needed
different header paths that I didn't know how to install on this
distro. So the built lina binaries or the prebuilt constants*.fs files
are MORE PORTABLE than the build system that produces them! This is
how much of a pain it is to ask C for constants.

Well one thing I can do is just include lina binaries in the project
then. They aren't IP encumbered and they can double as the
more-portable way to get constants for another Forth's use.

2. I couldn't build the VFX binaries either. Something about the
installation environment is just slightly off. I get errors after
loading MultiLin32.fth that I didn't get before.

3. My VFX and SF simple-websites both crashed after a while. As a
result of focusing on getting things working with lina, it looks like
I've introduced some bugs that the lina version doesn't have.

4. When I had the crashing problem I said, screw it, I'll just use
someone else's webserver for now. So I installed NPM and then NPM
installed http-server, this "simple, zero-configuration command-line
http server": https://www.npmjs.com/package/http-server

It didn't work. Some kind of cultural bitrot within node.js has taken
place and now there's a completely different way to get the result I
expected. Instead of a local script, usable by anyone, it wants a
bunch of stuff installed across the system by root, or to avoid that
requirement I have to use some new tool that has itself the same
requirement.

Well, I know there's a Python oneliner (which doesn't correctly
implement CGI) of a webserver. Surely Perl has one as well? ... No it
doesn't. It has https://metacpan.org/pod/HTTP::Server::Simple , and
there's no "just serve files from a directory" default. You have to
code all that.

In the end the most convenient thing for me to do was to find my
development laptop and then copy over the lina version of
simple-website. It's a statically linked 315KB binary that serves the
current directory, and that was enough to get the job done.


Current plans: get engines other than poll() working again, and all at
least capable again of serving this webapp without crashing since
ApacheBench wasn't enough of a test for that. Then write another
-website.frt that can serve WordPress sites with a CGI handler.

I'm most familiar with WordPress, but if someone is aware of a popular
web framework that uses *more* rather than fewer scripts to do the
heavy lifting for the site, please point it out to me. Rather than an
index.php that loads the entire framework and responds to every
request, I'd prefer software with lots of specialized scripts that
handle different requests on their own.

Obviously, because software like that can be more easily
piecemeal-replaced with Forth.


Cheers,
Julian

Albert van der Horst

unread,
Dec 23, 2016, 2:53:32 PM12/23/16
to
In article <b59ef1ef-f6a7-4c8f...@googlegroups.com>,
Julian Fondren <julian....@gmail.com> wrote:
>Incidentally, yesterday, on a computer with a stale dev environment
>and a different Linux distribution, I needed to serve files out of a
>directory to use an application. So I updated this project and
>
>1. I couldn't build lina at all, because constants32.fs needed
>different header paths that I didn't know how to install on this
>distro. So the built lina binaries or the prebuilt constants*.fs files
>are MORE PORTABLE than the build system that produces them! This is
>how much of a pain it is to ask C for constants.

You mean the following didn't compile? (lina gets
at its constants by running the resulting c-program)

--------------------------------------------
/* $Id: stealconstant.c,v 5.6 2016/01/06 22:21:09 albert Exp $ */
/* Copyright(2000): Albert van der Horst, HCC FIG Holland by GNU Public License */
#include <stdio.h>
#include <stdlib.h>
#include <sys/times.h>
#include <unistd.h>
#include <linux/unistd.h>
#include <string.h>
#include <sys/time.h>
#include <sys/stat.h>
#include <termios.h>
#include <signal.h>
#include <setjmp.h>
#include <fcntl.h>
#include <errno.h>
#include <asm/ioctls.h>
#include <sched.h>

/*****************************************************************************/
/* */
/* This c-source is only intended to be print a number of EUQATES */
/* in form that is palatable for the assemblers used in generating */
/* ciforth. */
/* */
/*****************************************************************************/

/* Steal the information what value B has, leave it in A */
#define STEAL(A,B) printf("%s\tEQU\t0x%x\n", A, B );

/* Have a A that has the same value than in C. */
#define STEALNAME(A) STEAL( #A, A)

/* Have a A that has the number of system call B */
#define STEALSYS(A) STEAL( #A, __NR_##A)

int main()
{
printf("_C{ ------------------------------------------------------------ }\n");
printf("_C{ Start of constants stolen from C. }\n");
printf("_C{ ------------------------------------------------------------ }\n");
printf("\n");
STEALNAME(SEEK_SET)
STEALNAME(TCGETS)
STEALNAME(TCSETS)
STEALNAME(ECHO)
STEALNAME(EAGAIN)
STEALNAME(EINTR)
STEALNAME(EPIPE)
STEALNAME(VMIN)
STEALNAME(VTIME)
STEALNAME(ICANON)
STEALNAME(O_RDWR)
STEALNAME(O_RDONLY)
STEALNAME(O_WRONLY)
STEALNAME(O_CREAT)
STEALNAME(O_NONBLOCK)
STEALNAME(CLONE_VM)

STEAL("SIZE_TERMIO",sizeof(struct termios))

printf("\n");
printf("_C{ Numbers of system calls. See \"Linux kernel Internals\" Appendix A. }\n");
printf("_C{ By M.Beck, H. Boehme e.a. Addison Wesley. }\n");
printf("_C{ The system calls themselves are extensively documented in chapter }\n");
printf("_C{ 2 of the man pages, e.g. \"man 2 exit\"}\n");
STEALSYS(exit)
STEALSYS(open)
STEALSYS(close)
STEALSYS(creat)
STEALSYS(unlink)
STEALSYS(chdir)
STEALSYS(read)
STEALSYS(select)
STEALSYS(_newselect)
STEALSYS(write)
STEALSYS(ioctl)
STEALSYS(ioperm)
STEALSYS(iopl)
STEALSYS(lseek)
STEALSYS(execve)
STEALSYS(fork)
STEALSYS(waitpid)
STEALSYS(pipe)
STEALSYS(clone)
STEALSYS(kill)
STEALSYS(signal)
STEALSYS(time)
STEALNAME(__NR_times)
printf("_C{ ------------------------------------------------------------ }\n");
printf("_C{ End of constants stolen from C. }\n");
printf("_C{ ------------------------------------------------------------ }\n");
exit(0);
}
--------------------------------------------

Of course the numbers belonging to system calls are cast in concrete,
and should be documented independantly such that one can write a
c-compiler (or a Forth compiler). Now c-compiler writers and
kernel builders forming a clique to keep other compiler builders (read
Forth compiler builders) out.

However I didn't have to tweak this file a lot, this is the relevant
difference in include files, since 2002:
--------------------------------------------------
diff -r4.1 -r5.6
1c1
< /* $Id: stealconstant.c,v 4.1 2002/10/17 13:44:22 albert Exp $
*/
---
> /* $Id: stealconstant.c,v 5.6 2016/01/06 22:21:09 albert Exp $
*/
16c16,17
< #include <ioctls.h>
---
> #include <asm/ioctls.h>
> #include <sched.h>
--------------------

By adapting the macro STEAL you should be able to have your constants
in an appropriate format without including the lina binary, provided
you have a c-compiler.

>
>Well one thing I can do is just include lina binaries in the project
>then. They aren't IP encumbered and they can double as the
>more-portable way to get constants for another Forth's use.

In fact lina is hardly involved here. I stole the constants
from c in as straightforward way as possible, and they are just
passed to the assembler listing.
Note that I only stole the constants that are needed for my
Forth kernel. You're better off if you have that part of your
project under control yourself.
Interesting adventure. Things are made more complicated all the
time, layers upon layers.

>
>
>Cheers,
>Julian

Groetjes Albert

Paul Rubin

unread,
Dec 23, 2016, 4:26:34 PM12/23/16
to
Julian Fondren <julian....@gmail.com> writes:
> Well, I know there's a Python oneliner (which doesn't correctly
> implement CGI) of a webserver.

I've used CGIHTTPServer in the past and it's worked for me. Do you mean
it's wrong in some detail of the spec, or you just couldn't get it to
work?

Julian Fondren

unread,
Dec 23, 2016, 4:41:20 PM12/23/16
to
On Friday, December 23, 2016 at 1:53:32 PM UTC-6, Albert van der Horst wrote:
>
> You mean the following didn't compile? (lina gets
> at its constants by running the resulting c-program)
>

I mean my constants.c didn't. It's very similar to your code.
I got it to work. It mishandles CGI scripts in some way that escapes
me now, that resulted in requests hanging. It's possible to get simple
I/O to subprocesses wrong and still have things seem to work most of
the time. For example, you might perform a single read() with a very
large buffer on the pipe to the subprocesses, and then never try to
read() again when the CGI script produces some abbreviated output
(headers, say) but then keeps running for a bit to produce more.

polymorph self

unread,
Dec 29, 2016, 4:32:28 PM12/29/16
to
What can you do with it?
Any dynamic forum etc kind of site up?
Hows maintainability?
Does it run on any free forth?
pfx?
gforth?

Julian Fondren

unread,
Dec 29, 2016, 10:42:15 PM12/29/16
to
On Thursday, December 29, 2016 at 3:32:28 PM UTC-6, polymorph self wrote:
> What can you do with it?

Reply to HTTP requests. The *-website.frt files show specific
kinds of replies.

> Any dynamic forum etc kind of site up?

No. I've only used it for static sites.

> Hows maintainability?

Probably not great.

> Does it run on any free forth?

Yes: ciforth.

> pfx?
> gforth?

No, but porting to these would be easy provided that a SYSCALL
(ciforth: XOS and XOS5) or a C interface was available. Actually it'd
probably be even easier, and would demand less of the runtime
environment and of the end user, to build the required primitives into
the Forth system. I might demonstrate this with gforth.

Albert van der Horst

unread,
Dec 30, 2016, 11:11:48 AM12/30/16
to
In article <1b4ce172-5944-4130...@googlegroups.com>,
Julian Fondren <julian....@gmail.com> wrote:
>> Does it run on any free forth?
>
>Yes: ciforth.
>
>> pfx?
>> gforth?
>
>No, but porting to these would be easy provided that a SYSCALL
>(ciforth: XOS and XOS5) or a C interface was available. Actually it'd
>probably be even easier, and would demand less of the runtime
>environment and of the end user, to build the required primitives into
>the Forth system. I might demonstrate this with gforth.

There is an exciting possibility to do that, I've been experimenting
with. My forth.lab accomodates booting binary, MSDOS, WINDOWS DPMI,
WINDOWS 32 bits, Linux 32 and 64 bits, Apple.
What if we add an adaptation to gforth linux? That amounts to replacing
screen 1 (with the WANT mechanism) and the screens containing
error descriptions. The remainder would stay universal, because
those gforth specific screens (like one with XOS and XOS5)
don't bother an MSDOS user. They are prevented from loading on MSDOS.
Most gforth specific screens would be common with other Forth
implementations on *x like Apple and lina64.
0 new messages