Re: Which scripting for embedded automation?

[rickman]

On 7/23/2015 5:24 AM, abhishekku...@gmail.com wrote:
> I was automating my home LAN.
> Need suggestion for automation scripting upon embedded Linux, to do some operations over LAN, such as scan and ping other devices, string like data transfer,etc. And there is need of some logic to parse received content. Perl, python, lua or simple shell scripting .. which one? There will be branch and loop operations, better to have more ready tools for network operations..
>
> Parser and LAN are the key here.

The obvious choice for this is Forth. You can commercial products from
Forth, Inc in the US or MPE in Europe. A good open source product is
Gforth among many others.

--

Rick

[Mark Wills]

Do any of them support TCP/IP out of the box, for pinging and general
socket related stuff? I've never seen any example code, and, in the case
of VFX, I don't recall seeing anything in the manual - maybe it's included
in the professional version.

What about Swift and GForth?

[Stephen Pelc]

On Fri, 24 Jul 2015 01:12:30 -0700 (PDT), Mark Wills
<markwi...@gmail.com> wrote:

>Do any of them support TCP/IP out of the box, for pinging and general
>socket related stuff? I've never seen any example code, and, in the case
>of VFX, I don't recall seeing anything in the manual - maybe it's included
>in the professional version.

The OP said this was on embedded Linux, so TCP/IP is supported out of
the box. All our Wndows/Mac/Linux hosted systems include a socket
library. The Professional versions include higher level code such
as a web server.

For deep embedded systems, the Professional versions of the
cross-compilers include target code for TCP/IP, HTTP, FTP, ARP, DNS
and so on, including both web and Telnet servers.

Stephen

--
Stephen Pelc, steph...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

[rickman]

On 7/24/2015 1:00 AM, Paul Rubin wrote:

> abhishekku...@gmail.com writes:
>> like data transfer,etc. And there is need of some logic to parse
>> received content. Perl, python, lua or simple shell scripting .. which
>> one? There will be branch and loop operations, better to have more

>> ready tools for network operations... Parser and LAN are the key here.
>
> I'd suggest Python of the ones you mention, assuming you have enough ram
> (a Raspberry pi would be more than plenty). I use it all the time
> myself. Or if you want something really different and interesting, look
> at Erlang.
>
> Perl is old-school by now, and shell scripts too inflexible. Lua is
> very elegantly implemented and compact but it lacks Python's creature
> comforts. Forth is likely to be too low level for this sort of
> programming. It's virtue is being able to run in very small memory
> space, like under 10K of code space and a few hundred ram bytes in an
> MCU. But it makes programming very low level and painstaking, almost
> like assembler.

I find your analysis of Forth to be very naive. Much the same as other
languages like C, Forth provides a basic capability with more advanced
capabilities provided through libraries. Suggesting that Forth is only
useful where the entire development system needs to reside in 10 KB of
code is a red herring. This capability comes with limitations and so is
only used where it is a significant advantage. Otherwise Forth can be
used to advantage in the same environments where you would use C, Python
or any others. It typically comes with exactly the sort of capabilities
needed for this project. A parser is built in with full capability of
being used in the application and most Forths running under an OS
support networking.

Referring to Forth as being "like assembler" just shows how little you
understand the language and appreciate its features.

--

Rick

[Paul Rubin]

rickman <gnu...@gmail.com> writes:
[Ok, replying to your crosspost even though my own earlier post was only
on comp.arch.embedded where the original question was. The question
regards scripting home automation stuff on an embedded Linux system].

>> Forth is likely to be too low level for this sort of programming.
>> It's virtue is being able to run in very small memory space, like
>> under 10K of code space and a few hundred ram bytes in an MCU.

> I find your analysis of Forth to be very naive. Much the same as
> other languages like C, Forth provides a basic capability with more
> advanced capabilities provided through libraries.

Sure, I'd consider C to also be low level though. I'd have to add that
Forth doesn't have nearly as many available libraries, because of the
smaller userbase, but I think partly from a minimalist "do everything
yourself" spirit in Forth culture.

> Suggesting that Forth is only useful where the entire development
> system needs to reside in 10 KB of code is a red herring.

I didn't mean to suggest a 10K maximum. I meant Forth had the
capability of running in such small space and that this is IMHO a big
achievement and advantage that it has, but it dictated a lot of Forth's
design decisions. Historically Forth was most successful in the era of
16-bit systems with up to 64k ram, or maybe on MSDOS PC's with up to
640k. Once you have tens of MB available the tradeoffs in scripting
become a lot different.

Other Forth advantages include pauseless operation (necessary for
real-time programs), higher cpu efficiency than Python even in simple
interpreters, and very high efficiency (comparable to C) if you use an
optimizing compiler. But, in the context of scripting, these are
usually not important. Python burns more machine resources, but in
return it takes care of a lot of issues for you that you'd have to spend
your own time looking after in Forth or C.

> Otherwise Forth can be used to advantage in the same environments
> where you would use C, Python or any others.

I didn't say anything about C and I'd consider it comparable to Forth in
most regards. C would IMHO be a poor choice of scripting language in
this situation too.

Have you actually used Python? It's not anything like C or Forth.
Lumping Python with C seems to miss the distinction. If you haven't
actually used both, then making comparisons between them isn't
convincing.

> It typically comes with exactly the sort of capabilities needed for
> this project. A parser is built in with full capability of being used
> in the application and most Forths running under an OS support
> networking.

If the OP has control over the network protocol then using the Forth
parser can be a simple and efficient solution, especially if the network
devices are all trusted. If there are devices speaking protocols like
XML (some home automation stuff is like that) then Forth starts to be
painful. I don't know if a Forth TLS stack even exists.

> Referring to Forth as being "like assembler" just shows how little you
> understand the language and appreciate its features.

I understand Forth pretty well at a technical level, though I'm not a
very fluent user in terms of being used to the idioms it takes to use it
productively.

[foxaudio...@gmail.com]

I believe the Forth solution would be a new DSL for the problem (s) at hand. So it's hard to evaluate it since it does not exist yet. This is always a strange discussion for Forth since it can create a new language and in fact out the box is not very useful until properly extended.

Will it include re-entrant string functions and regex ? It could. But who knows until it's coded.

My 2 cents

BF

[m...@iae.nl]

On Friday, July 24, 2015 at 10:12:32 AM UTC+2, Mark Wills wrote:
[..]

> Do any of them support TCP/IP out of the box, for pinging and general
> socket related stuff? I've never seen any example code, and, in the case
> of VFX, I don't recall seeing anything in the manual - maybe it's included
> in the professional version.
>
> What about Swift and GForth?

iForth version 4.0.1782, generated 21:06:07, March 6, 2014.
x86_64 binary, native floating-point, extended precision.
Copyright 1996 - 2014 Marcel Hendrix.

FORTH> needs -sockets
Creating --- Socket interface Version 1.10 ---

Try: hostname TYPE
.QUOTE
.TIME-OF-DAY
S" home.iae.nl" #79 OPEN-SERVICE CONSTANT finger
S" mhx" +cr finger WRITE-SOCKET
finger PAD #1024 READ-SOCKET CR TYPE
finger CLOSE-SOCKET

Note: Perform the write/read on the same line or you may miss the response
Note: cliserv.frt contains a complete client-server example
Note: Some more internet examples in /dfwforth/examples/internet
ok
FORTH> hostname TYPE me-tarzan ok
FORTH> .QUOTE cr
"Here's the rule for bargains: "Do other men, for they would do you."
That's the true business precept." Charles Dickens (1812-70)
ok
FORTH> .TIME-OF-DAY
8:15:30 PM 7/24/2015

[Paul Rubin]

Mark Wills <markwi...@gmail.com> writes:
> Do any of them support TCP/IP out of the box, for pinging and general

> socket related stuff? ... What about Swift and GForth?

VFX does have some kind of network library, a full TCP implementation in
Forth for embedded systems, I believe. I don't know about Swift.
Gforth has the ability to call C libraries and there is an undocumented
file in the distro that supplies a socket interface, but it's a wrapper
around the Unix/Linux system calls.to use it.

http://www.complang.tuwien.ac.at/cvsweb/cgi-bin/cvsweb/gforth/unix/

[Elizabeth D. Rather]

On 7/23/15 11:21 PM, Stephen Pelc wrote:
> On Fri, 24 Jul 2015 01:12:30 -0700 (PDT), Mark Wills
> <markwi...@gmail.com> wrote:
>
>> Do any of them support TCP/IP out of the box, for pinging and general
>> socket related stuff? I've never seen any example code, and, in the case
>> of VFX, I don't recall seeing anything in the manual - maybe it's included
>> in the professional version.
>
> The OP said this was on embedded Linux, so TCP/IP is supported out of
> the box. All our Wndows/Mac/Linux hosted systems include a socket
> library. The Professional versions include higher level code such
> as a web server.

Yes, FORTH, Inc.'s Windows/Mac/Linux systems (including SwiftX/ARM)
support TCP/IP as well.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[rickman]

I cross posted this here so the OP could get some insight into the Forth
community. However, many of the replies are only to the Forth group
where they will never be seen by the OP.

--

Rick

[WJ]

Paul Rubin wrote:

> > Otherwise Forth can be used to advantage in the same environments
> > where you would use C, Python or any others.
>
> I didn't say anything about C and I'd consider it comparable to Forth in
> most regards. C would IMHO be a poor choice of scripting language in
> this situation too.
>
> Have you actually used Python? It's not anything like C or Forth.
> Lumping Python with C seems to miss the distinction. If you haven't
> actually used both, then making comparisons between them isn't
> convincing.

It's obvious that the reason that he is recommending Forth
over all other languages is that he knows no language that
is higher-level than Forth. And he finds it too difficult
to learn another language.

If a hammer is the only tool that you have, then you use
a hammer for everything.

How would these tasks be done with a hammer?

Using Gauche Scheme:

Write a function that accepts a vector (1-dimensional array) of
16-bit numbers and replaces each number with its difference
from the average.

(use gauche.uvector)
(use gauche.collection)

(define (relativize vec)
(let ((avg (round->exact (/ (fold + 0 vec) (s16vector-length vec)))))
(s16vector-sub! vec avg)))

(define my-vec (s16vector 0 1 2 3 4 5 6 7 8 9))
(relativize my-vec)
my-vec
===>
#s16(-4 -3 -2 -1 0 1 2 3 4 5)


Take a string that contains numbers separated by semicolons or
commas and convert it to a list of numbers.

(map string->number (string-split "2,3.14;5.8,9" #/[,;]/))
===>
(2 3.14 5.8 9)

--
The report card by the American Society of Civil Engineers showed the national
infrastructure a single grade above failure, a step from declining to the point
where everyday things simply stop working the way people expect them to. ---
washingtonpost.com/local/trafficandcommuting/us-infrastructure-gets-d-in-annual-report/2013/03/19/c48cb010-900b-11e2-9cfd-36d6c9b5d7ad_story.html

[Don Y]

What role would these have in the scripting of an automation system?
IME, tasks in that domain tend to be more procedural oriented.

[rickman]

WJ is an odd sort. You won't find that he is much of a
conversationalist. Rather he communicates in terms of code for any
manner of interesting, even if irrelevant, examples.

Just one of the many odd birds flitting about in the Forth newsgroup.

--

Rick

[Mark Wills]

He may be an odd sort, but his point is spot on. For *embedded scripting*
I wouldn't go near Forth in a bazillion years. Okay, if the *embedded*
machine I was using had like, 1K of RAM, then yes, Forth. Otherwise, no
way.

Look, if you are doing scripting, then you are doing things like:

* Pinging IP addresses (e.g. a gateway) and changing routing tables on
the fly depending on whether a particular interface is up or not.
* Reading through log files, looking for the certain occurrence of strings
(error codes/strings, other filenames etc.)
* Deleting log or error files
* Moving log or error files
* Rotating log or error files depending on their age
* Searching for files of a given type, or date range, and moving them,
or zipping them, or deleting them.

Forth has *no* standard way to do any of these things. You'd even have
to write your own string array handler FFS. Forget enumerating a disk
directory, getting a list of all the files in a structure, with their
creation dates, modification dates, filtering by type, sorting by date,
looking for duplicate file names (so that you can delete the older
versions) etc.

Are you kidding me?

You want Perl, PHP, and Python. Any of those will take you everywhere
you want to go.

Forth is next to useless for this sort of thing, unless you have a rack
of libraries that you have personally built up over the years. Because,
you know, this is Forth, right, and we don't use each other's libraries
in the Forth world, because everyone else's code sucks, right? We can
*all* do *much* better jobs by ourselves, re-inventing the wheel over
and over and over again.

Using Forth for this kind of thing would be like setting out to build a
house with a hammer and a screwdriver. Someone says "hey, where's the
cement mixer?" and you hold up the hammer and the screwdriver and say
"we're gonna build our own! It's gonna be the best cement mixer EVER!"

Don't get me started! I don't even have a cat that I can kick ;-)

Next-door neighbour... he has a cat...

[jo...@planet.nl]

Hi Mark,
Forth took me everywhere where I wanted to go when I wrote my server controller.
Especially itools.frt which I did not write myself, but found thanks to:
https://groups.google.com/forum/?fromgroups#!forum/comp.lang.forth
was very useful.

Jos

[rickman]

No, but you are kidding me. It is clear that you have virtually no
experience with Forth. Does a language need a "standard" way to do
everything you want to do? No, none of the things you mention are in
the Forth "standard" and yet, many apps have been written that do
exactly these things. In fact, most are a part of the Forth systems
that you would use. But if you check the OP's requirements, he didn't
mention any of these. "such as scan and ping other devices, string like

data transfer,etc. And there is need of some logic to parse received
content."

> You want Perl, PHP, and Python. Any of those will take you everywhere
> you want to go.
>
> Forth is next to useless for this sort of thing, unless you have a rack
> of libraries that you have personally built up over the years. Because,
> you know, this is Forth, right, and we don't use each other's libraries
> in the Forth world, because everyone else's code sucks, right? We can
> *all* do *much* better jobs by ourselves, re-inventing the wheel over
> and over and over again.
>
> Using Forth for this kind of thing would be like setting out to build a
> house with a hammer and a screwdriver. Someone says "hey, where's the
> cement mixer?" and you hold up the hammer and the screwdriver and say
> "we're gonna build our own! It's gonna be the best cement mixer EVER!"
>
> Don't get me started! I don't even have a cat that I can kick ;-)
>
> Next-door neighbour... he has a cat...

Wow! You clearly have some issues with programming languages. Perhaps
you should put down the keyboard and walk away?

--

Rick

[rickman]

On 7/25/2015 4:28 AM, Mark Wills wrote:

BTW, you only replied to the forth group. If you feel so strongly that
Forth is such a bad language, why are you here?

--

Rick

[Paul Rubin]

rickman <gnu...@gmail.com> writes:
>> Look, if you are doing scripting, then you are doing things like:

>> * Pinging IP addresses (e.g. a gateway) and changing routing tables..
>> * Rotating log or error files depending on their age...

>> * Searching for files of a given type, or date range, and moving them,
>> or zipping them, or deleting them.
>> Forth has *no* standard way to do any of these things.

> No, but you are kidding me. It is clear that you have virtually no
> experience with Forth.

Based on Mark's earlier posts going back for years, he is an
accomplished Forth user and implementer.

> Does a language need a "standard" way to do everything you want to do?

It helps if there's at least a documented way in a given implementation.

> No, none of the things you mention are in the Forth "standard" and
> yet, many apps have been written that do exactly these things. In
> fact, most are a part of the Forth systems that you would use.

There's probably some way in gforth to do those things, but looking at
http://www.complang.tuwien.ac.at/forth/gforth/Docs-html/
for example, I don't how to get the timestamp (last modification
time) of a file. There might be some library in the distro somewhere
for it, but otherwise it means using the C interface.

> But if you check the OP's requirements, he didn't mention any of
> these.

It's a reasonable basic premise that a networked automation controller
should write logs of what it's been doing. I haven't heard of any
reasonable logging system in Forth. Logging is a surprisingly
complicated problem and there are lots of competing solutions in other
languages, that all have their good points and drawbacks. But the basic
requirements are:

- multiple concurrent tasks in the application have to be able to emit
log messages simultaneously, and these messages get written to the
same log without getting garbled
- logging should be asynchronous from the application, so if your
task logs a message it shouldn't have to wait for the log i/o before
it can keep running
- log messages can at times be emitted faster than the log output
device can keep up with, so there has to be some buffering
- you need strategies for when there is too much stuff to buffer,
for when the disk gets full, etc
- you want to be able to log to disk files and also over the network,
maybe simultaneously
- for logging on disk, you want to automatically rotate log files
every so often and/or when they reach a specified size
- you need precise control of the log format so you can satisfy the
formatting requirements of whatever is consuming the logs (that might
some analytics tool not written by you, or it might even be another
organization)
- etc. etc. etc.

The docs for the Python standard logging module are here:
https://docs.python.org/3/library/logging.html

That system is clumsy to use and I'd call it "unpythonic" (not a good
example of idiomatic Python) and I suspect it's patterned after a Java
package, but it does most or all of the stuff above, and the stuff is
not useless bloat. It really does become necessary from time to time,
so that module replaced earlier, simpler ones that were missing needed
functionality.

And no it's not always better to start with something simple and
implement what you need as you go along. Your program is may want to
use 3rd party modules that themselves have to log stuff, so you want a
standardized logging interface that implements everything up front, and
whose availability the different module authors can rely on.

> "such as scan and ping other devices, string like data
> transfer,etc. And there is need of some logic to parse received
> content."

Yes, that sounds like a big pain in Forth. For example, lots of home
devices (think of your DSL router, but these days even things like
refrigerators) have built-in web servers that show UI's meant to be used
by humans rather than programs. So if you want to automate those
devices, your script has to scrape the HTML and interact with it.
There's a nice Python library that I use all the time for that:

http://www.crummy.com/software/BeautifulSoup/bs4/doc/

I can't even imagine what a hassle it would be to do that in Forth.

As an issue not even mentioned, an automation system probably wants to
communicate with several targets concurrently, so some kind of
multitasking is helpful, or else you need asynchronous i/o facilities
and callback pain. I think Gforth now has preemptive multitasking using
Posix threads, but it's not currently documented. You might have to
deal with the traditional hazards of low level threaded programming.
Python has multiple concurrency approaches that provide a reasonable
level of safety, and Erlang is even better (independent lighweight
processes that communicate by message passing).

> Wow! You clearly have some issues with programming languages.
> Perhaps you should put down the keyboard and walk away?

I'm not trying to insult you since I know you're a strong implementer in
some areas, but I suspect you haven't done a lot of internet
programming. It's not like MCU programming where you're in control of
everything. There's much more interoperating with external services
written by other people and whose crufty requirements you have to deal
with, more weird formats to parse, concurrency is required rather often,
for non-stop systems you need code upgrade and failover mechanisms
(Erlang again has the best story for this: you can upgrade your app
without interrupting its existing network connections), etc.

Forth (and for similar reason C) work nicely when the data fits
naturally into cells (machine words). For variable sized data like
strings, Forth has some features that can be used with a bit of
headache. For more complicated data like HTML documents that are
manipulated as tree structure, you're really better off with a garbage
collected language.

You might give Python a try sometime. There's lots of intro material:

https://wiki.python.org/moin/BeginnersGuide/Programmers

I got started with the official tutorial, current version here:

https://docs.python.org/3/tutorial/index.html

You might want to look at section 3 (intro to the language) first. The
earlier sections are stuff like invocation instructions.

Here is a set of Forth solutions for the first 25 Euler problems:

http://www.fourtheye.org/cgi-bin/language.pl?language=forth

The same guy also did solutions in various other languages, so you can
compare their complexity:

http://www.fourtheye.org/projecteulertable.html

(his Python solutions are a bit clunky in my opinion, and warning, some
of them might only work in Python 2 since in Python 3 they changed some
stuff).

Here are Haskell solutions:

https://wiki.haskell.org/Euler_problems

Lots of the time it should be able to write Python code that's fairly
similar to the Haskell code.

[glid...@gmail.com]

Mark's lament is understandable. Forth doesn't even have a built-in
cell-sized-element array wordset. I think that sort of thing is where
he's coming from. But that's Forth. I long ago made peace with this
aspect and accept Forth as it is and for what it is (and isn't).

-Doug

[Alex McDonald]

on 26/07/2015 12:01:26, glidedog wrote:
\

>
> Mark's lament is understandable. Forth doesn't even have a built-in
> cell-sized-element array wordset.

The definition of a 1 dimensional array is trivial; perhaps that's why.

: 1dim create cells allot does> swap cells + ;

10 1dim myarray
10 0 myarray !
20 1 myarray !
0 myarray @ 1 myarray @ + .

[Paul Rubin]

"Alex McDonald" <bl...@rivadpm.com> writes:
> The definition of a 1 dimensional array is trivial; perhaps that's why.
> : 1dim create cells allot does> swap cells + ;

You're going to use arrays with no bounds checks to script potentially
hostile internet devices? Sounds dangerous to me.

[rickman]

I think that it can be argued that given the extensible nature of forth
that it is better to learn to craft simple structures like these in
order to help with learning how to use these principles in more
complicated and useful structures.

It seems like many prefer to use a language (or languages) which have
already done the work of providing lower level features for any given
application. While this seems like a good idea, it requires learning
the new language most appropriate for every new application. Notice
that Mark mentioned several languages and WJ is often giving examples in
several.

Is that really any better or easier than learning a single language
where you can create all of these advantages at the same time?

--

Rick

[Alex McDonald]

That wasn't the original complaint; it was that there was not even a cell
sized array in Forth.

: 1dim create dup , cells allot
does> 2dup @ 0 within abort" index out of bounds"
swap 1+ cells + ;

Now we have a cell sized bounds checked array.

[Julian Fondren]

One way to view this is, sure, your arrays should have bounds checking.
And so should every other level of every other part of your program.
You should everywhere find words running paranoid checks on values
right before those values are fed into other words that run the same
paranoid checks against them. Nowhere in your program may you say "I
trust what has gotten this far, because it's surely been checked by
now." You may, at best, have transition points where the tests kill
your program with "the impossible happened--bailing out!" rather than
gracefully handle an error condition. The only escape from this
drudgery? A really smart compiler, that can know when the checks
aren't unnecessary, and which can even come up with checks by itself.

The advantage of this view, if you stick with it in Forth, is that you
can, with some effort, achieve something like the absolutely miserable
results that people manage with other languages that offer more
advanced support for it.

Another way to view this is... huh? What do *internet devices* have to
do with *arrays*?! Let's say you're writing an IRC server. One thing
IRC servers do is allow, into a room, a number of users. Faced with
the famous "zero, one, or many" question of what limits to have, you of
course pick an arbitrary configurable number, and then document the
limits and resource implications of that number. Software, you see,
does not in fact run on hardware possessed of infinite resources, and
network services lack magic hardware even more severely than most
software. So your configurable number makes its way to two separate
places in your code: the size of an array of users in a room, where
there is no bounds checking whatsoever, and also to the 'join a user to
a room' function of the server, which refuses to join users to a room
that, by that number, is full. The cost of the bounds checking is
directly tied to a user's attempt to enter a room, which helps you
consider the costs of a user's total actions, which helps you kick
users who are abusing your server. The error condition of a user
trying to join a full room is obvious: "Sorry, the room is full." You
don't need to throw an exception from deep within your code--you're
already where you want to be, processing a user request and prepared to
reply to it.

Or you could, the moment you have a need for an array, spend some time
adding bounds checking to it, just so that you can kill the program
with an "index out of bounds" error. What a wonderful thing for a
server to do out of the blue. And it *will* be 'out of the blue'
because the only reason you would code like this is if the rest of the
server's code was a dark and scary labyrinth to you. You don't know
what you're doing, so you write defensive code, and the result is that
you still don't know what you're doing and your defensive code now
sometimes kills your server based on decisions made far away from your
code. So have fun debugging that!

-

Of course, dark and scary labyrinths are certainly out there. A while
back I added a feature to such a body of code. A bug in my addition
would have harmlessly caused the program to fail to do its job, which
would've been noticed and which would've resulted in mild
embarrassment, if not for the bug resulting later on in a function
getting called with a rather unpleasant argument. The result? The
program failed to do its job -- and also inflicted enormous performance
costs on the servers it ran on. It wasn't noticed for some time that
my code push coincided with the sudden spate of performance issues. I
spent a lot of time cursing and swearing while personally fighting the
fires that resulted from these performance issues, without suspecting
that I'd caused them...

Would that have been helped by boundary-checked data structures? No, I
had them. Would it have been helped by algebraic types and
compile-time checking of them by a really smart compiler? No, the
argument was validly typed as far as the performance-killing function
was concerned. And since we weren't asking the question of should the
potential arguments to this function be more constrained than that
function was designed to accept, we didn't answer that question with a
new constrained type, or an additional test, or scrutiny towards the
potential arguments to it that could arise in the body of code.
Additionally, I wasn't aware of how the function was respond to the
argument it got; if you'd asked me, I would guessed that it would very
quickly do nothing.

What might've actually helped: instead of trusting in my ability to
dive into a 500-line or whatever *massive* routine and make just the
demanded changes, I could've pulled out the section I was editing into
its own function, Forth-style, as a new factor with precisely
understood and considered inputs and outputs. I also could've followed
up on the logs and performance of this program after the push even
though I knew I had made very minor changes.

That is, what would've helped was all programmer-side, not
language-side.

-

Still, I introduced a bug and bad things happened. So also could
someone edit the 'join an IRC channel' code and accidently introduce a
bug that then caused a buffer overflow in that internal 'users in a
channel' array. And so, even with a clear and complete understanding
of your flawless IRC server, you might add some checks at some
boundaries, to protect parts of it from bugs that may later get
introduced by other people into other parts of it.

There's a fractal here: the network devices *could* be trusted because
the whole network is trusted and carefully run, and then there'd be no
reason even to write in "Sorry, the channel is full" sorts of code. But
you don't trust the network, and the same impulse you have to consider
such 'errors' coming the network, you can repeat at deeper and deeper
levels of your server. The network could be at harmony but instead the
server distrusts the network and the clients all distrust the server;
the server could be in harmony but instead libraries within the server
distrust the code that uses them; individual libraries could be in
harmony but instead functions distrust other functions within the same
library. At a deep enough level your real concern with that 1DIM array
word is that code from a completely different module can alter the
memory space 1DMIN uses for its array, without going through proper
array words or 'join a user to a channel' code or anything.

The point is, Alex wrote an array word in Forth. If you're going to
bother with Forth, you may as well try to get as far as you can with
harmonious, fully understood, brightly-lit code. There's where the
benefits lie. There's also where the deficiencies don't worry you as
much. You do not whinge, for example, about not having a potentially
untrustworthy array library that you'd defend your code against.

You're not dropping knowledge bombs on feckless monkeys who would come
out with software ridden through with buffer overflows. You just kinda
come off like a tourist.


Meanwhile, here is an array library with bounds checking:

https://code.google.com/p/ffl/wiki/car

Yes, even CAR-SET throws an exception. CAR-OFFSET does the work:

https://code.google.com/p/ffl/source/browse/trunk/ffl/car.fs


-- Julian

http://www.catb.org/jargon/html/Z/Zero-One-Infinity-Rule.html
(Wisdom that's appropriate when "I ran out of memory and crashed
trying to do what you just stupidly told me to do" is an acceptable
behavior. So, rarely. And never if you're writing a server.)

[Rod Pemberton]

On Sun, 26 Jul 2015 13:53:14 -0400, Alex McDonald <bl...@rivadpm.com> wrote:
> on 26/07/2015 12:01:26, glidedog wrote:

...

>> Mark's lament is understandable. Forth doesn't even have a built-in
>> cell-sized-element array wordset.
>
> The definition of a 1 dimensional array is trivial; perhaps that's why.
>
> : 1dim create cells allot does> swap cells + ;
>
> 10 1dim myarray
> 10 0 myarray !
> 20 1 myarray !
> 0 myarray @ 1 myarray @ + .
>

Okay, don't blow a fuse for me butting in without reading most of this, but ...

How is that on factoring?

Shouldn't there be a separate word to do array indexing so the indexing
functionality can be re-used? Then, it would be factored, correct? ...

E.g.,

: [] swap cells + ;
: 1dim create cells allot ;

10 1dim myarray
10 0 myarray [] !
20 1 myarray [] !
0 myarray [] @ 1 myarray [] @ + .

Isn't the non-factoring of Forth words the core issue Mark is getting at?

Or, perhaps, he is pointing out that everyone has forgotten Forth has or
had specialty words like +! and TOGGLE .

If Forth has or had *those* words, why shouldn't Forth have a word to index
into arrays?


Rod Pemberton

--
Scientists now say we'll experience a mini-ice in 2030.
So, I guess we need more global warming today ...

[Alex McDonald]

on 26/07/2015 22:11:25, "Rod Pemberton" wrote:
> On Sun, 26 Jul 2015 13:53:14 -0400, Alex McDonald <bl...@rivadpm.com>
> > wrote:
>> on 26/07/2015 12:01:26, glidedog wrote:
> wrote: ...
>
>>> Mark's lament is understandable. Forth doesn't even have a built-in
>>> cell-sized-element array wordset.
>>
>> The definition of a 1 dimensional array is trivial; perhaps that's why.
>>
>> : 1dim create cells allot does> swap cells + ;
>>
>> 10 1dim myarray
>> 10 0 myarray !
>> 20 1 myarray !
>> 0 myarray @ 1 myarray @ + .
>>
>
> Okay, don't blow a fuse for me butting in without reading most of
> this, but ...
>
> How is that on factoring?
>
> Shouldn't there be a separate word to do array indexing so the
> indexing functionality can be re-used? Then, it would be factored,
> correct? ...
>
> E.g.,
>
>: [] swap cells + ;
>: 1dim create cells allot ;

Not really. You're not actually factoring.

6 1dim ARRAY
8 1dim BRRAY

ARRAY and BRRAY use the same code compiled in the DOES> clause.

>
> 10 1dim myarray
> 10 0 myarray [] !
> 20 1 myarray [] !
> 0 myarray [] @ 1 myarray [] @ + .
>
> Isn't the non-factoring of Forth words the core issue Mark is getting
> at?

Just declare it as e.g. 6 1dim myarray[] . Then 10 0 myarray[] ! .

>
> Or, perhaps, he is pointing out that everyone has forgotten Forth has
> or had specialty words like +! and TOGGLE .
>
> If Forth has or had *those* words, why shouldn't Forth have a word to
> index into arrays?

A scheme with such a generic index makes it difficult to create cell,
char or other sized elements, or do simple bounds checking. The object
should do that for itself.

>
>
> Rod Pemberton
>

[Paul Rubin]

rickman <gnu...@gmail.com> writes:
> While this seems like a good idea, it requires learning

> the new language most appropriate for every new application....

> Is that really any better or easier than learning a single language
> where you can create all of these advantages at the same time?

Yes it is, because 1) it's impossible to create all of the advantages at
the same time, and 2) you don't need a new language for every new
application--you can reasonably call yourself a versatile programmer
with a handful of languages but not just one.
- a low level systems language like C or Forth,
- a dynamic scripting language like Python
- a typed, managed HLL for large scale development. The most widely
used of these is currently Java, which sucks, but there are starting
to be alternatives
- Subjectively I'd add a typed functional langage like ML or Haskell
just because of the way it broadens one's thought processes
- you will probably need SQL from time to time

[Elizabeth D. Rather]

See, that's the point. Where error checking goes should be a function of
the application design. Within the scope of the application, you can
know how reliable your data is (it's a property of the device). If it's
coming from a device, for example, you know exactly its range. If it's
from a less reliable source, you can check the value when it enters the
system (e.g., making a human user re-enter an illegal value). That kind
of approach yields both improved security and improved performance. A
generic definition cannot know the environment in which it will be used.

[Elizabeth D. Rather]

On 7/26/15 11:11 AM, Rod Pemberton wrote:
> On Sun, 26 Jul 2015 13:53:14 -0400, Alex McDonald <bl...@rivadpm.com> wrote:
>> on 26/07/2015 12:01:26, glidedog wrote:
> ...
>
>>> Mark's lament is understandable. Forth doesn't even have a built-in
>>> cell-sized-element array wordset.
>>
>> The definition of a 1 dimensional array is trivial; perhaps that's why.
>>
>> : 1dim create cells allot does> swap cells + ;
>>
>> 10 1dim myarray
>> 10 0 myarray !
>> 20 1 myarray !
>> 0 myarray @ 1 myarray @ + .
>>
>
> Okay, don't blow a fuse for me butting in without reading most of this,
> but ...
>
> How is that on factoring?
>
> Shouldn't there be a separate word to do array indexing so the indexing
> functionality can be re-used? Then, it would be factored, correct? ...
>
> E.g.,
>
> : [] swap cells + ;
> : 1dim create cells allot ;
>
> 10 1dim myarray
> 10 0 myarray [] !
> 20 1 myarray [] !
> 0 myarray [] @ 1 myarray [] @ + .

That is, in fact, very poor factoring, because it makes an extra
definition that you don't need. As Alex points out, multiple arrays
defined by 1dim will all share the same indexing code, so factoring that
out introduces an extra call.

> Isn't the non-factoring of Forth words the core issue Mark is getting at?
>
> Or, perhaps, he is pointing out that everyone has forgotten Forth has or
> had specialty words like +! and TOGGLE .
>
> If Forth has or had *those* words, why shouldn't Forth have a word to index
> into arrays?

Because you don't necessarily want all your arrays to have the same
runtime behavior. In addition to the fact that some will be cell-wise
and some byte-wise or even contain doubles or floats, some will require
bounds checking and some won't, and others will be more convenient if
they have more complex behaviors.

[Rod Pemberton]

If the "extra" call is such an issue, what is wrong with this?

: [] swap cells + ;
: 1dim create cells allot does> [] ;

10 1dim myarray
10 0 myarray !
20 1 myarray !
0 myarray @ 1 myarray @ + .

That preserves the functionality as a separate word and re-uses the same
code in the does> .

BTW, I see the "extra" call as being no different that the use of ! or
@ above. I.e., they're extraneous too and can be hidden within long,
non-factored definitions.

[Elizabeth D. Rather]

Offering the functionality as a separate word is advantageous when it is
a common sequence that is being reused. Such functions are worth
factoring and being given a clear name. The longer the phrase, the fewer
instances it takes to justify being factored. This particular phrase is
very short, quite unlikely to be repeated beyond the use here, and the
name [] conveys absolutely nothing resembling its function (square
brackets in Forth suggest immediacy).

[Rod Pemberton]

Sure. [] can be used with any address to perform indexing interactively,
just like @ and ! can be used.

> Such functions are worth factoring and being given a clear name.

Exactly. [] is just like @ and ! in that regard.

> The longer the phrase, the fewer instances it takes to justify
> being factored. This particular phrase is very short,

Yes. It's very short just like @ and ! are very short.
They were factored. Are you saying they shouldn't have been?

> quite unlikely to be repeated beyond the use here,

It can be used from the interpreted state with any address.
I.e., it's a generic addressing operation like @ ! +! TOGGLE
etc. So, why exactly would it be unlikely to be used? ...

> and the
> name [] conveys absolutely nothing resembling its function
> (square brackets in Forth suggest immediacy).

1) Sure it does. [] are used as array indexing in most languages.
2) If [] "suggests immediacy" in Forth, then you're implying Forth
*does* have some basic syntax rules ...

I know you attempted to point out why you shouldn't do things
this way, but from my perspective (above) it seems you only
confirmed why you should.

Isn't this really the fact that you hate attempts to make Forth
appear more C like and function that way too? I.e., coding in
Forth has a certain character and you'd like it to remain that way.
If so, doesn't that really go against the entire concept of Forth
being and adaptable and structureless or formless language? I.e.,
you're attempting to impose a form on it or your form. Aren't you?

C captures the essence of all generic microprocessors. Forth,
instead, usually attempts to impose it's own virtual machine.
So, in that regard, it would actually suit Forth to be more
C like. Even Hugh is having difficulty adding functionality to
Forth without recreating C functionality. He's done so twice
with projects even though he riles against C-like memory allocation
for Forth. Anton doesn't even bother attempting to hide from C.

[humptydumpty]

On Monday, July 27, 2015 at 12:11:11 AM UTC+3, Rod Pemberton wrote:

Hi!

For factoring I would go more deeper than that
and I'll propose "indexing" operator as '*/' is "scaling" operator:

: *+ ( a b c -- a+b*c ;"indexer" should be native code) * + ;

By commutativity of '*' we have flexibility for order of stack parameters.

So when in interactive mode do a 'see' a observe something like:

.. Alpha /BOX 5 *+ ...
or
.. Alpha 5 /BOX *+ ...

I would know that 'Alpha' is an array of Boxes.
If I want know size of a Box I would do '/BOX .'


If I want to write less I would do something equivalent with:

HERE 64 /BOX * allot /BOX 2CONSTANT Alpha

and use:

Alpha 5 *+


Have a nice day,
humptydumpty

[Anton Ertl]

humptydumpty <oua...@gmail.com> writes:
>For factoring I would go more deeper than that
>and I'll propose "indexing" operator as '*/' is "scaling" operator:
>
>: *+ ( a b c -- a+b*c ;"indexer" should be native code) * + ;

Following the stack effect of */, a word with this stack effect should
be called +*, or a word with this name should be defined as:

: *+ ( a b c -- a*b+c ) >r * r> + ;

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2015: http://www.mpeforth.com/euroforth2015/euroforth2015.htm

[Mark Wills]

I'll call you on that. Citations please.

[Anton Ertl]

This demonstrates why there is no built-in array word in Forth. You
may want named, bounds-checked arrays, you may want named
non-bounds-checked arrays, you may want unnamed bounds-checked arrays,
or you may want unnamed non-bounds-checked arrays. Now combined this
with the option of lower bound <>0, and you get 8 variants. And I
guess one can come up with cases where that is not sufficient.

BTW, instead of 0 WITHIN, you can also write U< 0= (or non-standard
U>=). Let's compare a few variations and see what VFX generates:

: foo1 0 within ; ok
see foo1
FOO1
( 080BF310 BA00000000 ) MOV EDX, 00000000
( 080BF315 2BD3 ) SUB EDX, EBX
( 080BF317 8B4D00 ) MOV ECX, [EBP]
( 080BF31A 2BCB ) SUB ECX, EBX
( 080BF31C 3BCA ) CMP ECX, EDX
( 080BF31E 1BDB ) SBB EBX, EBX
( 080BF320 8D6D04 ) LEA EBP, [EBP+04]
( 080BF323 C3 ) NEXT,
( 20 bytes, 8 instructions )

: foo2 u< 0= ;
FOO2 is redefined ok
see foo2
FOO2
( 080BF370 3B5D00 ) CMP EBX, [EBP]
( 080BF373 0F97C3 ) SETNBE/A BL
( 080BF376 F6DB ) NEG BL
( 080BF378 0FBEDB ) MOVSX EBX, BL
( 080BF37B F7D3 ) NOT EBX
( 080BF37D 8D6D04 ) LEA EBP, [EBP+04]
( 080BF380 C3 ) NEXT,
( 17 bytes, 7 instructions )

: foo3 u>= ; ok
see foo3
FOO3
( 080BF3A0 3B5D00 ) CMP EBX, [EBP]
( 080BF3A3 0F96C3 ) SETBE/NA BL
( 080BF3A6 F6DB ) NEG BL
( 080BF3A8 0FBEDB ) MOVSX EBX, BL
( 080BF3AB 8D6D04 ) LEA EBP, [EBP+04]
( 080BF3AE C3 ) NEXT,
( 15 bytes, 6 instructions )

[Anton Ertl]

"Rod Pemberton" <b...@fasdfrewar.cdm> writes:
>If Forth has or had *those* words, why shouldn't Forth have a word to index
>into arrays?

There is a commonly-used word

: th cells + ;

But this word is limited to cells. So we would also need

: fth floats + ;

and more for other types. Not sure if that really helps, although I
have occasionally used TH. Maybe CELLS, FLOATS etc. are the factors
you are looking for, and the array access words that are supposedly
lacking. I so, we should also have DOUBLES (or maybe 2CELLS), and, if
we define complex numbers, COMPLEXS; and for additional types that we
might want to put in array, similar words.

[Elizabeth D. Rather]

SwiftForth can do most of those things, and even our DOS-based systems
from years ago can do many of them.

>> In fact, most are a part of the Forth systems
>> that you would use. But if you check the OP's requirements, he didn't
>> mention any of these. "such as scan and ping other devices, string like
>> data transfer,etc. And there is need of some logic to parse received
>> content."

Exactly. Forths that are co-resident with an OS have a straightforward
interface to the OS's ability to do this stuff. And in many standalone
embedded projects we have worked on we have done many of them.

[Paul Rubin]

an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> This demonstrates why there is no built-in array word in Forth. You
> may want named, bounds-checked arrays, you may want named
> non-bounds-checked arrays, you may want unnamed bounds-checked arrays,
> or you may want unnamed non-bounds-checked arrays. Now combined this
> with the option of lower bound <>0, and you get 8 variants.

We're talking about scripting so there is no need for non-checked
arrays. I don't see why lower bound <>0 makes more variants: Ada and
Haskell support that for ordinary arrays. Other languages don't support
it, suggesting that it's not so useful most of the time. Of somewhat
more relevance for scripting is dictionaries as first-class values. For
that, native string values and GC or refcounting help a lot.

[Elizabeth D. Rather]

Yes. The difference is that @ and ! are ubiquitous, whereas this
particular, very specific indexing sequence would normally be
incorporated in the DOES> portion of a defining word, and never used
independently.

>> quite unlikely to be repeated beyond the use here,
>
> It can be used from the interpreted state with any address.
> I.e., it's a generic addressing operation like @ ! +! TOGGLE
> etc. So, why exactly would it be unlikely to be used? ...

It wouldn't be used independent of the array defining word because there
is no need for it except in that context. It has little value as an
independent facility.

>> and the
>> name [] conveys absolutely nothing resembling its function
>> (square brackets in Forth suggest immediacy).
>
> 1) Sure it does. [] are used as array indexing in most languages.
> 2) If [] "suggests immediacy" in Forth, then you're implying Forth
> *does* have some basic syntax rules ...

It has mnemonic value, consistent with [ ... ], ['], [CHAR], etc.

> I know you attempted to point out why you shouldn't do things
> this way, but from my perspective (above) it seems you only
> confirmed why you should.
>
> Isn't this really the fact that you hate attempts to make Forth
> appear more C like and function that way too? I.e., coding in
> Forth has a certain character and you'd like it to remain that way.
> If so, doesn't that really go against the entire concept of Forth
> being and adaptable and structureless or formless language? I.e.,
> you're attempting to impose a form on it or your form. Aren't you?

I don't see a value in making a horse look more like a cow just to prove
a point. Each is useful within its domain.

> C captures the essence of all generic microprocessors. Forth,
> instead, usually attempts to impose it's own virtual machine.
> So, in that regard, it would actually suit Forth to be more
> C like. Even Hugh is having difficulty adding functionality to
> Forth without recreating C functionality. He's done so twice
> with projects even though he riles against C-like memory allocation
> for Forth. Anton doesn't even bother attempting to hide from C.

People try to make Forth look like other languages either because the
other is more familiar to them or to prove a point. I'm primarily
interested in whether a potential functionality is directly applicable
to the project at hand, and what value it adds to the solution.

[Mark Wills]

Apps. He said apps. "Many apps have been written which do exactly these
things". I don't think so. A few? Sure. You just wouldn't go to Forth
for this sort of thing.

If I wanted to:

* Rotate error logs in a directory (say, keep the last three)
* Log on to a Cisco router
* Download the day's error logs and copy to the directory
* FTP the new error log file9s) to an off-site backup server
* Dynamically generate a clean-up script to clean the backup server
* FTP that file to the backup server
* Do a CHMOD on the FTPd backup file via a Unix shell to give it
execute permissions
* Execute the cleanup script
* Open the logs and analyse for particular occurences of error strings
* When found, read the specific error information, and update an SQL
database with the stats
* Execute an SQL stored procedure to insert a record into a database
to indicate that the script(s) ran/didn't run with/without errors
* Send an email to sysadmin indicating that the scripts ran with/without
errors

then I sure as hell wouldn't ever think of Forth in a million years.

The above is typical of many office-grown back-office admin scripts
written in their millions by millions of sys-admins the world over
every day of every week.

None of them will be written in Forth. They'll be written in Perl or
Python. Some of them (windows boxes) will be written in ASP though
that's rather old-hat these days. Perl and Python are cross-platform
so it's generally preferable to sys-admins to learn Perl and/or Python
as they can write scripts on a multitude of platforms: Win, Lin, Unix,
OSX, Sun etc.

[humptydumpty]

On Monday, July 27, 2015 at 10:35:56 AM UTC+3, Anton Ertl wrote:

Hi!

In time, I learned to not fight the stack, but follow it.
^^^^
I know that sounds like old jokes from "Tao of programming" .. :)

[Stephen Pelc]

On Mon, 27 Jul 2015 01:30:10 -0400, "Rod Pemberton"
<b...@fasdfrewar.cdm> wrote:

>C captures the essence of all generic microprocessors.

Huh? Justify that, please. What is a generic microprocessor?

Stephen

--
Stephen Pelc, steph...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

[Anton Ertl]

Paul Rubin <no.e...@nospam.invalid> writes:
>an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>> This demonstrates why there is no built-in array word in Forth. You
>> may want named, bounds-checked arrays, you may want named
>> non-bounds-checked arrays, you may want unnamed bounds-checked arrays,
>> or you may want unnamed non-bounds-checked arrays. Now combined this
>> with the option of lower bound <>0, and you get 8 variants.
>
>We're talking about scripting so there is no need for non-checked
>arrays.

The complaint was about Forth, which is not limited to scripting. So
if anybody would have added more array-specific words, he would have
to either make a choice between these variants, or provide them all.

>I don't see why lower bound <>0 makes more variants: Ada and
>Haskell support that for ordinary arrays. Other languages don't support
>it, suggesting that it's not so useful most of the time.

So, not useful most of the time, but useful some of the time, but at
extra implementation cost. Sounds like anybody providing such words
would have to make a choice, or provide all variants.

>Of somewhat
>more relevance for scripting is dictionaries as first-class values. For
>that, native string values and GC or refcounting help a lot.

In my experience, they would not help at all. What I missed from
standard Forth in this area is:

* A variant of WORDLIST that does not fold case, in order to be able
to use arbitrary data as keys. Gforth provides that with TABLE.

* A good interface to insert programmatically-generated keys into
WORDLISTs or TABLEs. Gforth provides NEXTNAME, or one could use
EXECUTE-PARSING, but both are not really satisfactory.

What I have not missed is the capability to delete specific items from
a wordlist/table. I have also not missed "native strings" (whatever
that may be) or GC or refcounting.

[glid...@gmail.com]

On 7/27/15 3:36 AM, Anton Ertl wrote:

> "Alex McDonald" writes:
>> on 26/07/2015 19:18:20, Paul Rubin wrote:

>>> "Alex McDonald" writes:
>>>> The definition of a 1 dimensional array is trivial; perhaps
>>>> that's why. : 1dim create cells allot does> swap cells + ;
>>>
>>> You're going to use arrays with no bounds checks to script
>>> potentially hostile internet devices? Sounds dangerous to me.
>>>
>>>
>>
>> That wasn't the original complaint; it was that there was not even
>> a cell sized array in Forth.
>>
>> : 1dim create dup , cells allot does> 2dup @ 0 within abort" index
>> out of bounds" swap 1+ cells + ;
>
> This demonstrates why there is no built-in array word in Forth.

Does it? Not sure what harm providing Alex's first-try version as built
in would cause. It's certainly a very common use case. It would not
not prevent one from creating different versions if desired.

As it is newbies to Forth must first learn the create ... does> idiom,
which is definitely not trivial in my opinion, before they can do
something as common as using an array the way Alex shows which I think
is a good one. Or they must search the various libraries.

It took me awhile to realize that the Forth standard, by its own
explicit declaration, is constructed to minimize the word set
to primarily support embedded programming.
From the Forth standard 29 Feb 2012
"A.1.2 Scope
The result of the effort to satisfy all of these objectives is a
Standard arranged so that the required word set remains small. Thus ANS
Forth can be provided for resource-constrained embedded systems." So I
take this as the standing rationale for keeping Forth stripped down.

This remark has since been removed. I don't know why. Nothing has changed.

-Doug

[Alex McDonald]

on 27/07/2015 11:02:41, anton wrote:

> In my experience, they would not help at all. What I missed from
> standard Forth in this area is:
>
> * A variant of WORDLIST that does not fold case, in order to be able
> to use arbitrary data as keys. Gforth provides that with TABLE.

Yes, this is very useful. I have

wordlist case-asis constant wid

CASE-ASIS modifies WID to be case insensitive. WID is actually an object
with 3 predefined & replaceable methods (create, find and iterate-over).

>
> * A good interface to insert programmatically-generated keys into
> WORDLISTs or TABLEs. Gforth provides NEXTNAME, or one could use
> EXECUTE-PARSING, but both are not really satisfactory.
>
> What I have not missed is the capability to delete specific items from
> a wordlist/table.

A variant of HIDE would suffice?

[foxaudio...@gmail.com]

The only exception to your rational that I can see, is if the OP chose to EXTEND Forth to do those kind of operations by building a custom forth-like language for the purpose, which can be done, but one could ask why?

Possible reasons

1. Forth might run faster than the alternatives
2. Forth can interpret a file almost as fast as it can load from disk
3. Adding some unique features to a commercial product that have value to a customer set
4. Creating high level features that make admin easier for operators
5. a domain specific language is in the specifications
6. Because they can
7. To prove a point to their boss
8. Your reason goes here...

[Alex McDonald]

on 27/07/2015 12:52:50, wrote:
> On 7/27/15 3:36 AM, Anton Ertl wrote:
>> "Alex McDonald" writes:
>>> on 26/07/2015 19:18:20, Paul Rubin wrote:
>>>> "Alex McDonald" writes:
>>>>> The definition of a 1 dimensional array is trivial; perhaps
>>>>> that's why. : 1dim create cells allot does> swap cells + ;
>>>>
>>>> You're going to use arrays with no bounds checks to script
>>>> potentially hostile internet devices? Sounds dangerous to me.
>>>>
>>>>
>>>
>>> That wasn't the original complaint; it was that there was not even
>>> a cell sized array in Forth.
>>>
>>> : 1dim create dup , cells allot does> 2dup @ 0 within abort" index
>>> out of bounds" swap 1+ cells + ;
>>
>> This demonstrates why there is no built-in array word in Forth.
>
> Does it? Not sure what harm providing Alex's first-try version as
> built in would cause. It's certainly a very common use case. It would
> not not prevent one from creating different versions if desired.

Why should the language standard provide one use case and not others?

>
> As it is newbies to Forth must first learn the create ... does> idiom,
> which is definitely not trivial in my opinion, before they can do
> something as common as using an array the way Alex shows which I think
> is a good one. Or they must search the various libraries.

Which is why Hugh Aguilar wrote his "novice" package, which provides zero
based, specifiable size element, arrays with up to 6 (!!!) possible
dimensions. (Why 6?)

If that helps you, then I suspect that you're never going to get very far
as a competent Forth programmer. Providing such a set of arrays IMHO
demonstrates the futility of trying to cover every eventuality, a goal it
fails to achieve.

What is true, and Rod Pemberton showed this in his response upthread, is
a misunderstanding of how CREATE DOES> works. But contrast that with
understanding how to use it successfully. Using CREATE DOES> is pretty
simple and straightforward, as in this example, and I don't understand
why you might think it so hard.

[Alex McDonald]

on 27/07/2015 13:57:04, foxaudio wrote:

To what or whom are you replying? You've trimmed away the entire context.

[glid...@gmail.com]

On 7/26/15 3:25 PM, Alex McDonald wrote:
> on 26/07/2015 19:18:20, Paul Rubin wrote:
>> "Alex McDonald" writes:
>>> The definition of a 1 dimensional array is trivial; perhaps
>>> that's why. : 1dim create cells allot does> swap cells + ;
>>
>> You're going to use arrays with no bounds checks to script
>> potentially hostile internet devices? Sounds dangerous to me.
>
> That wasn't the original complaint; it was that there was not even a
> cell sized array in Forth.

No, the original complaint was from Mark Wills, partial quote follows:

"Forth is next to useless for this sort of thing, unless you have a rack
of libraries that you have personally built up over the years. Because,
you know, this is Forth, right, and we don't use each other's libraries
in the Forth world, because everyone else's code sucks, right? We can
*all* do *much* better jobs by ourselves, re-inventing the wheel over
and over and over again."

I simply commented that I could understand his lament and chose the
array as just one tiny example, perhaps not the best choice but
sufficient I think. Thanks for the follow-up thoughts though.

-Doug

[glid...@gmail.com]

On 7/27/15 9:01 AM, Alex McDonald wrote:> on 27/07/2015 12:52:50, wrote:
>> On 7/27/15 3:36 AM, Anton Ertl wrote:
>>> "Alex McDonald" writes:
>>>> on 26/07/2015 19:18:20, Paul Rubin wrote:
>>>>> "Alex McDonald" writes:
>>>>>> The definition of a 1 dimensional array is trivial; perhaps
>>>>>> that's why. : 1dim create cells allot does> swap cells + ;
>>>>>
>>>>> You're going to use arrays with no bounds checks to script
>>>>> potentially hostile internet devices? Sounds dangerous to me.
>>>>>
>>>>>
>>>>
>>>> That wasn't the original complaint; it was that there was not even
>>>> a cell sized array in Forth.
>>>>
>>>> : 1dim create dup , cells allot does> 2dup @ 0 within abort" index
>>>> out of bounds" swap 1+ cells + ;
>>>
>>> This demonstrates why there is no built-in array word in Forth.
>>
>> Does it? Not sure what harm providing Alex's first-try version as
>> built in would cause. It's certainly a very common use case. It would
>> not not prevent one from creating different versions if desired.
>
> Why should the language standard provide one use case and not others?

Because of the part you snipped. To keep the wordset as small as
possible to primarily support embedded programming. It also explains why
not even the one (most?)common array use case is provided. I accept all
that. Doesn' explain the existence of words like cell+, cells, 1+, and
many other truly trivial words, which I find somewhat baffling.

-Doug

[Alex McDonald]

on 27/07/2015 14:16:56, wrote:
> On 7/27/15 9:01 AM, Alex McDonald wrote:> on 27/07/2015 12:52:50,
> > wrote:
>>> On 7/27/15 3:36 AM, Anton Ertl wrote:

> > wrote:
>>>> "Alex McDonald" writes:

> > wrote:
>>>>> on 26/07/2015 19:18:20, Paul Rubin wrote:

> > wrote:
>>>>>> "Alex McDonald" writes:

> > wrote:
>>>>>>> The definition of a 1 dimensional array is trivial; perhaps

> > wrote:
>>>>>>> that's why. : 1dim create cells allot does> swap cells + ;

> > wrote:

>>>>>>
> > wrote:
>>>>>> You're going to use arrays with no bounds checks to script

> > wrote:
>>>>>> potentially hostile internet devices? Sounds dangerous to me.

> > wrote:
>>>>>>
> > wrote:
>>>>>>
> > wrote:

>>>>>
> > wrote:
>>>>> That wasn't the original complaint; it was that there was not even

> > wrote:
>>>>> a cell sized array in Forth.

> > wrote:

>>>>>
> > wrote:
>>>>> : 1dim create dup , cells allot does> 2dup @ 0 within abort" index

> > wrote:
>>>>> out of bounds" swap 1+ cells + ;

> > wrote:

>>>>
> > wrote:
>>>> This demonstrates why there is no built-in array word in Forth.

> > wrote:

>>>
> > wrote:
>>> Does it? Not sure what harm providing Alex's first-try version as

> > wrote:
>>> built in would cause. It's certainly a very common use case. It would

> > wrote:
>>> not not prevent one from creating different versions if desired.

> > wrote:

>>
> > wrote:
>> Why should the language standard provide one use case and not others?

> wrote:
>
> Because of the part you snipped. To keep the wordset as small as
> possible to primarily support embedded programming. It also explains
> why not even the one (most?)common array use case is provided. I
> accept all that. Doesn' explain the existence of words like cell+,
> cells, 1+, and many other truly trivial words, which I find somewhat
> baffling.
>
> -Doug
>

The optional wordset may be large, but the core wordset is still very
small. One reason that CELL+ 1+ etc make it is due to (most not all)
processor architectures where such operations can be single instructions.
Most Forth systems, especially DTC and ITC, don't provide optimisations
to make 1 + an INCrement for instance. And the baggage of history.

There are no array words because no-one since Forth-79 some 46 years ago
has ever seen the need to standardise them. Provide them, sure; and many
have done so.

[Alex McDonald]

46? Time flies, but not that fast. 36 was meant.

[Anton Ertl]

glid...@gmail.com writes:
>On 7/27/15 3:36 AM, Anton Ertl wrote:
>> This demonstrates why there is no built-in array word in Forth.
>
>Does it? Not sure what harm providing Alex's first-try version as built
>in would cause. It's certainly a very common use case.

On what statistics do you base this claim on? I don't use it, and I
don't remember coming across it in any advanced code I have seen from
others. To me it seems to be a common textbook example that is too
limited for general usage.

The harm in standardizing it is that it would make beginners think
that this is the way to go (of course its use in textbooks may have a
similar effect, even if the textbook explains that the example is used
to demonstrate CREATE...DOES>, not arrays). It also would bloat the
standard with another little-used definition.

>As it is newbies to Forth must first learn the create ... does> idiom,
>which is definitely not trivial in my opinion, before they can do
>something as common as using an array the way Alex shows which I think
>is a good one.

It's not a good one, because it's only benefit is if you want to write
words that access one particular array. As soon as you want to write
words that can process several aways, it's at best useless. So
newbies don't need to learn create...does> before they can use arrays.

>It took me awhile to realize that the Forth standard, by its own
>explicit declaration, is constructed to minimize the word set
>to primarily support embedded programming.
>From the Forth standard 29 Feb 2012
>"A.1.2 Scope
>The result of the effort to satisfy all of these objectives is a
>Standard arranged so that the required word set remains small. Thus ANS
>Forth can be provided for resource-constrained embedded systems." So I
>take this as the standing rationale for keeping Forth stripped down.
>
>This remark has since been removed. I don't know why. Nothing has changed.

I don't remember our discussions concerning this paragraph. However,
my point of view is that Forth-94 has failed at the objective of being
a standard for resource-constrained embedded systems. People writing
Forth systems and particularly Forth programs for resource-constrained
embedded systems tend to deviate from Forth-94 and Forth-2012 in
various ways (e.g., by using a cross-compiled system, which is not
covered in the standard, by leaving out some required words or having
them behave in non-standard ways (e.g., create...does> on flash
systems), and for programs, by using non-standard features.

OTOH, the goal of supporting such systems did influence our decisions.
E.g., it would not be easy to standardize a word (other than ALLOCATE
and RESIZE themselves) that ALLOCATEs memory. Instead, we standardize
interfaces where the caller has to provide the buffer, and then check
whether the result fits, and maybe repeat it after resizing the buffer
(i.e., totally horrible interfaces).

[Anton Ertl]

"Alex McDonald" <bl...@rivadpm.com> writes:
>on 27/07/2015 11:02:41, anton wrote:
>
>> In my experience, they would not help at all. What I missed from
>> standard Forth in this area is:
>>
>> * A variant of WORDLIST that does not fold case, in order to be able
>> to use arbitrary data as keys. Gforth provides that with TABLE.
>
>Yes, this is very useful. I have
>
>wordlist case-asis constant wid
>
>CASE-ASIS modifies WID to be case insensitive.

Cute!

>> What I have not missed is the capability to delete specific items from
>> a wordlist/table.
>
>A variant of HIDE would suffice?

As I said, I have not missed it. So "nothing" suffices.

[Julian Fondren]

On Monday, July 27, 2015 at 3:44:16 AM UTC-5, Mark Wills wrote:
> If I wanted to:
>
> * Rotate error logs in a directory (say, keep the last three)
> * Log on to a Cisco router
> * Download the day's error logs and copy to the directory
> * FTP the new error log file9s) to an off-site backup server
> * Dynamically generate a clean-up script to clean the backup server
> * FTP that file to the backup server
> * Do a CHMOD on the FTPd backup file via a Unix shell to give it
> execute permissions
> * Execute the cleanup script

* sling files around
* invoke other programs
* do something dubious to the backup server

> * Open the logs and analyse for particular occurences of error strings
> * When found, read the specific error information, and update an SQL
> database with the stats
> * Execute an SQL stored procedure to insert a record into a database
> to indicate that the script(s) ran/didn't run with/without errors
> * Send an email to sysadmin indicating that the scripts ran with/without
> errors
>
> then I sure as hell wouldn't ever think of Forth in a million years.

That's too bad. I thought of Forth as soon as you got to

* log analysis

I'm sure you'll say that log analysis will be I/O bound and therefore
not benefit from Forth. I had a fellow say to me that a task was I/O
bound and that he couldn't help his script doing it in literally
1/10,000th the time of the script that his was intended to replace.

The rest of that is just 'invoke other programs'. Considering that
Perl scripts by sysadmins also tend to have bash oneliners in them, and
that none of these examples even require reading the output of said
commands, it seems like Forth already has what it needs for these
tasks.

s" logrotated -f /etc/logrotate.d/mysqlerr" system drop
s" rsync -av --ignore-existing logserver:error-logs/ error-logs/"
system drop

> The above is typical of many office-grown back-office admin scripts
> written in their millions by millions of sys-admins the world over
> every day of every week.

There are more than four types of scripts, though. Let's suppose that
you have a bunch of users on your servers, and that these users
sometimes manage to have thirteen million files in their home
directories. This being inconvenient for filesystem-wide operations,
as well as often completely unintended, you'd like to generate a report
on the location of the files in this user's directory so that you can
give the user himself some time to clean it up.

These are such numbers, by the way, where one avoids the use of 'ls' to
look at directory contents. So, would you like to use a
filesystem-walking function in a high-level language that first builds
and then sorts a list of thirteen million files (all in one directory,
of course; the entire rest of the user's files in other directories do
not add up to a fifty thousand), after running internally through a C
opendir/readir loop, which is itself an abstraction over the getdents
syscall?

Sure, why not? Filesystem-walking is I/O bound, after all :-p

> None of them will be written in Forth.

You made that giant wall of four repeated things that Forth can't do,
which it can do with a single word that's built into probably all
hosted Forth systems. You now present this thunderous conclusion that
absolutely everybody reading you will find it easier to attribute to
Forth not being as popular or as well known as those languages. If
Forth caught on as a fad among system administrators, the scripts would
appear, using a library providing file-slinging conveniences. These
things happen. Or do you think that Javascript is a naturally
brilliant server-side scripting language?

You should search for 'Passanti' and see how it's done.


-- Julian

[foxaudio...@gmail.com]

Sorry. Using the iPad at the breakfast table... doesn't quote the post I replied to. It was Mark's post regarding

"If I wanted to:

* Rotate error logs in a directory (say, keep the last three)
* Log on to a Cisco router
* Download the day's error logs and copy to the directory
* FTP the new error log file9s) to an off-site backup server
* Dynamically generate a clean-up script to clean the backup server
* FTP that file to the backup server
* Do a CHMOD on the FTPd backup file via a Unix shell to give it
execute permissions
* Execute the cleanup script
* Open the logs and analyse for particular occurences of error strings
* When found, read the specific error information, and update an SQL
database with the stats
* Execute an SQL stored procedure to insert a record into a database
to indicate that the script(s) ran/didn't run with/without errors
* Send an email to sysadmin indicating that the scripts ran with/without
errors"

BF

[glid...@gmail.com]

On 7/27/15 9:50 AM, Anton Ertl wrote:
> glid...@gmail.com writes:
>> On 7/27/15 3:36 AM, Anton Ertl wrote:
>>> This demonstrates why there is no built-in array word in Forth.
>>
>> Does it? Not sure what harm providing Alex's first-try version as
>> built in would cause. It's certainly a very common use case.
>
> On what statistics do you base this claim on?

None. Pulled it out of thin air. Guilty.
Though I know I've coded it that way before. I see a byte array in
Starting Forth coded that way. Alex did it. VFX provides an array
defined that way (with some VFXisms added). But point taken.

> I don't use it, and I don't remember coming across it in any advanced
> code I have seen from others. To me it seems to be a common textbook
> example that is too limited for general usage.
>
> The harm in standardizing it is that it would make beginners think
> that this is the way to go (of course its use in textbooks may have
> a similar effect, even if the textbook explains that the example is
> used to demonstrate CREATE...DOES>, not arrays). It also would bloat
> the standard with another little-used definition.

Yes. The minimal word set thing I've acknowledged as part of Forth's
(more or less official) position as an embedded-use language. I'll take
your "little-used definition" assertion on good faith.

Supporting embedded systems primarily. Clearly.

-Doug

[Paul Rubin]

"Alex McDonald" <bl...@rivadpm.com> writes:
> The optional wordset may be large, but the core wordset is still very
> small.

I was surprised to notice that the core wordset contains quite a bit of
what I'd consider to be bloat, e.g. words for fancy output formatting
that typical embedded applications would have no way to use (no text
output device!), and that for interactive testing are at best only
slightly useful.

[hughag...@gmail.com]

On Sunday, July 26, 2015 at 10:29:59 PM UTC-7, Rod Pemberton wrote:
> C captures the essence of all generic microprocessors. Forth,
> instead, usually attempts to impose it's own virtual machine.
> So, in that regard, it would actually suit Forth to be more
> C like. Even Hugh is having difficulty adding functionality to
> Forth without recreating C functionality. He's done so twice
> with projects even though he riles against C-like memory allocation
> for Forth. Anton doesn't even bother attempting to hide from C.

Nothing that Rod says is true --- he should be ignored.

My FMITE processor and the associated LOTD language have quotations that have access to the parent function's local variables --- I am not "recreating C functionality" because C doesn't have this functionality --- the FMITE is intended to obsolete both C and ANS-Forth in the micro-controller world as both C and ANS-Forth are lacking quotations. Also, I didn't have any difficulty with designing the FMITE --- it was easy --- mostly just a matter of taking the time to do it.

Another thing: most of the micro-processors of 20-30 years ago were CISC and had instructions provided specifically to support C (the 65c816 and MC6808 both had indexing addressing into the return-stack, for example) --- they weren't "generic microprocessors" but were in fact C-engines (as compared to processors such as the MiniForth or the Novix that were Forth-engines) --- nowadays we have RISC processors such as the ARM that are much more generic in design, and hence are reasonable targets for C or Forth or any other language.

I think that it is uncool that people should talk about me in threads that I'm not involved in, especially if they make up stuff that isn't true. I wasn't even going to read this thread because I agree with Mark that Forth is a bad choice as a scripting language (the MiniForth was used in motion-control of a laser-etcher and competed well against the competitor's 68K system written in C, which is a lot different than trying to compete against Python as described in this thread) --- I only read the thread because I was bored, but then I find the usual cast of idiots talking about me and saying that I'm "recreating C functionality" (that's not true; I compared LOTD to C in my document showing how different they were; if anything, I'm recreating Factor functionality).

[Elizabeth D. Rather]

Don't forget, the core wordset doesn't have to be present in a kernel or
running system. In fact, it is explicitly ok to provide words as
loadable options or even on paper. Sure, an embedded system without
output wouldn't use a lot of things. But programmers want some general
level of facilities they can count on being able to find if they need them.

[rickman]

On 7/27/2015 2:02 PM, hughag...@gmail.com wrote:
>
> I think that it is uncool that people should talk about me in threads
> that I'm not involved in

Beetlejuice, Beetlejuice, Beetlejuice! ;-)

--

Rick

[hughag...@gmail.com]

I have been saying since 2009 that Forth-200x is doomed to failure because it is a single language-standard that is supposed to work on hugely different targets. A better plan would be to have two versions of Forth, and both standardized --- one for micro-controllers and one for desktop-computers.

The micro-controller Forth would always be cross-compiled (the cross-compiler would be written in the desktop-computer Forth). It would be quite minimal compared to the grossly bloated ANS-Forth standard. It would work with different cell sizes (16-bit or 32-bit typically).

The desktop-computer Forth would be 64-bit and would have a lot more support for text and numerical programming. It would also be designed specifically to support a cross-compiler though; it would have features not available in Forth-200x (such as being able to examine an xt and determine what word-list it is in and whether it is immediate or not).

Realistically, you can't have one language standard that works on both the 64-bit x86 and the 16-bit MSP-430 --- you are going to be a "jack of all trades and a master of none" --- this is the path to failure that Forth-200x is striding down, following closely on ANS-Forth's heals.

[Paul Rubin]

"Elizabeth D. Rather" <era...@forth.com> writes:
> Don't forget, the core wordset doesn't have to be present in a kernel
> or running system. In fact, it is explicitly ok to provide words as
> loadable options or even on paper.

I would have thought "loadable options" was another way to say
"optional" and the organization of the standard was that words that
weren't important in a minimal core would be put in optional word sets.

> Sure, an embedded system without output wouldn't use a lot of
> things. But programmers want some general level of facilities they can
> count on being able to find if they need them.

Sure, but given typical Forth programs, I'd have thought that the
formatted output stuff was relatively deluxe and belonged in an optional
set. By contrast I'd keep KEY, EMIT, and friends in the core set, even
though those same embedded systems couldn't use them.

[Elizabeth D. Rather]

On 7/27/15 8:24 AM, Paul Rubin wrote:
> "Elizabeth D. Rather" <era...@forth.com> writes:
>> Don't forget, the core wordset doesn't have to be present in a kernel
>> or running system. In fact, it is explicitly ok to provide words as
>> loadable options or even on paper.
>
> I would have thought "loadable options" was another way to say
> "optional" and the organization of the standard was that words that
> weren't important in a minimal core would be put in optional word sets.

No, that is not the intent. Optional wordsets don't have to be provided
in any form. Core words can be provided in many forms, not necessarily
in a kernel or always resident.

>> Sure, an embedded system without output wouldn't use a lot of
>> things. But programmers want some general level of facilities they can
>> count on being able to find if they need them.
>
> Sure, but given typical Forth programs, I'd have thought that the
> formatted output stuff was relatively deluxe and belonged in an optional
> set. By contrast I'd keep KEY, EMIT, and friends in the core set, even
> though those same embedded systems couldn't use them.
>

The Core Wordset is the minimal expectation for a programmable system,
and includes a lot of things that might be useful in the programming
process but not necessarily in a final application. It also includes,
for example, words that would be difficult or inefficient to add without
"carnal knowledge".

[Alex McDonald]

on 27/07/2015 19:16:19, wrote:
> On Monday, July 27, 2015 at 10:47:29 AM UTC-7, Paul Rubin wrote:
>> "Alex McDonald" <bl...@rivadpm.com> writes:
>> > The optional wordset may be large, but the core wordset is still very
>> > small.
>>
>> I was surprised to notice that the core wordset contains quite a bit of
>> what I'd consider to be bloat, e.g. words for fancy output formatting
>> that typical embedded applications would have no way to use (no text
>> output device!), and that for interactive testing are at best only
>> slightly useful.
>
> I have been saying since 2009 that Forth-200x is doomed to failure

It's 2015 and it still hasn't failed (and failure of the standard needs
more tightly defined than "Hugh doesn't like it").

> because it is a single language-standard that is supposed to work on

> hugely differe nt targets. A better plan would be to have two versions

> of Forth, and both standardized --- one for micro-controllers and one
> for desktop-computers.
>
> The micro-controller Forth would always be cross-compiled (the

> cross-compil er would be written in the desktop-computer Forth). It
> would be quite minim al compared to the grossly bloated ANS-Forth
> standard. It would work with d ifferent cell sizes (16-bit or 32-bit

> typically).
>
> The desktop-computer Forth would be 64-bit and would have a lot more

> suppor t for text and numerical programming. It would also be designed
> specificall y to support a cross-compiler though; it would have

> features not available in Forth-200x (such as being able to examine an

> xt and determine what word- list it is in and whether it is immediate

> or not).
>
> Realistically, you can't have one language standard that works on both
> the 64-bit x86 and the 16-bit MSP-430 --- you are going to be a "jack

> of all tr ades and a master of none" --- this is the path to failure

> that Forth-200x is striding down, following closely on ANS-Forth's
> heals.
>

I've not seen here any demand to engineer solutions based on this dual
standard agenda. Is anyone interested in this as a program?

[Alex McDonald]

on 27/07/2015 19:02:21, hugher wrote:
> I think that it is uncool that people should talk about me in threads
> that I'm not involved in, especially if they make up stuff that isn't
> true.

Jeez, Hugh, John Passiniti hasn't posted here in years, yet you still
keep calling out to him.

[Paul Rubin]

"Elizabeth D. Rather" <era...@forth.com> writes:
> No, that is not the intent. Optional wordsets don't have to be
> provided in any form. Core words can be provided in many forms, not
> necessarily in a kernel or always resident.

Aha, thanks for pointing out that distinction.

> The Core Wordset is the minimal expectation for a programmable system,
> and includes a lot of things that might be useful in the programming
> process but not necessarily in a final application. It also includes,
> for example, words that would be difficult or inefficient to add
> without "carnal knowledge".

If the criterion is "might be useful" that covers the kitchen sink!
I'd expect the formatted output stuff is straightforward to implement in
portable Forth, and not likely that important in the programming process
given the availability of dot, dot-quote, TYPE, and so on.

[Julian Fondren]

On Monday, July 27, 2015 at 2:49:16 PM UTC-5, Paul Rubin wrote:
> If the criterion is "might be useful" that covers the kitchen sink!

If the ridiculous were the case, then things would be
ridiculous.

> I'd expect the formatted output stuff is straightforward to implement in
> portable Forth, and not likely that important in the programming process
> given the availability of dot, dot-quote, TYPE, and so on.

OK, next time my time machine is available, I'll hop into it
and try get the Forth-79 people to not include pictured numeric
output. I had "settle WWI in favor of Germany" next on my
list of things to do, but this is probably a better cause.


Speaking of Forth-79, it has an interesting section that
included 'uncontrolled word definitions', "included for public
reference of words that have present usage and/or are
candidates for future standardization".

Interesting as these words seem to be pretty obscure.

!BITS ( n1 addr n2 -- )
Store the value of n1 masked by n2 into the equivalent
masked part of the contents of addr, without affecting bits
outside the mask.

+BLOCK ( n1 -- n2 )
Leave the sum of n1 plus the number of the block being
interpreted.

-' ( -- addr 0 | -1 )
Leave the paraeter field of <name> beneath zero (false) if
the name can be found in the CONTEXT vocabulary; leave only
true if not found.

COM ( n1 -- n2 )
Leave the one's complement of n1.

And there several words that have new names.

IFTRUE ... OTHERWISE ... IFEND f.e. works similarly to
[IF] ... [ELSE] ... [THEN]

REMEMBER is basically MARKER

SET ( n addr -- )
Defines a word <name> which, when executed, will cause the
value n to be stored at address.

WHERE
Output information about the status of FORTH, (e.g., after
an error abort). Indicate at least the last word compiled
and the last word accessed.


-- Julian

[Elizabeth D. Rather]

Yes, well, the choice was a "negotiated settlement" between advocates of
radical minmalism and "kitchen sink" enthusiasts! In the case of the
formatted output stuff, though, it would be really hard to know how to
implement words like # #S HOLD etc. without knowing where the numeric
output string was being constructed and how to manage the process.

[Rod Pemberton]

On Mon, 27 Jul 2015 04:19:50 -0400, Elizabeth D. Rather <era...@forth.com> wrote:

> On 7/26/15 7:30 PM, Rod Pemberton wrote:
>> On Sun, 26 Jul 2015 22:49:50 -0400, Elizabeth D. Rather
>> <era...@forth.com> wrote:

>>> The longer the phrase, the fewer instances it takes to justify
>>> being factored. This particular phrase is very short,
>>
>> Yes. It's very short just like @ and ! are very short.
>> They were factored. Are you saying they shouldn't have been?
>
> Yes. The difference is that @ and ! are ubiquitous, whereas this
> particular, very specific indexing sequence would normally be
> incorporated in the DOES> portion of a defining word, and never used
> independently.

So, it would never be used with PAD or to display a BIOS string
which has no defining word for it's address? ... The user can
enter an address directly on the stack, then use [] to index whatever
is there. The big problem with Forth is there is no ability to know
the size of the data the pointer points to. I.e., you'd need []
for cell size data and c[] for character sized data.

>>> quite unlikely to be repeated beyond the use here,
>>
>> It can be used from the interpreted state with any address.
>> I.e., it's a generic addressing operation like @ ! +! TOGGLE
>> etc. So, why exactly would it be unlikely to be used? ...
>
> It wouldn't be used independent of the array defining word because
> there is no need for it except in that context. It has little value
> as an independent facility.

C is a pointer based language where every object in C maps onto
an array. It uses this functionality all the time for everything.
The ANS Forth standard requires the same inherent layout. And,
Forth deals with raw addresses too. I.e., I think you're strongly
down-playing the usefulness here.


Rod Pemberton

--
Scientists now say we'll experience a mini-ice in 2030.
So, I guess we need more global warming today ...

[Rod Pemberton]

On Mon, 27 Jul 2015 14:02:23 -0400, <hughag...@gmail.com> wrote:
> On Sunday, July 26, 2015 at 10:29:59 PM UTC-7, Rod Pemberton wrote:

>> C captures the essence of all generic microprocessors. Forth,
>> instead, usually attempts to impose it's own virtual machine.
>> So, in that regard, it would actually suit Forth to be more
>> C like. Even Hugh is having difficulty adding functionality to
>> Forth without recreating C functionality. He's done so twice
>> with projects even though he riles against C-like memory allocation
>> for Forth. Anton doesn't even bother attempting to hide from C.
>
> Nothing that Rod says is true --- he should be ignored.

Everything I say is true.

> My FMITE processor and the associated LOTD language have quotations
> that have access to the parent function's local variables --- I am
> not "recreating C functionality" because C doesn't have this
> functionality --- the FMITE is intended to obsolete both C and
> ANS-Forth in the micro-controller world as both C and ANS-Forth
> are lacking quotations. Also, I didn't have any difficulty with
> designing the FMITE --- it was easy --- mostly just a matter of
> taking the time to do it.

I have no idea what FMITE is ... Clearly, I wasn't criticizing it.

> I think that it is uncool that people should talk about me in
> threads that I'm not involved in,

So, you never mentioned me in a thread I wasn't involved in?
Whatever ...

> saying that I'm "recreating C functionality" (that's not true;

It is. You are. I've pointed it out a few times now.
Simply denying this doesn't change reality.

[Rod Pemberton]

On Mon, 27 Jul 2015 15:17:25 -0400, Alex McDonald <bl...@rivadpm.com> wrote:
> on 27/07/2015 19:02:21, hugher wrote:

>> I think that it is uncool that people should talk about me in threads
>> that I'm not involved in, especially if they make up stuff that isn't
>> true.
>
> Jeez, Hugh, John Passiniti hasn't posted here in years,

It's true he hasn't posted under his own name, but
it's probably not true that he's not posting here.
See Alex Wegel and/or WJ.

> yet you still keep calling out to him.

Well, you're one to talk. You keep mentioning me in
threads I'm not involved in and usually haven't read.

[Julian Fondren]

On Monday, July 27, 2015 at 6:31:14 PM UTC-5, Rod Pemberton wrote:
> The big problem with Forth is there is no ability to know
> the size of the data the pointer points to. I.e., you'd need []
> for cell size data and c[] for character sized data.

The big problem with C is that you have to know the size of the data
that a pointer points to. You can't just say [] and get it. You
have to say what size it is when you get it:

void example (float *fs) {

and you have to say what size variable you're fetching it to:

float c = *fs;

Just look at that last example again, next to the Forth:

float c = *fs;
fs f@

Isn't it just awful, that one of these examples requires you to know
*so* much more than the other does? Why can't that one language be
as smart, on your behalf, as the other?

-- Julian

[Rod Pemberton]

On Mon, 27 Jul 2015 19:44:26 -0400, Julian Fondren <julian....@gmail.com> wrote:
> On Monday, July 27, 2015 at 6:31:14 PM UTC-5, Rod Pemberton wrote:

>> The big problem with Forth is there is no ability to know
>> the size of the data the pointer points to. I.e., you'd need []
>> for cell size data and c[] for character sized data.
>
> The big problem with C is that you have to know the size of the
> data that a pointer points to.

No, you don't.

The subscript operator [] in C works with the size of the object the
pointer points to, as long as it's not a void pointer. For a[b]
either a or b must be a pointer and the other must be an offset or
index. If b is the offset, then a[b] is identical to:

*(*a)((a)+((b)x(sizeof(*a))))

That's index b multiplied by the size of the object the pointer
a points to, i.e., scale = index x size, plus the pointer a, the
sum of which is type cast to the same type as pointer a, then it's
dereferenced to fetch the appropriately sized value.

Please note that the index or offset is automatically scaled by the
size of the appropriate type.

This is usually represented in C standards in simplied form without
the scaling shown:

E1[E2] equivalent to *((E1)+(E2))

Although the subscript operator [] has the appearance of an array
index, it doesn't have to be used with just declared arrays. It
can be used with any pointer in C or any object type which "decays
into" a pointer such as declared arrays and parameters to functions.
The is also no specification on which element must be the offset
and which must be the pointer. E.g.,

/* c is set to the char at index 1, i.e., 'S' */
/* c indexing starts at zero */
/* "ASDF" is not a character constant but a name for a pointer to char */
/* ASDF\0 is the actual character constant which is stored elsewhere */

c = "ASDF"[1] /* pointer_to_char[index] */

> You can't just say [] and get it.

What? Sure you can.

int myint, myintarray*,myintsecond[5]={1,2,3,4,5};
char mychar, mychararray*,mycharsecond[]="Hello";

mychar=mycharsecond[3];
myint=myintsecond[4];

mychar will be set to the 4th char in the array "mychararray".
myint will be set to the 5th int in the array "myintarray".

You can do the same with pointers instead of arrays.

myintarray= ... ; /* assign pointer to something */
mychararray= ... ;

myint=myintarray[5];
mychar=mychararray[3];

I.e., the index is scaled by the appropriate object size
for both arrays and pointers. The C subscript operater
[] "knows" nothing about arrays. Except for declarations,
C doesn't actually have arrays. The behavior of arrays
is emulated purely by the subscript operator and the
addresses C uses to identify objects.

> You have to say what size it is when you get it:
>
> void example (float *fs) {
>
> and you have to say what size variable you're fetching it to:
>
> float c = *fs;

No, you don't. At least, not where you placed "float".
"float" would normally be used where "c" is declared, not
where it is assigned, unless you're using C++ or a C99 feature.
I.e., normal C code to dereference a pointer to an object:

float c; /* declare c to be type float */

/* bunch of code ... */

c=*fs; /* dereference fs to fetch float, store in c */

> Just look at that last example again, next to the Forth:
>
> float c = *fs;
> fs f@
>
> Isn't it just awful, that one of these examples requires you to know
> *so* much more than the other does? Why can't that one language be
> as smart, on your behalf, as the other?

I'm not sure about the use of floats in C, as I don't use them much.
But, you don't have to do what you did that way in C for other C
object types, structs, or unions. You can if you want though ...

[humptydumpty]

On Sunday, July 26, 2015 at 2:01:26 PM UTC+3, glid...@gmail.com wrote:
> On 7/25/15 10:20 AM, rickman wrote:
> > On 7/25/2015 4:28 AM, Mark Wills wrote:
> >> On Saturday, 25 July 2015 01:52:11 UTC+1, rickman wrote:
> >>> On 7/24/2015 8:00 PM, Don Y wrote:
> >>>> On 7/24/2015 4:06 PM, WJ wrote:
> >>>>
> >>>>> It's obvious that the reason that he is recommending Forth
> >>>>> over all other languages is that he knows no language that is
> >>>>> higher-level than Forth. And he finds it too difficult to
> >>>>> learn another language.
> >>>>>
> >>>>> If a hammer is the only tool that you have, then you use a
> >>>>> hammer for everything.
> >>>>>
> >>>>> How would these tasks be done with a hammer?
> >>>>>
> >>>>> Using Gauche Scheme:
> >>>>>
> >>>>> Write a function that accepts a vector (1-dimensional array)
> >>>>> of 16-bit numbers and replaces each number with its
> >>>>> difference from the average.
> >>>>>
> >>>>> (use gauche.uvector) (use gauche.collection)
> >>>>>
> >>>>> (define (relativize vec) (let ((avg (round->exact (/ (fold +
> >>>>> 0 vec) (s16vector-length vec))))) (s16vector-sub! vec avg)))
> >>>>>
> >>>>> (define my-vec (s16vector 0 1 2 3 4 5 6 7 8 9)) (relativize
> >>>>> my-vec) my-vec ===> #s16(-4 -3 -2 -1 0 1 2 3 4 5)
> >>>>>
> >>>>>
> >>>>> Take a string that contains numbers separated by semicolons
> >>>>> or commas and convert it to a list of numbers.
> >>>>>
> >>>>> (map string->number (string-split "2,3.14;5.8,9" #/[,;]/))
> >>>>> ===> (2 3.14 5.8 9)
> >>>>
> >>>> What role would these have in the scripting of an automation
> >>>> system? IME, tasks in that domain tend to be more procedural
> >>>> oriented.
> >>>
> >>> WJ is an odd sort. You won't find that he is much of a
> >>> conversationalist. Rather he communicates in terms of code for
> >>> any manner of interesting, even if irrelevant, examples.
> >>>
> >>> Just one of the many odd birds flitting about in the Forth
> >>> newsgroup.
> >>>
> >>> --
> >>>
> >>> Rick
> >>
> >> He may be an odd sort, but his point is spot on. For *embedded
> >> scripting* I wouldn't go near Forth in a bazillion years. Okay, if
> >> the *embedded* machine I was using had like, 1K of RAM, then yes,
> >> Forth. Otherwise, no way.
> >>
> >> Look, if you are doing scripting, then you are doing things like:
> >>
> >> * Pinging IP addresses (e.g. a gateway) and changing routing tables
> >> on the fly depending on whether a particular interface is up or
> >> not. * Reading through log files, looking for the certain
> >> occurrence of strings (error codes/strings, other filenames etc.) *
> >> Deleting log or error files * Moving log or error files * Rotating
> >> log or error files depending on their age * Searching for files of
> >> a given type, or date range, and moving them, or zipping them, or
> >> deleting them.
> >>
> >> Forth has *no* standard way to do any of these things. You'd even
> >> have to write your own string array handler FFS. Forget enumerating
> >> a disk directory, getting a list of all the files in a structure,
> >> with their creation dates, modification dates, filtering by type,
> >> sorting by date, looking for duplicate file names (so that you can
> >> delete the older versions) etc.
> >>
> >> Are you kidding me?
> >>
> >> You want Perl, PHP, and Python. Any of those will take you
> >> everywhere you want to go.
> >>
> >> Forth is next to useless for this sort of thing, unless you have a
> >> rack of libraries that you have personally built up over the years.
> >> Because, you know, this is Forth, right, and we don't use each
> >> other's libraries in the Forth world, because everyone else's code
> >> sucks, right? We can *all* do *much* better jobs by ourselves,
> >> re-inventing the wheel over and over and over again.
> >>
> >> Using Forth for this kind of thing would be like setting out to
> >> build a house with a hammer and a screwdriver. Someone says "hey,
> >> where's the cement mixer?" and you hold up the hammer and the
> >> screwdriver and say "we're gonna build our own! It's gonna be the
> >> best cement mixer EVER!"
> >>
> >> Don't get me started! I don't even have a cat that I can kick ;-)
> >>
> >> Next-door neighbour... he has a cat...
> >
> > BTW, you only replied to the forth group. If you feel so strongly
> > that Forth is such a bad language, why are you here?
>
> Mark's lament is understandable. Forth doesn't even have a built-in
> cell-sized-element array wordset. I think that sort of thing is where
> he's coming from. But that's Forth. I long ago made peace with this
> aspect and accept Forth as it is and for what it is (and isn't).
>
> -Doug

Hi!

As no one said it, I say it:

Maybe is not needed so badly. For traversing an array (or parts) would suffice
( HI-limit start-addr ) 'DO ... +LOOP', or 'BEGIN ...' loops & '@+','!+'-like
kind of operators.

From some comments here, it looks as Forth should give away his freedom to
make a room for only one boring way to do things.

Have a nice day,
humptydumpty

[Elizabeth D. Rather]

On 7/27/15 1:31 PM, Rod Pemberton wrote:
> On Mon, 27 Jul 2015 04:19:50 -0400, Elizabeth D. Rather
> <era...@forth.com> wrote:
>
>> On 7/26/15 7:30 PM, Rod Pemberton wrote:
>>> On Sun, 26 Jul 2015 22:49:50 -0400, Elizabeth D. Rather
>>> <era...@forth.com> wrote:
>
>>>> The longer the phrase, the fewer instances it takes to justify
>>>> being factored. This particular phrase is very short,
>>>
>>> Yes. It's very short just like @ and ! are very short.
>>> They were factored. Are you saying they shouldn't have been?
>>
>> Yes. The difference is that @ and ! are ubiquitous, whereas this
>> particular, very specific indexing sequence would normally be
>> incorporated in the DOES> portion of a defining word, and never used
>> independently.
>
> So, it would never be used with PAD or to display a BIOS string
> which has no defining word for it's address? ... The user can
> enter an address directly on the stack, then use [] to index whatever
> is there. The big problem with Forth is there is no ability to know
> the size of the data the pointer points to. I.e., you'd need []
> for cell size data and c[] for character sized data.

If the indexing is incorporated in the execution portion of a defining
word, it's possible for the size of the array to be stored with it, so
reasonable bounds checking can be performed if it's appropriate for the
application. A generic word like [] cannot do this.

>>>> quite unlikely to be repeated beyond the use here,
>>>
>>> It can be used from the interpreted state with any address.
>>> I.e., it's a generic addressing operation like @ ! +! TOGGLE
>>> etc. So, why exactly would it be unlikely to be used? ...
>>
>> It wouldn't be used independent of the array defining word because
>> there is no need for it except in that context. It has little value
>> as an independent facility.
>
> C is a pointer based language where every object in C maps onto
> an array. It uses this functionality all the time for everything.
> The ANS Forth standard requires the same inherent layout. And,
> Forth deals with raw addresses too. I.e., I think you're strongly
> down-playing the usefulness here.

Well, I don't think that functionality is appropriate all the time for
everything. I prefer to write exactly the functionality I need for every
instance. Obviously that could be a pain in many languages, but if it
costs you ~1 or 2 lines of code and 30 sec. thought, it's easier than
searching for the right library.

[Elizabeth D. Rather]

On 7/27/15 1:33 PM, Rod Pemberton wrote:
> On Mon, 27 Jul 2015 15:17:25 -0400, Alex McDonald <bl...@rivadpm.com> wrote:
>> on 27/07/2015 19:02:21, hugher wrote:
>
>>> I think that it is uncool that people should talk about me in threads
>>> that I'm not involved in, especially if they make up stuff that isn't
>>> true.
>>
>> Jeez, Hugh, John Passiniti hasn't posted here in years,
>
> It's true he hasn't posted under his own name, but
> it's probably not true that he's not posting here.
> See Alex Wegel and/or WJ.

LOL, Passiniti had a well-defined particular point of view and set of
experiences on which he based his comments, which are poles apart from
those of Alex Wegel and WJ.

[glid...@gmail.com]

> Maybe is not needed so badly. For traversing an array (or parts)
> would suffice ( HI-limit start-addr ) 'DO ... +LOOP', or 'BEGIN ...'
> loops & '@+','!+'-like kind of operators.
>
> From some comments here, it looks as Forth should give away his
> freedom to make a room for only one boring way to do things.

No. If one (boring to you) way to do something is provided there is
absolutely *no* freedom taken to do it any other way one wants.
The only really valid problem would be wasted kernel space in resource
constrained environments if that way were never used, however big a
problem that is. And then only if it were made CORE.

-Doug

[Alex McDonald]

on 28/07/2015 09:05:54, "Elizabeth D. Rather" wrote:
> On 7/27/15 1:33 PM, Rod Pemberton wrote:
>> On Mon, 27 Jul 2015 15:17:25 -0400, Alex McDonald <bl...@rivadpm.com> wrote:
>>> on 27/07/2015 19:02:21, hugher wrote:
>>
>>>> I think that it is uncool that people should talk about me in threads
>>>> that I'm not involved in, especially if they make up stuff that isn't
>>>> true.
>>>
>>> Jeez, Hugh, John Passiniti hasn't posted here in years,
>>
>> It's true he hasn't posted under his own name, but
>> it's probably not true that he's not posting here.
>> See Alex Wegel and/or WJ.
>
> LOL, Passiniti had a well-defined particular point of view and set of
> experiences on which he based his comments, which are poles apart from
> those of Alex Wegel and WJ.
>
> Cheers,
> Elizabeth
>

There's tone deaf and then there's Rod.

I enjoyed Passaniti's witty style and exposition. Andrew Haley (whom I
mention even though he's not in this thread; please forgive me) wrote
thoughtful & provocative posts too, but he seems to have given up on us.

[humptydumpty]

Hi!

Sorry, I think it was a little misunderstanding from my side,
sometimes I've got tired to hear over and over again something like:
"Oh my dear, there is no native support for arrays, what a damned language!"

[glid...@gmail.com]

On 7/28/15 8:28 AM, humptydumpty wrote:
> Sorry, I think it was a little misunderstanding from my side,
> sometimes I've got tired to hear over and over again something like:
> "Oh my dear, there is no native support for arrays, what a damned language!"

Feel free to killfile me if that makes you feel better. Btw I don't use profanity.

-Doug

[humptydumpty]

No way to killfile you!

[Bernd Linsel]

> On Mon, 27 Jul 2015 19:44:26 -0400, Julian Fondren
> <julian....@gmail.com> wrote:
>> On Monday, July 27, 2015 at 6:31:14 PM UTC-5, Rod Pemberton wrote:
>
>>> The big problem with Forth is there is no ability to know
>>> the size of the data the pointer points to. I.e., you'd need []
>>> for cell size data and c[] for character sized data.
>>
>> The big problem with C is that you have to know the size of the
>> data that a pointer points to.

IN C you even may write

double foo(float* fp)
{
double z = 3.1415926536 * *fp; /* or fp[0] instead */
return z;
}

The compiler knows which size the object pointed to by fp is, you don't
have to indicate when dereferencing the pointer. THe above statement is
valid because of the implicit type promotion rules in C: A single/float
is silently promoted to double if invoked in operations with other
doubles (for a pure float expression, you should've written 3.14159f).

ACK

[Bernd Linsel]

Am 27.07.2015 um 20:02 schrieb hughag...@gmail.com:
> On Sunday, July 26, 2015 at 10:29:59 PM UTC-7, Rod Pemberton wrote:
>> C captures the essence of all generic microprocessors.

Nope. C works fine on machines with huge register sets, a hardware call
stack, and a frame pointer. You're even better off if it supports
indexed register-indirect addressing modes.

I wouldn't call that a "generic microprocessor". Look at the (vain)
efforts to implement C on the 1980's common processors (8080/5, 8086,
...). Motorolas 68K was the first part decently usable for C.

>> Forth,
>> instead, usually attempts to impose it's own virtual machine.

Forth runs perfectly on stack machines, e.g. RTX 2000. You should read
Philip J. Koopman, Jr's publications:
http://users.ece.cmu.edu/~koopman/stack_computers/

>> So,
>> in that regard, it would actually suit Forth to be more C like.
>> Even Hugh is having difficulty adding functionality to Forth
>> without recreating C functionality. He's done so twice with
>> projects even though he riles against C-like memory allocation for
>> Forth. Anton doesn't even bother attempting to hide from C.
>

> [snip]

>
> Another thing: most of the micro-processors of 20-30 years ago were
> CISC and had instructions provided specifically to support C (the
> 65c816 and MC6808 both had indexing addressing into the return-stack,
> for example) --- they weren't "generic microprocessors" but were in
> fact C-engines (as compared to processors such as the MiniForth or
> the Novix that were Forth-engines)

yep

> --- nowadays we have RISC
> processors such as the ARM that are much more generic in design, and
> hence are reasonable targets for C or Forth or any other language.

After having implemented several Forth VMs and native compilers on MIPS
and ARM embedded systems, I may remark that Forth on RISC machines tends
to be rather clumsy, regarding load delay slots, missing diplaced
indirect addressing, and so on.
You have to optimize very carefully if you want to have a stall-free
running Forth on a RISC.
(Not even to mention cache thrashing and similar effects that nowadays
also CISC processors suffer from if you're mixing code and data).

An optimizing native compiler gains of course performance of a large
register set, if you analyze the data flow on the stack and map it on
the registers, eliminating nearly all DUPs, SWAPs, DROPs etc (with the
exception that the stack/register mapping must be consistent on all
junction points).

RISC is in my opinion just muscles without the brain; raw force without
any sophistication. If directed (programmed) accurately, you may achieve
great performance -- if not, you can fail epically.

>
> I think that it is uncool that people should talk about me in threads
> that I'm not involved in, especially if they make up stuff that isn't
> true. I wasn't even going to read this thread because I agree with
> Mark that Forth is a bad choice as a scripting language (the
> MiniForth was used in motion-control of a laser-etcher and competed
> well against the competitor's 68K system written in C, which is a lot
> different than trying to compete against Python as described in this
> thread) --- I only read the thread because I was bored, but then I
> find the usual cast of idiots talking about me and saying that I'm
> "recreating C functionality" (that's not true; I compared LOTD to C
> in my document showing how different they were; if anything, I'm
> recreating Factor functionality).
>

--
Regards
Bernd

[Elizabeth D. Rather]

May I repeat once again, the fact that a word is in the CORE wordset
doesn't mean it has to be resident in a kernel. It just has to be
available in some form (even documentation, to be typed in) to users of
a Standard System. If you're programming on a resource-constrained
system, you are perfectly free to remove anything you don't think you
need from the run-time environment.

[rickman]

Some days I think newsgroups would be better if everyone were
automatically killfiled in every newsreader.

--

Rick

[Walter Banks]

On 27/07/2015 2:02 PM, hughag...@gmail.com wrote:
> (the 65c816 and MC6808 both had indexing addressing into the return-stack, for example) ---
> they weren't "generic microprocessors" but were in fact C-engines

I worked on the MC6808 ISA it was hardly a "C-engine" It didn't meet
the minimum feature set required to support C despite the fact that C
compilers (including ours) were written for it. The design goals for
the MC6808 were compatibility with 6805 and embedded systems
applications in general.

w..

[Walter Banks]

On 28/07/2015 1:22 PM, Bernd Linsel wrote:
> Am 27.07.2015 um 20:02 schrieb hughag...@gmail.com:
>> On Sunday, July 26, 2015 at 10:29:59 PM UTC-7, Rod Pemberton wrote:
>>> C captures the essence of all generic microprocessors.
>
> Nope. C works fine on machines with huge register sets, a hardware call
> stack, and a frame pointer. You're even better off if it supports
> indexed register-indirect addressing modes.
>
> I wouldn't call that a "generic microprocessor". Look at the (vain)
> efforts to implement C on the 1980's common processors (8080/5, 8086,
> ...). Motorolas 68K was the first part decently usable for C.
>

You are correct that the M68k was the first really decent platform for
C. But C can and has been implemented to create application code on a
very wide variety of target processors.

This has happened because that language itself is very flexible not
requiring stacks or very many addressing modes and still can support
reasonably effecient application implementations. We have targeted
about 30 targets mostly for embedded systems for C including COP8 a bit
serial processor, Microchip PIC a processor without a user accessible
stack with as little as 512 instruction space and 32 bytes of ram.

w..

[glid...@gmail.com]

Thanks for that, Elizabeth.

-Doug

[Rod Pemberton]

On Tue, 28 Jul 2015 13:22:48 -0400, Bernd Linsel <b...@gmx.com> wrote:

> Am 27.07.2015 um 20:02 schrieb hughag...@gmail.com:
>> On Sunday, July 26, 2015 at 10:29:59 PM UTC-7, Rod Pemberton wrote:

>>> C captures the essence of all generic microprocessors.
>

You've replied to both Hugh and I in the same post. That's likely to
incite Hugh. Well, everything incites Hugh ... It's easier to follow
if you keep replies to the last person, and reply to the other person
on their post.

>>> C captures the essence of all generic microprocessors.
>
> Nope. C works fine on machines with huge register sets, a hardware
> call stack, and a frame pointer. You're even better off if it supports
> indexed register-indirect addressing modes.

Yes. Everything an 8-bit microprocessor can do, and the general
purpose instructions of 16-bit processors, can be done directly in
C without needing to use assembly. The only thing missing is a
carry flag and an overflow flag.

While it's much easier to implement C on a machine with a hardware
stack, C was implemented on mainframes without them. That's why the
C standards don't require a stack. However, IIRC, it was Dennis
Ritchie who showed in one of his papers that a stack is the easiest
and cleanest way to implement recursion in C, which is required.

Also, C works fine on machines with few registers. C works best on
microprocessors with addressable 8-bit bytes, contiguous memory,
two's complement arithmetic, a hardware stack, the largest integer
a multiple of the 8-bit byte, and pointers the same size as the
largest integers.

Forth generally likes these too, but needs two stacks to implement
instead of one since it separates the data and control flow stack.

Integers a multiple of the size of a byte is needed because C maps
every object onto an array of bytes. Pointers the same size as the
largest integer is convenient because of C's lossless type conversions,
i.e., pointer to integer to pointer. C is difficult to implement on
machines with a 16-bit word size, machines where the smallest
addressable unit is much larger than a character in size, machines
where the address pointer is greater in size than the largest integer.
All of these situations require constructing special code, e.g., to
save extra bits for pointer conversions or to extract characters
packed into larger integers.

> I wouldn't call that a "generic microprocessor". Look at the (vain)
> efforts to implement C on the 1980's common processors (8080/5, 8086,
> ...). Motorolas 68K was the first part decently usable for C.

While it seems you're familiar with some stuff from the 1980's, it
seems you forgot about Ron Cain's SmallC. It was originally written
for the Intel 8080 and also ported to PDP-11. If it can run on 8080,
it can run on all other 8-bits. IIRC, the code emitted by the 8086
port needed only *two* registers. I'd suspect it needed a few more
on the 8080.

>>> Forth, instead, usually attempts to impose it's own virtual machine.
>
> Forth runs perfectly on stack machines, e.g. RTX 2000.

The RTX 2000 was a dedicated Forth processor. Why would you need to
implement a virtual machine for it? ...

More commonly, when Forth is implemented on a generic processor,
Forth is implemented as an interpreter which uses a virtual machine.
Specifically, it uses a virtual processor, generally with registers
W, IP, PSP, RSP, UP, and/or X, and/or TOS. If you're interested in
this, you can read Bradford J. Rodriguez's Forth code for various
8-bit processors:

Bradford J. Rodriguez, PHD
http://www.bradrodriguez.com/papers/

However, I'd assume that if you're familiar with Koopman then
you're likely familiar with Rodriguez.

> You should read Philip J. Koopman, Jr's publications:
> http://users.ece.cmu.edu/~koopman/stack_computers/

I'm quite familiar with some of his work, including portions of his
book, since I consulted it for my Forth interpreter, e.g., instruction
set frequencies and similar papers by Anton Ertl's too.

[Rod Pemberton]

On Tue, 28 Jul 2015 04:05:53 -0400, Elizabeth D. Rather <era...@forth.com> wrote:

> On 7/27/15 1:33 PM, Rod Pemberton wrote:
>> On Mon, 27 Jul 2015 15:17:25 -0400, Alex McDonald <bl...@rivadpm.com> wrote:
>>> on 27/07/2015 19:02:21, hugher wrote:

>>>> I think that it is uncool that people should talk about me in threads
>>>> that I'm not involved in, especially if they make up stuff that isn't
>>>> true.
>>>
>>> Jeez, Hugh, John Passiniti hasn't posted here in years,
>>
>> It's true he hasn't posted under his own name, but
>> it's probably not true that he's not posting here.
>> See Alex Wegel and/or WJ.
>
> LOL, Passiniti had a well-defined particular point of view and set of
> experiences on which he based his comments, which are poles apart from
> those of Alex Wegel and WJ.

Alex is emotionally distraught over past Passiniti conversations,
conversations for which he wasn't even present, and is emotionally
hurt over them in the manner you'd expect Passiniti to be.

WJ codes anagrams in non-Forth languages the same way as Passiniti did.

[Anton Ertl]

"Rod Pemberton" <b...@fasdfrewar.cdm> writes:
>Yes. Everything an 8-bit microprocessor can do, and the general
>purpose instructions of 16-bit processors, can be done directly in
>C without needing to use assembly. The only thing missing is a
>carry flag and an overflow flag.

Yes, these are two things that you cannot express directly in C.

Another big restriction of C is that you are limited in the kind of
control-flow you can do. Threaded code? Not in standard C.
Tail-call optimization? Neither guaranteed nor commonplace (in
general, not possible). Coroutining? Nope. Prolog-like backtracking?
Nope.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2015: http://www.mpeforth.com/euroforth2015/euroforth2015.htm

[Paul Rubin]

an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> Tail-call optimization? Neither guaranteed nor commonplace (in
> general, not possible). Coroutining? Nope. Prolog-like backtracking?

How do you do coroutines portably in ANS Forth?

[Alexander Wegel]

Rod Pemberton <b...@fasdfrewar.cdm> wrote:

> Alex is emotionally distraught over past Passiniti conversations,
> conversations for which he wasn't even present, and is emotionally
> hurt over them in the manner you'd expect Passiniti to be.

No, i'm just pissed off by back-biting arseholes that can't (or just
don't care to) check even the simplest facts (like you are demonstrating
here).

I hope you'll change for the better after your coming-out (albeit it
probably wont make you smarter).

[Anton Ertl]

Not possible. But the comparison was with the capabilities of CPUs,
not with Forth.

PAF contains features to allow these things (in particular,
first-class labels and guaranteed tail-call optimization).
Coroutining is possible in PAF by implementing the stacks explicitly;
it might be a good idea to reify the implicit PAF stack for a more
efficient implementation.

[Anton Ertl]

Bernd Linsel <b...@gmx.com> writes:
>After having implemented several Forth VMs and native compilers on MIPS
>and ARM embedded systems, I may remark that Forth on RISC machines tends
>to be rather clumsy, regarding load delay slots, missing diplaced
>indirect addressing, and so on.
>You have to optimize very carefully if you want to have a stall-free
>running Forth on a RISC.
>(Not even to mention cache thrashing and similar effects that nowadays
>also CISC processors suffer from if you're mixing code and data).
>
>An optimizing native compiler gains of course performance of a large
>register set, if you analyze the data flow on the stack and map it on
>the registers, eliminating nearly all DUPs, SWAPs, DROPs etc (with the
>exception that the stack/register mapping must be consistent on all
>junction points).

Inlining is also important, because otherwise calls and returns are
the most frequent control-flow operation and, more importantly the
most common register allocation boundary, which makes the large
register set and all the sophistication you mention above much less
effective; alternatively, interprocedural register allocation, but
that is harder and probably less effective.

>RISC is in my opinion just muscles without the brain; raw force without
>any sophistication.

The idea in RISC originally was that you design the architecture
together with the compiler (or at least with a view to what compilers
can do and actually do); so I guess the brains are then in the compiler.

>If directed (programmed) accurately, you may achieve
>great performance -- if not, you can fail epically.

Actually, the RISC-specific performance issues like branch or load
delay slots cannot produce epic wins or epic fails; the numbers I
remember are on the order of 10%.

Other things, like good register allocation, is also important on
CISCs. And combining several Forth words into instructions may be
more important on CISCs than on RISCs; e.g. "cells + @ +" can (and
probably should) be combined into one instruction on IA-32 and AMD64,
whereas translating each word into one instruction does not miss any
optimization opportunity on MIPS.

By contrast, avoiding cache misses can have a much bigger effect, but
caches are not RISC-specific. Indeed, the problem with mixing code
and data is, in a way, CISC-specific; not because of RISC design
principles, but because separate I and D caches were introduced at
about the same time as RISCs, so RISC architectures were designed for
them (and required that software synchronizes the caches explicitly),
whereas older (CISC) architectures did the synchonization in hardware,
sometimes more often than necessary, at a steep cost.

The overall design principle of general-purpose hardware has been to
make the common case fast. Another design principle is to make some
case very fast, and to leave it to software to make that case common;
this design principle is most common in supercomputing, but
occasionally hardware designers try to apply it elsewhere, mostly
unsuccessfully (e.g., IA-64 and the Cell architecture in PS3).

[Bernd Linsel]

Am 29.07.2015 um 05:47 schrieb Rod Pemberton:
> On Tue, 28 Jul 2015 13:22:48 -0400, Bernd Linsel <b...@gmx.com> wrote:
>> Nope. C works fine on machines with huge register sets, a hardware
>> call stack, and a frame pointer. You're even better off if it supports
>> indexed register-indirect addressing modes.
>
> Yes. Everything an 8-bit microprocessor can do, and the general
> purpose instructions of 16-bit processors, can be done directly in
> C without needing to use assembly. The only thing missing is a
> carry flag and an overflow flag.
>
> While it's much easier to implement C on a machine with a hardware
> stack, C was implemented on mainframes without them. That's why the
> C standards don't require a stack. However, IIRC, it was Dennis
> Ritchie who showed in one of his papers that a stack is the easiest
> and cleanest way to implement recursion in C, which is required.
>
> Also, C works fine on machines with few registers. C works best on
> microprocessors with addressable 8-bit bytes, contiguous memory,
> two's complement arithmetic, a hardware stack, the largest integer
> a multiple of the 8-bit byte, and pointers the same size as the
> largest integers.

The possibility to implement C on a certain processor doesn't
automatically imply that this processor is suited well for C.
Indeed, the PDP-7 and -11 were, while many contemporary parts weren't.
But this may also be the reason why C was created like it is?!

> Forth generally likes these too, but needs two stacks to implement
> instead of one since it separates the data and control flow stack.
>
> Integers a multiple of the size of a byte is needed because C maps
> every object onto an array of bytes. Pointers the same size as the
> largest integer is convenient because of C's lossless type conversions,
> i.e., pointer to integer to pointer. C is difficult to implement on
> machines with a 16-bit word size, machines where the smallest
> addressable unit is much larger than a character in size, machines
> where the address pointer is greater in size than the largest integer.
> All of these situations require constructing special code, e.g., to
> save extra bits for pointer conversions or to extract characters
> packed into larger integers.

[Power of two] multiples of 8 bit-quantities are common since at least
mid-80s, but that doesn't define genericity either.
And this suits of course Forth as well as C, Java, Pascal, and so on.
Only languages that abstract from machine integer sizes, or work with
strange basic integers because they're tagging their data are above that
[Lisp, Smalltalk...].

>> I wouldn't call that a "generic microprocessor". Look at the (vain)
>> efforts to implement C on the 1980's common processors (8080/5, 8086,
>> ...). Motorolas 68K was the first part decently usable for C.
>
> While it seems you're familiar with some stuff from the 1980's, it
> seems you forgot about Ron Cain's SmallC. It was originally written
> for the Intel 8080 and also ported to PDP-11. If it can run on 8080,
> it can run on all other 8-bits. IIRC, the code emitted by the 8086
> port needed only *two* registers. I'd suspect it needed a few more
> on the 8080.

8080 and 8086 were even in their time regarded as register-starved. And,
as I stated above, the possibility of realizing a decent C compiler on
these platforms doesn't define them as well-suited for C.
Btw, the top performer for the '86 was Watcom C, still used today for
x86-based embedded parts.

>>>> Forth, instead, usually attempts to impose it's own virtual machine.
>>
>> Forth runs perfectly on stack machines, e.g. RTX 2000.
>
> The RTX 2000 was a dedicated Forth processor. Why would you need to
> implement a virtual machine for it? ...
>
> More commonly, when Forth is implemented on a generic processor,
> Forth is implemented as an interpreter which uses a virtual machine.
> Specifically, it uses a virtual processor, generally with registers
> W, IP, PSP, RSP, UP, and/or X, and/or TOS.

At least W, IP, PSP, and RSP are needed. But you can do with a bunch
less, see Rodriguez' CamelForth for 8051...

On the other hand, optimal prerequisites for (a classic, interpreted)
Forth are:
>= 8 gp registers, predecrementing and postincrementing indirect
addressing modes, at its best with an additional immediate and/or
register offset, register- and memory-indirect jumps... Everything that
makes up CISC in contrast to RISC's load-store philosophy.
And, not to forget, a unified cache instead Von Neumann architecture
would help.
Naturally, you don't need everything of that. Then, however, you have to
make compromises here or there.

> If you're interested in
> this, you can read Bradford J. Rodriguez's Forth code for various
> 8-bit processors:
>
> Bradford J. Rodriguez, PHD
> http://www.bradrodriguez.com/papers/
>
> However, I'd assume that if you're familiar with Koopman then
> you're likely familiar with Rodriguez.

Certainly.

>
>> You should read Philip J. Koopman, Jr's publications:
>> http://users.ece.cmu.edu/~koopman/stack_computers/
>
> I'm quite familiar with some of his work, including portions of his
> book, since I consulted it for my Forth interpreter, e.g., instruction
> set frequencies and similar papers by Anton Ertl's too.

These (Koopman, Rodriguez, Ertl & Co) are apperently the most familiar
papers for anyone who wants to develop an optimizing Forth compiler.

--
Best regards
Bernd Linsel

[Bernd Linsel]

Am 29.07.2015 um 12:28 schrieb Anton Ertl:> Bernd Linsel <b...@gmx.com>

writes:
>> After having implemented several Forth VMs and native compilers on MIPS
>> and ARM embedded systems, I may remark that Forth on RISC machines tends
>> to be rather clumsy, regarding load delay slots, missing diplaced
>> indirect addressing, and so on.
>> You have to optimize very carefully if you want to have a stall-free
>> running Forth on a RISC.
>> (Not even to mention cache thrashing and similar effects that nowadays
>> also CISC processors suffer from if you're mixing code and data).
>>
>> An optimizing native compiler gains of course performance of a large
>> register set, if you analyze the data flow on the stack and map it on
>> the registers, eliminating nearly all DUPs, SWAPs, DROPs etc (with the
>> exception that the stack/register mapping must be consistent on all
>> junction points).
>
> Inlining is also important, because otherwise calls and returns are
> the most frequent control-flow operation and, more importantly the
> most common register allocation boundary, which makes the large
> register set and all the sophistication you mention above much less
> effective; alternatively, interprocedural register allocation, but
> that is harder and probably less effective.

It's not just inlining, it's better to recompile (constraining the
dictionary to its state at definition time of the 'inlined' word,
otherwise you would produce "evaluate-macros"), because only then
junction points are eliminated and the stack/register mapping at compile
time unfolds its full power.

So you save the efforts for indeed mostly not very effective and complex
interprocedural register allocation, and achieve nice performance though.

>> RISC is in my opinion just muscles without the brain; raw force without
>> any sophistication.
>
> The idea in RISC originally was that you design the architecture
> together with the compiler (or at least with a view to what compilers
> can do and actually do); so I guess the brains are then in the compiler.

Okay, but they designed mostly C-compilers, not Forths.

And today it is common sense that the end-of-80s' goal of leaving the
hard job of optimizing to the compiler has failed, or at least, hasn't
been reached until today. But in the meantime we have undoubtedly
learned a lot about compilation, program and data flow analysis, and
code generation.

>> If directed (programmed) accurately, you may achieve
>> great performance -- if not, you can fail epically.
>
> Actually, the RISC-specific performance issues like branch or load
> delay slots cannot produce epic wins or epic fails; the numbers I
> remember are on the order of 10%.

My own experience was at about 50%, when placed at the most crucial
positions in a Forth interpreter: NEXT, DOCOL, SEMI-S.

> Other things, like good register allocation, is also important on
> CISCs. And combining several Forth words into instructions may be
> more important on CISCs than on RISCs; e.g. "cells + @ +" can (and
> probably should) be combined into one instruction on IA-32 and AMD64,
> whereas translating each word into one instruction does not miss any
> optimization opportunity on MIPS.
>
> By contrast, avoiding cache misses can have a much bigger effect, but
> caches are not RISC-specific. Indeed, the problem with mixing code
> and data is, in a way, CISC-specific; not because of RISC design
> principles, but because separate I and D caches were introduced at
> about the same time as RISCs, so RISC architectures were designed for
> them (and required that software synchronizes the caches explicitly),
> whereas older (CISC) architectures did the synchonization in hardware,
> sometimes more often than necessary, at a steep cost.

Mixing code and data is just as well expensive on RISCs, because cache
synchronization always takes its time, regardless whether initiated from
software or hardware. Doing it from software may be a bit more
effective, because the programmer knows best when it is appropriate and
needed.
In both cases the programmer is responsible for avoiding thrashing effects.

> The overall design principle of general-purpose hardware has been to
> make the common case fast. Another design principle is to make some
> case very fast, and to leave it to software to make that case common;
> this design principle is most common in supercomputing, but
> occasionally hardware designers try to apply it elsewhere, mostly
> unsuccessfully (e.g., IA-64 and the Cell architecture in PS3).
>
> - anton
>
--

Best regards,
Bernd Linsel

[Walter Banks]

On 29/07/2015 3:22 AM, Anton Ertl wrote:
> "Rod Pemberton" <b...@fasdfrewar.cdm> writes:
>> Yes. Everything an 8-bit microprocessor can do, and the general
>> purpose instructions of 16-bit processors, can be done directly in
>> C without needing to use assembly. The only thing missing is a
>> carry flag and an overflow flag.
>
> Yes, these are two things that you cannot express directly in C.
>
> Another big restriction of C is that you are limited in the kind of
> control-flow you can do. Threaded code? Not in standard C.
> Tail-call optimization? Neither guaranteed nor commonplace (in
> general, not possible). Coroutining? Nope. Prolog-like backtracking?
> Nope.

Tail-call optimization is part of most current embedded C compilers.
It for the most part almost eliminates the need for stack variables
compilers targeting small embedded processors.

Threaded code for small processors no common implementation.

w..