Why buffer contents can be influenced by EMIT or TYPE?

[Zbiggy]

There is a statement in "Starting Forth":

"...system will copy block 1 of the currently open file into one of the system
buffers. BLOCK also leaves on the stack the address of the beginning of the
buffer (1024 bytes, remember) that it used. The contents of this buffer are
guaranteed to stay valid until you execute a word from the set of procedures
with `multitasking impact', like EMIT or TYPE".


1. Actually, why EMIT, which simply outputs a character on the screen, should
affect contents of any buffer prepared to hold block contents?

2. How one should understand this "multitasking impact" exactly?

3. What are the words with "multitasking impact"? And which of the words
have such impact (but EMIT and TYPE)?

--
Forth is a preserver of health (Hippocrates)

[Andrew Haley]

Zbiggy <zbigniew2...@gmail.remove.com> wrote:
> There is a statement in "Starting Forth":
>
> "...system will copy block 1 of the currently open file into one of the system
> buffers. BLOCK also leaves on the stack the address of the beginning of the
> buffer (1024 bytes, remember) that it used. The contents of this buffer are
> guaranteed to stay valid until you execute a word from the set of procedures
> with `multitasking impact', like EMIT or TYPE".
>
>
> 1. Actually, why EMIT, which simply outputs a character on the screen, should
> affect contents of any buffer prepared to hold block contents?

Because EMIT might block waiting to send a character to the device, so
this is an opportunity to switch to another task, if there is one.
That other task might call BLOCK and invalidate the buffer.

> 2. How one should understand this "multitasking impact" exactly?

See section C.6:
http://forth.sourceforge.net/std/dpans/dpansc.htm#C.6

> 3. What are the words with "multitasking impact"? And which of the words
> have such impact (but EMIT and TYPE)?

They're in the list http://forth.sourceforge.net/std/dpans/dpans7.htm#7.3.3

Andrew.

[Brad]

On Feb 13, 7:17 am, Zbiggy <zbigniew2011REM...@gmail.REMOVE.com>
wrote:

> 2. How one should understand this "multitasking impact" exactly?

Any word that calls PAUSE. Usually I/O words. Perhaps in some Forths
tasks share buffers. I'm not sure this applies to modern Forths, since
sharing buffers to save memory is no longer necessary. If you even use
buffers (most people use only files).

> --
> Forth is a preserver of health (Hippocrates)

It didn't work out that way for Jeff. Maybe the wrong dialect?

-Brad

[Andrew Haley]

Brad <hwf...@gmail.com> wrote:
> On Feb 13, 7:17?am, Zbiggy <zbigniew2011REM...@gmail.REMOVE.com>

> wrote:
>> 2. How one should understand this "multitasking impact" exactly?
>
> Any word that calls PAUSE. Usually I/O words. Perhaps in some Forths
> tasks share buffers.

In all of them: it's not just about space saving, it's about
correctness. You don't want tasks with different opinions about the
contents of a particular block.

Andrew.

[Brad]

On Feb 13, 9:26 am, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> In all of them: it's not just about space saving, it's about
> correctness.  You don't want tasks with different opinions about the
> contents of a particular block.

I was going to say that embedded Forthers like me wouldn't use blocks,
but then it just occurred to me that blocks would be a great way for a
16-bit Forth to deal with large SPI flash.

-Brad

[Zbiggy]

In comp.lang.forth, Brad wrote:

> Any word that calls PAUSE. Usually I/O words. Perhaps in some Forths
> tasks share buffers. I'm not sure this applies to modern Forths, since
> sharing buffers to save memory is no longer necessary. If you even use
> buffers (most people use only files).

Thanks, Brad and Andrew. But I understand, that all this is related only
to multitasking-capable Forths, and the "simpler" ones are free of the
restriction (I mean: I don't have to worry that by executing EMIT sometimes
I can change assigned buffer's contents, by giving "green light" to another
task etc.)?

The mentioned restriction is strictly related to multitasking ability, right?

>> --
>> Forth is a preserver of health (Hippocrates)
>
> It didn't work out that way for Jeff. Maybe the wrong dialect?

Well, who knows? We're unable to find out, how long he would live _without_
Forth...

[Elizabeth D. Rather]

The other responses answer your question, but I'll provide a little more
background.

The classic Forth multitasker (originally developed by Chuck and still
in use in many native & embedded Forths) is "cooperative" -- a task
voluntarily relinquishes the CPU. One advantage of this is that there is
far less context to save and restore, so task switches are much faster
than with pre-emptive algorithms. Another advantage is that the
programmer knows when the program is doing something that may cause a
task switch.

The disadvantage is that a task may "hog" the CPU and prevent other
tasks from having timely access. The compromise which makes the
cooperative algorithm work is that whenever a task performs an I/O
operation it must relinquish the CPU. This is brilliant because I/O
takes a long time, comparatively speaking, and in most applications
(especially control systems, for which Forth was originally designed)
there isn't a lot to do between I/O operations. So this makes very
efficient use of the CPU, making the majority of its cycles available
for doing real work.

One must take a broad view of what defines I/O: for example, writing a
character to a screen is I/O, even if the "screen" is in a local frame
buffer. And any drivers that the programmer adds must also follow the
rule. So long as everyone respects this rule, cooperative multitasking
is effortless and in over 30 years of application programming I don't
recall any problems.

There are several ways to implement this. One is the high-level word
PAUSE, which someone mentioned. PAUSE relinquishes the CPU for exactly
one pass through the task list. A more sophisticated way, appropriate in
an interrupt-driven system, is to deactivate the task when the I/O is
initiated, and let the interrupt driver that responds when the action is
completed set its status to be active again.

And Andrew makes an important point about BLOCK buffers: they must be
shared in a multitasking system, to avoid the possibility of multiple
copies of a block with different contents. Sound application design
avoids conflicts. It's true that most modern Forths that run under a
host OS don't use BLOCK for disk I/O, but blocks are, indeed, quite
useful for managing flash or other mass storage.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[Elizabeth D. Rather]

If you're writing portable code that may run on a multitasked Forth you
should respect the rule.

[Zbiggy]

In comp.lang.forth, Elizabeth D. Rather wrote:

> If you're writing portable code that may run on a multitasked Forth you
> should respect the rule.

Of course, but I had in mind the ability of blocks - mentioned in SF - which
I particularly like: to use it as kind of "swap partition" equivalent. It
seems, it should be used with much greater care, than I thought at first: we
don't want data "swapped" to blocks to get corrupted.

Or maybe it's not possible at all in multitasked environment - at least not
in the way similar to one, in which Linux uses its swap partitions/files:
actually, we cannot block I/O operations at all just to make sure, they won't
allow buffers affection. The machine would be unusable.

[Zbiggy]

In comp.lang.forth, Elizabeth D. Rather wrote:

> One must take a broad view of what defines I/O: for example, writing a
> character to a screen is I/O, even if the "screen" is in a local frame
> buffer. And any drivers that the programmer adds must also follow the
> rule. So long as everyone respects this rule, cooperative multitasking
> is effortless and in over 30 years of application programming I don't
> recall any problems.

I'm afraid, I didn't completely understood: what you mean by writing, that
"cooperative multitasking is effortless"? What it was unable to do?

[Zbiggy]

In comp.lang.forth, Zbiggy wrote:

> I'm afraid, I didn't completely understood: what you mean by writing, that
> "cooperative multitasking is effortless"? What it was unable to do?

Did you mean perhaps, that by obeying the mentioned rule there wasn't any
problems with taking over entire CPU time?

[Elizabeth D. Rather]

On 2/13/12 9:12 AM, Zbiggy wrote:
> In comp.lang.forth, Zbiggy wrote:
>
>> I'm afraid, I didn't completely understood: what you mean by writing, that
>> "cooperative multitasking is effortless"? What it was unable to do?
>
> Did you mean perhaps, that by obeying the mentioned rule there wasn't any
> problems with taking over entire CPU time?

That's exactly what I mean: you don't have to *do* anything other than
respecting the rules to make everything work ok.

[Albert van der Horst]

In article <b6d4be14-a3dc-49f3...@18g2000yqe.googlegroups.com>,
Brad <hwf...@gmail.com> wrote:
>On Feb 13, 9:26=A0am, Andrew Haley <andre...@littlepinkcloud.invalid>

>wrote:
>> In all of them: it's not just about space saving, it's about

>> correctness. =A0You don't want tasks with different opinions about the

>> contents of a particular block.
>
>I was going to say that embedded Forthers like me wouldn't use blocks,
>but then it just occurred to me that blocks would be a great way for a
>16-bit Forth to deal with large SPI flash.

Or on chip flash for that matter.
My renesas ciforth takes advantage of a 1 Mbyte on chip flash
to store the library. Bottom line is that it behaves much like
a lina or wina system. It has renesas specific code to make a
turnkey, such that e.g. after reset it behaves as a clock
playing big ben at the hours.

See also renesas.html on my site below.

>
>-Brad

Groetjes Albert

--
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[Paul Rubin]

Andrew Haley <andr...@littlepinkcloud.invalid> writes:
> In all of them: it's not just about space saving, it's about
> correctness. You don't want tasks with different opinions about the
> contents of a particular block.

But that sounds pretty awful-- a task allocates a buffer and another
task can go and mess with it if you do a debugging print? It sounds
like a potential source of bugs, given no obvious way to tell whether a
given word might have some i/o effect. I'd have expected in a memory
constrained system, to just put a single task in charge of any buffers
on a particular file.

[Elizabeth D. Rather]

There's really a lot of design involved in the block management scheme,
really more than it's possible to summarize neatly here. If you have
access to my Forth Programmer's Handbook (pdf included in SwiftForth
free eval) there's an extended discussion of multitasking, blocks and
buffers, and all that jazz.

Briefly, buffers aren't "allocated". They are configured as a part of
the architecture of the system. They're always there. The disk
management system (which includes the word BLOCK, BUFFER, and the rest
of the machinery, makes a block in a buffer available to any task that
requests it. Actual reads and writes are transparent.

As for knowing "whether a given word might have some I/O effect," it's
really pretty obvious: you know when you accept input, display text,
read or write disk, or access any application-specific device, etc.,
don't you?

[A. K.]

On 13.02.2012 19:23, Elizabeth D. Rather wrote:

> One must take a broad view of what defines I/O: for example, writing a
> character to a screen is I/O, even if the "screen" is in a local frame
> buffer. And any drivers that the programmer adds must also follow the
> rule. So long as everyone respects this rule, cooperative multitasking
> is effortless and in over 30 years of application programming I don't
> recall any problems.

Well... in a system with different task priority levels (eg PLCs) one
usually wants deterministic time behaviour and triggered emergency programs.

It can become quite fiddlish to do it with cooperative multitasking only.

But fortunately not all RT systems demand it. So generally you are right.

Andreas

[Elizabeth D. Rather]

Depending on what response time you require, it's usually possible to
ensure adequate response. Actually, I only remember one occasion in
which we had to tweak the algorithm a bit, and that was 20+ years ago,
when processors were a lot slower than they are today. Guaranteeing
response times with a millisecond is no problem, and usually you can do
much better.

[Andrew Haley]

In practice it's not a problem. If you want an array on disk you can
do something like

xxx constant block#
: A@ ( index - n) cells 1024 /mod block# + block + @ ;
: A! ( n index) cells 1024 /mod block# + block + ! ;

Untested, but you get the idea. BLOCK itself is very fast if the
block is in a buffer, so it's quite OK to call it from such a word.

Andrew.

[Andrew Haley]

The "trick", if you can call it one, is that the multi-tasker is so
lightweight that it has almost no overhead. Deterministic behaviour
to guarantee strict response times can be achieved by doing things
that really need to be done immediately (i.e. microseconds) in an
interrupt routine and passing the rest to a task. Because a task
switch is done on every I/O operation, the multi-tasker is running
very much faster than conventional time-sliced systems, perhaps
thousands of times faster. A task switch is not much more expensive
than a call, and usually most tasks will be sleeping most of the time.
So, the time between an interrupt happening and the associated task
doing the work can be very short. Millisecond response times are not
hard to do at all, and that was true even on the processors of twenty
years ago.

Andrew.

[Albert van der Horst]

In article <slrnjjir92.ald.z...@Tichy.myhome.org>,

Zbiggy <zbigniew2...@gmail.REMOVE.com> wrote:
>In comp.lang.forth, Elizabeth D. Rather wrote:
>
>> If you're writing portable code that may run on a multitasked Forth you
>> should respect the rule.
>
>Of course, but I had in mind the ability of blocks - mentioned in SF - which
>I particularly like: to use it as kind of "swap partition" equivalent. It
>seems, it should be used with much greater care, than I thought at first: we
>don't want data "swapped" to blocks to get corrupted.

It is not too bad. Each time you use the BLOCK word, it is checked
whether the BLOCK is in memory.
E.g.
HEX 10 0 DO I #LINE * DUP BLOCK + #LINE TYPE CR LOOP

Don't do the BLOCK outside of the loop!

>
>Or maybe it's not possible at all in multitasked environment - at least not
>in the way similar to one, in which Linux uses its swap partitions/files:
>actually, we cannot block I/O operations at all just to make sure, they won't
>allow buffers affection. The machine would be unusable.

[A. K.]

Yeah, you're right, of course.

However in some RT systems you need to implement a scheduler. It manages
interrupt handling priorities, monitors time slices, and kills blocked
tasks or stops tasks that do not behave nicely (eg that overuse their
assigned time contingent).

Then it's a bit more than PAUSE. ;-)

And: cooperative multitasking offends against Murphy's law. :-))

Andreas

[Andrew Haley]

Albert van der Horst <alb...@spenarnc.xs4all.nl> wrote:
> In article <slrnjjir92.ald.z...@Tichy.myhome.org>,
> Zbiggy <zbigniew2...@gmail.REMOVE.com> wrote:
>>In comp.lang.forth, Elizabeth D. Rather wrote:
>>
>>> If you're writing portable code that may run on a multitasked Forth you
>>> should respect the rule.
>>
>>Of course, but I had in mind the ability of blocks - mentioned in SF - which
>>I particularly like: to use it as kind of "swap partition" equivalent. It
>>seems, it should be used with much greater care, than I thought at first: we
>>don't want data "swapped" to blocks to get corrupted.
>
> It is not too bad. Each time you use the BLOCK word, it is checked
> whether the BLOCK is in memory.
> E.g.
> HEX 10 0 DO I #LINE * DUP BLOCK + #LINE TYPE CR LOOP

That won't work: you need to copy the line to e.g. PAD before calling
TYPE. Forth, Inc have always included a word to do this.

Andrew.

[Andrew Haley]

A. K. <a...@nospam.org> wrote:

> However in some RT systems you need to implement a scheduler. It
> manages interrupt handling priorities, monitors time slices, and
> kills blocked tasks or stops tasks that do not behave nicely (eg
> that overuse their assigned time contingent).

I suspect that a lot of the heroic efforts made by systems with
priority schedulers really just compensate for the fact that they're
not fast enough. Also, as soon as you have pre-emption you need a lot
of code to handle mutexes, semaphores, and so on, and this slows down
the system even more.

It's different, of course, if you are sharing a general-purpose
operating system with untrusted code, but that's a very different kind
of system.

> Then it's a bit more than PAUSE. ;-)
>
> And: cooperative multitasking offends against Murphy's law. :-))

I don't think so: if you have all that code that depends on priorities
to work correctly, you have a lot more that can go wrong.

Andrew.

[A. K.]

On 14.02.2012 14:52, Andrew Haley wrote:
> A. K.<a...@nospam.org> wrote:
>
>> However in some RT systems you need to implement a scheduler. It
>> manages interrupt handling priorities, monitors time slices, and
>> kills blocked tasks or stops tasks that do not behave nicely (eg
>> that overuse their assigned time contingent).
>
> I suspect that a lot of the heroic efforts made by systems with
> priority schedulers really just compensate for the fact that they're
> not fast enough.

Sad but true. But industry automation cannot afford to chase after every
hardware innovation of the semiconductor industry. Typical DCS life
spans are 15 years.

> Also, as soon as you have pre-emption you need a lot
> of code to handle mutexes, semaphores, and so on, and this slows down
> the system even more.

But whether you do this on top of a RTOS or as part of your Forth
application doesn't matter. You have to spend processor cycles on
managing RT events either way.

> It's different, of course, if you are sharing a general-purpose
> operating system with untrusted code, but that's a very different kind
> of system.

Why untrusted? Just because it's developed by others? How can you be
sure that your own code is bug-free?

Andreas

[Andrew Haley]

A. K. <a...@nospam.org> wrote:
> On 14.02.2012 14:52, Andrew Haley wrote:
>> A. K.<a...@nospam.org> wrote:
>
>> Also, as soon as you have pre-emption you need a lot of code to
>> handle mutexes, semaphores, and so on, and this slows down the
>> system even more.
>
> But whether you do this on top of a RTOS or as part of your Forth
> application doesn't matter. You have to spend processor cycles on
> managing RT events either way.

Of course, but that's way different from doing things that are only
necessary because of the type of scheduler in use.

>> It's different, of course, if you are sharing a general-purpose
>> operating system with untrusted code, but that's a very different
>> kind of system.
>
> Why untrusted? Just because it's developed by others? How can you be
> sure that your own code is bug-free?

You can't, but pre-emption doesn't necessarily make the system as a
whole any more reliable. If you end up with a system that is more
complex than it needs to be then there's more of it to go wrong.
Adding things like pre-emption for tasks that have run too long is
just sticking plaster. Sure, it's always good to have a watchdog, and
this will trigger if any task runs too long, but that's really for
hardware failures. It's a poor substitute for getting the software
right.

Andrew.

[A. K.]

On 14.02.2012 16:24, Andrew Haley wrote:
> A. K.<a...@nospam.org> wrote:
>> Why untrusted? Just because it's developed by others? How can you be
>> sure that your own code is bug-free?
>

> Sure, it's always good to have a watchdog, and
> this will trigger if any task runs too long, but that's really for
> hardware failures.

Yessir, hardware wins mostly here. It's the machine that counts, not
those tiny little bits inside. :-)

But: poor race conditions, not-responding servers, human errors(!),
faulty algorithms, floating-point math noise, et cetera et cetera, are
very often software related and can cause havoc to cooperative MT. And
then you'll just _want_ a last resource safety net in critical systems.

Andreas

[Andrew Haley]

A. K. <a...@nospam.org> wrote:
> On 14.02.2012 16:24, Andrew Haley wrote:
>> A. K.<a...@nospam.org> wrote:
>>> Why untrusted? Just because it's developed by others? How can you be
>>> sure that your own code is bug-free?
>>
>> Sure, it's always good to have a watchdog, and this will trigger if
>> any task runs too long, but that's really for hardware failures.
>
> Yessir, hardware wins mostly here. It's the machine that counts, not
> those tiny little bits inside. :-)
>
> But: poor race conditions, not-responding servers, human errors(!),
> faulty algorithms, floating-point math noise, et cetera et cetera, are
> very often software related and can cause havoc to cooperative MT.

They'll cause havoc anyway, not just if you have co-operative MT. I'm
not convinced that pre-emptive MT is part of the solution set rather
than part of the problem set.

> And then you'll just _want_ a last resource safety net in critical
> systems.

And, indeed, a watchdog is that. Pre-emptive multitasking, IMO, is
not: it solves one class of problems but only at the cost of
introducing another.

Andrew.

[A. K.]

On 14.02.2012 16:49, Andrew Haley wrote:
> A. K.<a...@nospam.org> wrote:
>> But: poor race conditions, not-responding servers, human errors(!),
>> faulty algorithms, floating-point math noise, et cetera et cetera, are
>> very often software related and can cause havoc to cooperative MT.
>
> They'll cause havoc anyway, not just if you have co-operative MT.

Well IMO that's phrased a bit too simple. You need deterministic
reaction to encounter such troubles.

> I'm not convinced that pre-emptive MT is part of the solution set rather
> than part of the problem set.

Assuming that you are not religious on this old issue: The vast majority
of process scheduling systems work preemptive today. (Unix being one of
them).

But when you don't need a scheduler because you program for a friendly
closed world: Why not go the easy way, go cooperative, it's faster!

:-)

[Andrew Haley]

A. K. <a...@nospam.org> wrote:
> On 14.02.2012 16:49, Andrew Haley wrote:
>> A. K.<a...@nospam.org> wrote:
>>> But: poor race conditions, not-responding servers, human errors(!),
>>> faulty algorithms, floating-point math noise, et cetera et cetera, are
>>> very often software related and can cause havoc to cooperative MT.
>>
>> They'll cause havoc anyway, not just if you have co-operative MT.
>
> Well IMO that's phrased a bit too simple. You need deterministic
> reaction to encounter such troubles.
>
>> I'm not convinced that pre-emptive MT is part of the solution set rather
>> than part of the problem set.
>
> Assuming that you are not religious on this old issue: The vast
> majority of process scheduling systems work preemptive today. (Unix
> being one of them).

Well, yes, I think that's always been true, even back when the Forth,
Inc multi-tasker was first written. The question isn't which is more
popular, but the trade-offs in a particular environment. Co-operative
schedulers can work better in some circumstances, given a team of
programmers who know what they're doing. If you want sparklingly
fast, predictable response times on resource-constrained systems they
can be superb.

UNIX is a bit different from an RTOS, because you have a bunch of
different users doing different things: it's a time-sharing system.
Some of those users might be trying to crash that system, so it has to
be highly defensive.

I love UNIX-like systems, and that's what I develop in my day job;
they pay my wages. But I've heard "Oh no, you can't do high
reliability real-time systems with a co-operative scheduler" many
times before. It is just FUD, and I'm not having it, for good
*technical* reasons.

> But when you don't need a scheduler because you program for a
> friendly closed world: Why not go the easy way, go cooperative, it's
> faster!

Egzackerly.

Andrew.

[Elizabeth D. Rather]

Tsk, tsk, Andrew, shame on you!

: A! ( n index) cells 1024 /mod block# + block + ! UPDATE ;

[Andrew Haley]

Elizabeth D. Rather <era...@forth.com> wrote:
> On 2/14/12 12:36 AM, Andrew Haley wrote:
>>
>> In practice it's not a problem. If you want an array on disk you can
>> do something like
>>
>> xxx constant block#
>> : A@ ( index - n) cells 1024 /mod block# + block + @ ;
>> : A! ( n index) cells 1024 /mod block# + block + ! ;
>>
>> Untested, but you get the idea. BLOCK itself is very fast if the
>> block is in a buffer, so it's quite OK to call it from such a word.
>
> Tsk, tsk, Andrew, shame on you!
>
> : A! ( n index) cells 1024 /mod block# + block + ! UPDATE ;

Haha, that's what I get for not testing!

Andrew.

[Elizabeth D. Rather]

How long has it been since you wrote code for a block-based system?
:-)

[Andrew Haley]

Elizabeth D. Rather <era...@forth.com> wrote:
> On 2/14/12 8:20 AM, Andrew Haley wrote:
>> Elizabeth D. Rather<era...@forth.com> wrote:
>>> On 2/14/12 12:36 AM, Andrew Haley wrote:
>>>>
>>>> In practice it's not a problem. If you want an array on disk you can
>>>> do something like
>>>>
>>>> xxx constant block#
>>>> : A@ ( index - n) cells 1024 /mod block# + block + @ ;
>>>> : A! ( n index) cells 1024 /mod block# + block + ! ;
>>>>
>>>> Untested, but you get the idea. BLOCK itself is very fast if the
>>>> block is in a buffer, so it's quite OK to call it from such a word.
>>>
>>> Tsk, tsk, Andrew, shame on you!
>>>
>>> : A! ( n index) cells 1024 /mod block# + block + ! UPDATE ;
>>
>> Haha, that's what I get for not testing!
>
> How long has it been since you wrote code for a block-based system?
> :-)

Ummm, I'm not sure. At least two decades, I think. Less than three.

Andrew.

[John Passaniti]

On Feb 14, 11:44 am, "A. K." <a...@nospam.org> wrote:
> But when you don't need a scheduler because you program for a
> friendly closed world: Why not go the easy way, go
> cooperative, it's faster!

This newsgroup is filled with false dichotomies. Either you do X or
Y. How depressingly binary; how about doing both?

In a system at work, I'm looking at embedded Linux *and* cooperative
multitasking. The preemptive multitasking of Linux is there to let us
run multiple applications, both our own and third-party "middleware".
There, preemptive multitasking lets us not worry so much about
combining these applications because they are all independent
processes protected from each other. It also lets us run user scripts
(which we sandbox, but otherwise have no control over) in their own
protected space. Within our application code, we would be using
cooperative multitasking in two forms. The first is a coroutine-based
scheduler for handling networking (mostly a web-based user
interface). The second form of cooperative multitasking is from using
an event-based programming style. Our system generates events for
practically everything, and so a natural programming style is to hook
up functions to these events that get executed when the event fires.
And sometimes, it makes sense to handle those events with functions
that fire off processes or threads.

[A. K.]

Before we end up with another false lobotomy ...
I am ... ... ... flabbergasted

:D

[P.M.Lawrence]

On Feb 14, 10:16 am, "Elizabeth D. Rather" <erat...@forth.com> wrote:
> On 2/13/12 1:08 PM, A. K. wrote:
>
>
>
> > On 13.02.2012 19:23, Elizabeth D. Rather wrote:
>
> >> One must take a broad view of what defines I/O: for example, writing a
> >> character to a screen is I/O, even if the "screen" is in a local frame
> >> buffer. And any drivers that the programmer adds must also follow the
> >> rule. So long as everyone respects this rule, cooperative multitasking
> >> is effortless and in over 30 years of application programming I don't
> >> recall any problems.
>
> > Well... in a system with different task priority levels (eg PLCs) one
> > usually wants deterministic time behaviour and triggered emergency
> > programs.
>
> > It can become quite fiddlish to do it with cooperative multitasking only.
>
> > But fortunately not all RT systems demand it. So generally you are right.
>
> Depending on what response time you require, it's usually possible to
> ensure adequate response. Actually, I only remember one occasion in
> which we had to tweak the algorithm a bit, and that was 20+ years ago,
> when processors were a lot slower than they are today. Guaranteeing
> response times with a millisecond is no problem, and usually you can do
> much better.

Maybe usually, but not always.

Here in Australia, the firm Colour Vision Systems used Forth to
control its fruit sorting machines, a Forth they implemented and
optimised for their own needs (I think you've gone more Linux since,
but they probably kept a lot of the old stuff on that, and they would
still have had to support their older equipment too). They put some of
their Forth documentation and source up on the internet, so I once
found an archive that still had it and had a look at it (sorry, you'll
have to search for it yourselves - I don't recall the link).

That showed that Colour Vision Systems had gone for a pre-emptive
multitasking approach as the co-operative approach didn't give their
hardware enough fine grained control for the stepper motors etc. the
way they were using those in that niche application field (think pin
ball machine style flippers diverting sub-standard or different
quality fruit from a conveyor according to analysis of spectrographic
readings of the surface and interior of the fruit at visible and other
wavelengths - I oversimplify, but not by much). P.M.Lawrence.

[Andrew Haley]

P.M.Lawrence <pml5...@gmail.com> wrote:
>>
>> Depending on what response time you require, it's usually possible to
>> ensure adequate response. Actually, I only remember one occasion in
>> which we had to tweak the algorithm a bit, and that was 20+ years ago,
>> when processors were a lot slower than they are today. Guaranteeing
>> response times with a millisecond is no problem, and usually you can do
>> much better.
>
> Maybe usually, but not always.
>
> Here in Australia, the firm Colour Vision Systems used Forth to
> control its fruit sorting machines, a Forth they implemented and
> optimised for their own needs (I think you've gone more Linux since,
> but they probably kept a lot of the old stuff on that, and they would
> still have had to support their older equipment too). They put some of
> their Forth documentation and source up on the internet, so I once
> found an archive that still had it and had a look at it (sorry, you'll
> have to search for it yourselves - I don't recall the link).
>
> That showed that Colour Vision Systems had gone for a pre-emptive
> multitasking approach as the co-operative approach didn't give their
> hardware enough fine grained control for the stepper motors etc. the
> way they were using those in that niche application field (think pin
> ball machine style flippers diverting sub-standard or different
> quality fruit from a conveyor according to analysis of spectrographic
> readings of the surface and interior of the fruit at visible and other
> wavelengths - I oversimplify, but not by much).

I think there must be more to it than this. You'd usually be
controlling the stepper motors in an interrupt routine.

Andrew.

[Albert van der Horst]

In article <4f3a8f63$0$7626$9b4e...@newsspool1.arcor-online.net>,

I think we must add something to just the cooperative term.

The cooperative systems in Forth (manx2, Arab baggage handling etc.)
have in common that it is designed as a whole.

I have worked on a Java system (thousands of modules, thousands
of threads) running on a heavy duty Pentium, and it couldn't
count of a sensor that changes at most once each ten millisecond.

I guess for too complicated systems, you just need priorities/preemptiveness
such that the remainder of the system can pretty much be ignored.

[A. K.]

On 15.02.2012 12:06, Albert van der Horst wrote:
> In article<4f3a8f63$0$7626$9b4e...@newsspool1.arcor-online.net>,
> A. K.<a...@nospam.org> wrote:
>> But when you don't need a scheduler because you program for a friendly
>> closed world: Why not go the easy way, go cooperative, it's faster!
>
> I think we must add something to just the cooperative term.
>
> The cooperative systems in Forth (manx2, Arab baggage handling etc.)
> have in common that it is designed as a whole.
>
> I have worked on a Java system (thousands of modules, thousands
> of threads) running on a heavy duty Pentium, and it couldn't
> count of a sensor that changes at most once each ten millisecond.
>
> I guess for too complicated systems, you just need priorities/preemptiveness
> such that the remainder of the system can pretty much be ignored.
>

In the end it's the application domain that "decides" the suitable
software technique, not the programmer who is working on a subdomain.

Your example reminds me of a DCS. But there the automation level has
their own CPUs and they just feed a shared process image to the slower
Java world running in the servers. Fast switch counting (or fast analog
integration) would be executed by an automation CPU which writes counted
values, not just raw sensor inputs, into the process image.

It also reminds me of Jerry Avin's famous signature "Engineering is the
art of making what you want from things you can get". BTW I hope he is
still alive and happy...

Andreas

[Elizabeth D. Rather]

Wow, that generated a wave of nostalgia! In about 1980 we programmed a
series of peach-sorting machines for FMC in California. Dean Sanderson
was the lead programmer, and told tales of being covered in peach pulp
during acceptance test runs! As I recall, the computer was 8080-based.
As Andrew suggests, the motors and other devices were all
interrupt-driven, and our multitasker worked superbly.

I suspect that since Colour Vision developed their own Forth, they
didn't have a good model of Forth-style multitasking to follow, and thus
followed conventional wisdom in terms of design.

[Elizabeth D. Rather]

On 2/15/12 1:06 AM, Albert van der Horst wrote:
> In article<4f3a8f63$0$7626$9b4e...@newsspool1.arcor-online.net>,
> A. K.<a...@nospam.org> wrote:
>> On 14.02.2012 16:49, Andrew Haley wrote:

...

>>> I'm not convinced that pre-emptive MT is part of the solution set rather
>>> than part of the problem set.
>>
>> Assuming that you are not religious on this old issue: The vast majority
>> of process scheduling systems work preemptive today. (Unix being one of
>> them).
>>
>> But when you don't need a scheduler because you program for a friendly
>> closed world: Why not go the easy way, go cooperative, it's faster!
>
> I think we must add something to just the cooperative term.
>
> The cooperative systems in Forth (manx2, Arab baggage handling etc.)
> have in common that it is designed as a whole.

Yes, I'll agree with that. The application can be extremely complex (the
baggage handling was for American Airlines in LA; the system in Saudi
Arabia controlled the entire airport, including security system, runway
lights, fuel farm, water distribution, etc., with over 500 networked
computers), but should be a holistic system programmed by a team of
people who can agree on global design issues and principles. IMO all
complex projects can benefit from that approach.

[BruceMcF]

On Feb 14, 5:49 am, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> So, the time between an interrupt happening and the associated task
> doing the work can be very short.  Millisecond response times are not
> hard to do at all, and that was true even on the processors of twenty
> years ago.

And if that is not prompt enough and you *need* to prioritize, the
simplest priority is a main task that runs its own round robin of its
own subtasks. In cases where once around the round robin is sometimes
too much lag, and there are things that are not done as comfortably in
an interrupt routine, like a couple of high priority processing
pipelines and a set of more patient result delivery tasks, that could
well do it.

But OTOH if they are truly patient tasks, cleaning up the task design
and task scheduling so there are not too many lower priority tasks in
the round robin at once may well give the same effect for much less
programming overhead, and often for more efficient resource use, since
scheduled in that way the patient tasks are better able to share
memory resources. A low priority task may well just be one that calls
the low priority scheduler when it exits, so the low priority
scheduler can launch the next task in the low priority queue.

And if the problem is that the average task-run times of the lower
priority tasks in the round robin are fine but maximum task-run time
profiles cause problems when a few lower priority tasks hit their
longer task-run times in the same run around the round robin, it may
be that getting more aggressive with PAUSE in the lower priority tasks
to pull down their maximum task-run times is all you need.

[Paul Rubin]

"Elizabeth D. Rather" <era...@forth.com> writes:
> Briefly, buffers aren't "allocated". They are configured as a part of
> the architecture of the system. They're always there.

Well ok, I didn't mean allocated in the sense of a memory pool, but
rather I'd have expected they were somehow assigned to tasks. I've been
meaning to read up on the block system though.

> As for knowing "whether a given word might have some I/O effect," it's
> really pretty obvious: you know when you accept input, display text,
> read or write disk, or access any application-specific device, etc.,
> don't you?

Really, in a big program, you might not necessarily know. There is the
obvious example where putting a debugging print into some word changes
the multitasking behavior. But more generally, maybe you write the
program with a configuration parameter "100 constant widgetsize" and
later someone changes the widget-related words to read the widget size
from a file or database. In a big program it's easy to not notice the
consequences of changes like that.

[Elizabeth D. Rather]

On 2/15/12 11:20 AM, Paul Rubin wrote:
> "Elizabeth D. Rather"<era...@forth.com> writes:
>> Briefly, buffers aren't "allocated". They are configured as a part of
>> the architecture of the system. They're always there.
>
> Well ok, I didn't mean allocated in the sense of a memory pool, but
> rather I'd have expected they were somehow assigned to tasks. I've been
> meaning to read up on the block system though.

No, they are very definitely not assigned to tasks. Honestly, there are
fewer conflict issues when tasks can share blocks than when they're
assigned! That's experience talking!

>> As for knowing "whether a given word might have some I/O effect," it's
>> really pretty obvious: you know when you accept input, display text,
>> read or write disk, or access any application-specific device, etc.,
>> don't you?
>
> Really, in a big program, you might not necessarily know. There is the
> obvious example where putting a debugging print into some word changes
> the multitasking behavior. But more generally, maybe you write the
> program with a configuration parameter "100 constant widgetsize" and
> later someone changes the widget-related words to read the widget size
> from a file or database. In a big program it's easy to not notice the
> consequences of changes like that.

Well, a good defensive strategy is to assume you have control only
through a sequence of operations that are simple enough that you know.
In general, it's inadvisable to expect to use an address in a block for
longer than a simple operation. For example:

1000 CONSTANT MY-BLOCK
256 CONSTANT #ITEMS

: my-data ( i -- a ) #ITEMS MIN CELLS MY-BLOCK BLOCK + ;
: show ( -- ) #ITEMS 0 DO I my-data @ .
I 10 MOD 0= IF CR THEN
LOOP CR ;

In this example, all the words between BLOCK in my-data and . (which
does I/O) are known not to have multitasking impact.

The important thing here is that you call BLOCK for every item, rather
than assume you can hold on to the block's address while you display the
data. If the buffer hasn't been re-used, there will not be a disk read,
and this will take very little time. If other tasks are accessing disk
frequently, on the other hand, your block may be in a different buffer
every time. You don't care, the code will always work.

It is possible to prevent other tasks' access to disk (or another shared
facility) for a time, but that is generally discouraged as it adversely
impacts overall system performance.

Again, this is a big topic, and I am only touching the high points here.
Using blocks in a multitasking environment involves a whole philosophy;
once you "get it" it's effortless, like using the stack, but it's
different enough from other programming that there's a learning curve.

[Paul Rubin]

"Elizabeth D. Rather" <era...@forth.com> writes:
> In general, it's inadvisable to expect to use an address in a

> block for longer than a simple operation. For example: ...

OK, you effectively put a critical section around the operation on the
block contents. But that either 1) locks out task switching for a
potentially long time; or 2) requires you to copy the block data
elsewhere so you can work on it while multitasking (causing extra
copying overhead, and maybe running into memory constraints on small
systems).

> Again, this is a big topic, and I am only touching the high points
> here. Using blocks in a multitasking environment involves a whole
> philosophy; once you "get it" it's effortless, like using the stack,
> but it's different enough from other programming that there's a
> learning curve.

I guess it's possible to do something customized to the problem in cases
where the general purpose block system hits obstacles like described
above. So it's sufficient if the block system does something reasonable
most of the time.

[Elizabeth D. Rather]

On 2/15/12 1:31 PM, Paul Rubin wrote:
> "Elizabeth D. Rather"<era...@forth.com> writes:
>> In general, it's inadvisable to expect to use an address in a
>> block for longer than a simple operation. For example: ...
>
> OK, you effectively put a critical section around the operation on the
> block contents. But that either 1) locks out task switching for a
> potentially long time; or 2) requires you to copy the block data
> elsewhere so you can work on it while multitasking (causing extra
> copying overhead, and maybe running into memory constraints on small
> systems).

In the example I gave there was no such "critical section", and nothing
was locked out. But following the invocation of BLOCK, which returned
the address, there were only the words + ; and @ (in the calling word).
None of those have multitasking impact, so therefore the address
returned by BLOCK was safe. However, following the . (which displays the
number) the address was no longer reliable.

There may be circumstances in which it's worth copying data to a
temporary location, but usually not. For example, the block editor
works with data in the buffer, calling BLOCK for every access. Overall,
that's far less overhead than attempting to manage a "critical section"
(which, as you say, impacts other tasks) or doing unnecessary copies.
Statistically, in such a scenario (editing a block) you are invoking the
block frequently, and it will stay in memory.

>> Again, this is a big topic, and I am only touching the high points
>> here. Using blocks in a multitasking environment involves a whole
>> philosophy; once you "get it" it's effortless, like using the stack,
>> but it's different enough from other programming that there's a
>> learning curve.
>
> I guess it's possible to do something customized to the problem in cases
> where the general purpose block system hits obstacles like described
> above. So it's sufficient if the block system does something reasonable
> most of the time.

What obstacle? In my view, it does something "reasonable" all the time,
providing you understand the rules!

[P.M.Lawrence]

I just hunted up that (Russian) archive, which is at
http://www.forth.org.ru/~mlg/mirror/coldforth.teegra.net.

It's pretty clear from http://www.forth.org.ru/~mlg/mirror/coldforth.teegra.net/tcpippaper/tcp_ip.html
that they DID have Forth models to follow, and made considerable use
of them, e.g. Anton Ertl’s OOP work.
http://www.forth.org.ru/~mlg/mirror/coldforth.teegra.net/multitaskingpaper/multitasking_paper.html
gives their design reasoning for choosing pre-emptive multitasking. It
states inter alia that "To get the smooth option do I need preemptive
multitasking? The s[h]ort answer is no. You can do it with no
multitasking. If an application programmer was trying to do the job
with an OS that supported no multitasking they would simply call the
ramp function regularly... The single loop multitasker brings one big
advantage. You do not have to know anything about the other functions,
you simply exit to the multitasker often, and give others a go." So
they did know about this approach! The rest of the paper clarifies
their choices, particularly the issues of jerkiness in motors. So it
wasn't a case of being necessary because they didn't know anything
else, but that they thought it reconciled all their system needs best,
including mechanical stuff and TCP/IP. P.M.Lawrence.

[John Passaniti]

On Feb 15, 8:38 pm, "P.M.Lawrence" <pml540...@gmail.com> wrote:
> It's pretty clear fromhttp://www.forth.org.ru/~mlg/mirror/coldforth.teegra.net/tcpippaper/t...

> that they DID have Forth models to follow, and made considerable use

> of them, e.g. Anton Ertl’s OOP work.http://www.forth.org.ru/~mlg/mirror/coldforth.teegra.net/multitasking...

> gives their design reasoning for choosing pre-emptive multitasking. It
> states inter alia that "To get the smooth option do I need preemptive
> multitasking? The s[h]ort answer is no. You can do it with no
> multitasking. If an application programmer was trying to do the job
> with an OS that supported no multitasking they would simply call the
> ramp function regularly... The single loop multitasker brings one big
> advantage. You do not have to know anything about the other functions,
> you simply exit to the multitasker often, and give others a go." So
> they did know about this approach! The rest of the paper clarifies
> their choices, particularly the issues of jerkiness in motors. So it
> wasn't a case of being necessary because they didn't know anything
> else, but that they thought it reconciled all their system needs best,
> including mechanical stuff and TCP/IP. P.M.Lawrence.

The other way to look at this is to ask the question, where do you
want to spend your development time? In the case of "hard" real-time
systems (those with strict timing requirements that if violated cause
the system to fail), you're can define "simple" as a system that
requires the programmer do things like put in explicit yields to the
multitasker, measure the worst-case time of function calls, and
structure the code so the timing requirements are met. You could also
define "simple" as a system where the programmer deferred this to a
more complicated pre-emptive system that was configured to ensure
those timing requirements, but now asked the programmer to define
critical sections and/or some system of semaphores/locks/mutexes/
whatever to keep context switches from changing state inappropriately.

Neither approach is wrong, and both are "simple" depending on the
larger system context. But both approaches (and the endless
variations on those themes) all really are subject to "conservation of
complexity." The choice of one or the other is really a choice
between where you want to put the complexity. And that is generally
true not just of multitasking, but everything. The traditions in
Forth amount to just a different set of choices in what one person
considers to be complex.