progress dots in ved
Jonathan L Cunningham
six weeks (mid January to early March) and used over 1175 hours of cpu
time, on a 1.8GHz pentium (and used 200MB of heap).[*]
It was dealing with lists that grew to more than a million items long.
I think that's impressive for any piece of software, and nicely
demonstrates that poplog is robust!
However, I did have a minor problem. In order to show that the program
hadn't crashed or was looping, I had it print out dots "." every 100th
item processed. (Which could take minutes.)
The problem was that I forgot that output was buffered, so instead of
printing a dot every minute or so, it printed 100 dots every couple of
hours.
Before I run the program again (after optimising it a bit - six weeks is
a little long to wait!) I want to fix this. Unfortunately, I can't
remember how to flush output buffers in xved (or winpop).
Here's a test prog:
repeat 3 times pr('.'), syssleep(200) endrepeat;
Instead of printing 3 dots at 2 second intervals, it waits 6 seconds and
then prints all three dots at once. I need:
repeat 3 times pr('.'), flush(), syssleep(200) endrepeat;
What is the definition of flush() when running in "immediate" mode in a
ved buffer (xved or winpop)?
I tried, sysflush(popdevout) and also vedrefresh(). Neither worked.
I have skimmed some of the documentation (REF vedprocs, REF sysio) but
if the answer is there I missed it.
Help!
Jonathan
[*] In case you are wondering, it was to solve a minor graph-theoretical
problem: to find out how many different graphs there were of a certain
form, by exhaustively enumerating them. And yes, I did check Sloane's
Encyclopedia of Integer Sequences first, to see if the answer was in
there :-)
Aaron Sloman
> I've just run a program in Windows Poplog Common Lisp that ran for over
> six weeks (mid January to early March) and used over 1175 hours of cpu
> time, on a 1.8GHz pentium (and used 200MB of heap).[*]
That must be a record for poplog, though some people have used
pop-11 in evolutionary computation experiments run on multiple cpus
for a shorter time, which may have used as much cpu time and memory
in total.
> It was dealing with lists that grew to more than a million items long.
Remember the days when we used to have to share a 256Kbyte PDP11/40
between about 16 simultaneous users?
> I think that's impressive for any piece of software, and nicely
> demonstrates that poplog is robust!
>
> However, I did have a minor problem. In order to show that the program
> hadn't crashed or was looping, I had it print out dots "." every 100th
> item processed. (Which could take minutes.)
>
> The problem was that I forgot that output was buffered, so instead of
> printing a dot every minute or so, it printed 100 dots every couple of
> hours.
>....
> Instead of printing 3 dots at 2 second intervals, it waits 6 seconds and
> then prints all three dots at once. I need:
>
> repeat 3 times pr('.'), flush(), syssleep(200) endrepeat;
>
> What is the definition of flush() when running in "immediate" mode in a
> ved buffer (xved or winpop)?
There are two problems here
(a) getting the poplog process to flush its output buffer, which in
pop11 you can do most easily with charout(0).
So if I run this from the terminal,
repeat 3 times pr('.'); charout(0); syssleep(200); endrepeat;
I see it print a dot every two seconds, whereas if I leave out the
charout instruction it prints three dots after six seconds.
Presumably there's a simple lisp equivalent.
I don't know if there is a way to get windows poplog to print
directly to a 'terminal' window instead of sending output to a Ved
buffer. If there is that will solve your problem. (Or you could run
linux using vmware or colinux or ....????).
(b) If you have to output to Ved, then the second problem is getting
the ved output file to re-display itself, which it normally will not
do unless ved is waiting for some input or the contents of the
display need to be refreshed.
As far as I recall that will not happen while it is simply filling
up a line of text, or while another (non-ved) poplog process is
running.
There may be a ved command that forces a refresh that I have not
remembered.
It will refresh when a new line starts, so you could shorten the
screen width to make it refresh more often.
Alternatively, you could try printing a newline after each dot,
though after the screen is full you will then not see any change,
unless you alternate between two characters or print something
different every N characters.
I tried this in XVed on linux
vars x = 0;
repeat 50 times
charout(`0`+x mod 2);
;;; output new line
charout(10);
x+1->x;
syssleep(50);
endrepeat;
and it worked printing 0 and 1 on alternate lines every half second.
It also worked in Ved, but was more jumpy, as if refreshing only
after every three or four lines. That could be a linux buffer
problem.
But that solution has a problem, whether you use Ved or XVed: the
Ved buffer will grow much larger if every character is on a separate
line. I.e. it will need a string of length 1 per character and a
much longer vector for the ved buffer, which will keep growing.
You can fix that by making it clear the Ved buffer every now and
again, e.g.
vars x = 0;
repeat 5000 times
x+1->x;
if x mod 100 == 0 then
;;; clear Ved window and buffer
ved_clear();
;;; keep a record of the total number so far
pr(x);
else
charout(`0`+x mod 2);
endif;
charout(10);
;;; very short sleep time for testing!
syssleep(1);
endrepeat;
That worked as expected, printing 0 and 1 on alternate lines, and
every now and again the total number of cycles, just after
truncating the ved buffer. (I interrupted instead of waiting for
5000 cycles, of course.)
There's also the risk of integer overflow in x. It will eventually
become a biginteger, but that may be insignificant for you. If it is
a problem you can truncate x every now and again and have another
variable recording the number of truncations. (Though that just
postpones the problem.)
Yet another problem is that every time you print something on a new
line Ved updates its status line which will consume some time and
increase the number of garbage collections. But if you don't print
too often that may be insignificant. Clearing the Ved buffer will
also produce garbage.
None of this should be necessary. There should be a guaranteed way
for a program to wake up ved and make it display recent changes to
the ved buffer. There probably is, and it is probably different for
Ved and XVed, but I can't find it.
Another solution is for you to write to a disc file and have another
process interrogating the file at regular intervals. That avoids
the use of the Ved buffer and in the past I have seen programs
running much faster if they send trace output to a file rather than
to Ved. But that was a program with lots of output.
Aaron
Jonathan L Cunningham
> Jonathan L Cunningham wrote:
>
> > I've just run a program in Windows Poplog Common Lisp that ran for over
> > six weeks (mid January to early March) and used over 1175 hours of cpu
> > time, on a 1.8GHz pentium (and used 200MB of heap).[*]
Actually an AMD Athlon 1800+, running at 1.49GHz.
> That must be a record for poplog, though some people have used
> pop-11 in evolutionary computation experiments run on multiple cpus
> for a shorter time, which may have used as much cpu time and memory
> in total.
>
> > It was dealing with lists that grew to more than a million items long.
>
> Remember the days when we used to have to share a 256Kbyte PDP11/40
> between about 16 simultaneous users?
Yes. I was reminded of how a lecturer ended a course when I was an
undergraduate. As part of the last lecture he estimated the maximum size
of computation that could ever be performed: assuming the mainframe can
do one million instructions per second, and that it usually breaks down
and needs fixing (by calling out an engineer) every couple of days, we
can assume that one fortnight is the maximum practical limit. So that
would be about 10^6 * 10^6 = 10^12 instructions. (I don't remember if
this was the exact figure he derived - but you get the idea.)
The machine I used this year [brief google] is probably around 3000
VAX11/780 MIPS. (I once estimated, back in the 1980s, that 50 MIPS would
be enough for a machine to pass the Turing Test, if only we knew how to
program it. Possibly an underestimate!) So my run must have executed
around 3*10^9 * 1175.4 * 3600 = c. 10^16 instructions - and that is on a
home PC, not a university mainframe or a supercomputer!
A mere 10^12 instructions, the lecturer's estimate, would only take
about five or six minutes.
To return to the subject:
> > However, I did have a minor problem. In order to show that the program
> > hadn't crashed or was looping, I had it print out dots "." every 100th
> > item processed. (Which could take minutes.)
> >
> > The problem was that I forgot that output was buffered, so instead of
> > printing a dot every minute or so, it printed 100 dots every couple of
> > hours.
(snip)
> There are two problems here
>
> (a) getting the poplog process to flush its output buffer, which in
> pop11 you can do most easily with charout(0).
(snip)
> (b) If you have to output to Ved, then the second problem is getting
> the ved output file to re-display itself, which it normally will not
(snip)
Yes. Thanks for the suggestions.
The solution I will use will be to implement a little "progress
indicator" widget in my portable graphics library.
Yes, I have (finally!) returned to the problem of a portable graphics
layer for poplog that runs on Windows Poplog as well as Linux/Unix, and
I have now got some useful working code. I'll make it available when it
is more complete.
In case anyone is interested, some comments about the design:
(1) I previously made a number of false starts, which I have now
abandoned. The current implementation has got further, and I foresee no
problem completing it. (Famous last words!!)
(2) I decided that the ability to port/support LIB rc_graphic was *not*
going to be a design input. Instead, it is a design requirement. This is
a subtle, but important, difference. I am making design decisions
without any reference to any other libraries or graphic packages. I am,
of course, using *experience* gained with other packages - but this is
going to be a "clean" design. If it can't, later, be used to support
higher level packages (any others) then it is not a good design. I may
have to *add* additional features, but I expect the basic design to be
sound.
(3a) Roger Evans has previously mentioned that it would be nice to do
everything in pop11 (by which I think he meant user pop11, rather than
the system dialect) rather than relying on external libraries written in
other languages. I agree with the philosophy, but it was the basis of
one of my false starts. I now think it is an ideal to be strived for,
rather than something completely achievable. [NB this is not an
objection to importing pre-existing libraries written in other
languages: it's an ideal for writing *new* poplog libraries.]
(3b) There will be a (small) external library that will need to be
loaded (for the Windows version) - there are a couple of good reasons
for this: one reason is to make (4) simpler. The code in this external
library could, in theory, be mostly written as part of "core" poplog,
but I prefer to keep it separate: monolithic code is harder to maintain.
(4) Graphics windows will run in a separate OS thread. (All the
different graphic windows can share the same thread.) Certain events
will be handled in this thread: in particular, moving, resizing and
painting (drawing on) the window. The poplog thread will not draw on the
window: it will tell the window what needs to be drawn.
(5) There will be no callbacks from the external library into poplog.
This decision is the result of a lot of thought (and may still be
revised). It means that the poplog thread must poll for events. This is
not as draconian as it sounds: it avoid the problems (and dangers) of
code running in asychronous interrupts. Neither poplog, nor any of the
OSes it has been ported to, are hard real time systems. Waiting a few
microseconds won't hurt.
(6a) This does not mean that there are no *pop11* (or other poplog
language) callbacks. But these will all occur inside poplog, as part of
the main poplog thread.
(6b) On the poplog/pop11 side, there will be an object oriented package
(using LIB objectclass) that hides the implementation details. In
particular, it will allow the user to define event handlers and pop11
callbacks, and will set up one of four polling modes. The four polling
modes are:
(i) Explicit: user code must contain its own event loop which repeatedly
calls get_event() to get the next event (and then process it). It is not
expected that this mode would be used, but the mechanisms are needed
anyway, so they may as well be made available.
(ii) User controlled: user code must call process_pending_events() at
intervals, e.g. the beginning of a loop. This will be very fast if there
are no events pending. (I've used a system like this, and it works
well.) In this mode graphics windows will be "dead" when poplog is
waiting for input (but you will still be able to move them, exposed
windows will be painted -- any events which are handled in the graphics
thread itself will be processed). For example, you might click on a
button, and get visual feedback that the button has been clicked, but if
it has been attached to a pop11 function, that pop11 function will not
be called at that time: it will be called on the next call of
process_pending_events(). [Obviously in this mode you will need a
function to flush the event queue before running your user program.]
(iii) Timer interrupts: the function process_pending_events() is called
every millisecond (or user-defined interval) by a timer interrupt.
(iv) Implicit: the poplog compilers *already* plant code to poll for
asynchronous events. Most asynchronous events simply set a flag, because
it is not safe to run arbitrary code asynchronously. And the compilers
plant code to check this flag and run user code "interrupt handlers" at
times when it is safe to do so. (This is in the poplog documentation.)
It may be possible to use this mechanism directly, but if not, it would
be only a tiny change to core poplog source code (but would necessitate
rebuilding the core pop).
Rather a long post - I hope someone found it of interest!
Jonathan
Aaron Sloman
> Yes, I have (finally!) returned to the problem of a portable graphics
> layer for poplog that runs on Windows Poplog as well as Linux/Unix, and
> I have now got some useful working code. I'll make it available when it
> is more complete.
Sounds good.
> In case anyone is interested, some comments about the design:
>
> (1) I previously made a number of false starts, which I have now
> abandoned. The current implementation has got further, and I foresee no
> problem completing it. (Famous last words!!)
> ....
I expect you can do it!
> (2) I decided that the ability to port/support LIB rc_graphic was *not*
> going to be a design input. Instead, it is a design requirement. This is
> a subtle, but important, difference.
LIB rc_graphic is a very small subset of what is currently
available. If you meant LIB rclib that would be considerably
more useful! rclib uses rc_graphic plus a lot of other things.
The lower level (or more general) requirement might have been to
support everything that is in the Xpw interface (does that include
the stuff in REF xt_libs Xt on top of which is built rc_graphic and
rclib (rclib uses a lot more than rc_graphic does, though it does
not use motif which propsheet does, for example).
I expect, though you have not said so, that you have decided not
necessarily to replicate all of the Xpw functionality.
That may be OK if all the most widely useful higher level
functionality supported by Xpw (and LIB xt_libs ?) is supported.
> I am making design decisions
> without any reference to any other libraries or graphic packages. I am,
> of course, using *experience* gained with other packages - but this is
> going to be a "clean" design.
If you ever write a design requirements document it could be a good
thing to circulate, for various reasons.
> If it can't, later, be used to support
> higher level packages (any others) then it is not a good design. I may
> have to *add* additional features, but I expect the basic design to be
> sound.
>
> (3a) Roger Evans has previously mentioned that it would be nice to do
> everything in pop11 (by which I think he meant user pop11, rather than
> the system dialect) rather than relying on external libraries written in
> other languages.
That's exactly why I abandoned propsheet and other things requiring
motif and started work on rclib. But I had to rely on the interface
to X11 written in C, plus the pop11 libraries provided to go with
that.
> I agree with the philosophy, but it was the basis of
> one of my false starts. I now think it is an ideal to be strived for,
> rather than something completely achievable. [NB this is not an
> objection to importing pre-existing libraries written in other
> languages: it's an ideal for writing *new* poplog libraries.]
Yes. Some powerful, low level, facilities have to be provided
externally as a basis for what Pop-11 does, even if they come with
the operating system. (I don't know if the latter comment applies to
Windows or OS X, but in Linux/Unix there's a clear separation of OS
and graphical facilities).
> (3b) There will be a (small) external library that will need to be
> loaded (for the Windows version) - there are a couple of good reasons
> for this: one reason is to make (4) simpler. The code in this external
> library could, in theory, be mostly written as part of "core" poplog,
> but I prefer to keep it separate: monolithic code is harder to maintain.
This sounds like the Xpw philosophy. $usepop/pop/x/Xpw includes
about 11,000 lines of C code + comments, file headers and footers,
change notes, etc. in 15 .c files and 15 .ht files.
> (4) Graphics windows will run in a separate OS thread. (All the
> different graphic windows can share the same thread.) Certain events
> will be handled in this thread: in particular, moving, resizing and
> painting (drawing on) the window. The poplog thread will not draw on the
> window: it will tell the window what needs to be drawn.
>
> (5) There will be no callbacks from the external library into poplog.
> This decision is the result of a lot of thought (and may still be
> revised). It means that the poplog thread must poll for events. This is
> not as draconian as it sounds: it avoid the problems (and dangers) of
> code running in asychronous interrupts. Neither poplog, nor any of the
> OSes it has been ported to, are hard real time systems. Waiting a few
> microseconds won't hurt.
In RCLIB I tried to eliminate direct callbacks. Instead various
event handlers (button X up/down/drag, key X up/down, etc.) detect
the event and associated information (e.g. mouse location, whether
modifier keys are depressed), and report the events to be handled in
a queue. Eventually the queue manager passes the events to
appropriate objectclass methods.
There are still some problems when a graphical window associated
with a simulation (for example) can be changed either by a mouse
event or by a running program -- the program and the mouse can
specify contradictory moves. This can happen in the sheepdog demo
which uses SimAgent and rclib, shown in
http://www.cs.bham.ac.uk/research/projects/poplog/figs/simagent/
or the 'emotional agents' demo that is part of the test library for
use after installing linux poplog:
$usepop/bin/demos
> (6a) This does not mean that there are no *pop11* (or other poplog
> language) callbacks. But these will all occur inside poplog, as part of
> the main poplog thread.
That's very like the objective of RCLIB, except that in order to
implement that I had to use the existing callback mechanisms, to
get events put on the pop11 event queue.
I can't remember enough about RCLIB's event handling to see clearly
how it relates to your polling modes. I suspect I have a messy
subset of your ideas, and redoing it in your framework might clean
it up.
It mostly works as users expect, but occasionally moving objects
leave 'droppings' on the screen for reasons that I have not been
able to track down. It will be interesting to see if you can provide
a system that avoids that.
It may be connected with my decision to use 'xor' to make arbitrary
pictures movable. You will have to decide whether to follow that or
provide support for movable bitmaps, which will require a more
complex system for remembering things that have been covered.
Another serious cause of problems was the use of LIB rc_graphic,
which allowed commands like rc_drawline, rc_drawto, etc. to make use
of global variables (e.g. for origin, scale, current location,
etc.). That was very useful for writing simple picture-drawing code,
but getting all the context-switches right proved quite difficult.
> Rather a long post - I hope someone found it of interest!
At least one reader has.
In the past I've noticed that many people design graphical toolkits
mainly to support user controlled interactions, and without support
for program driven moving pictures. This can seriously limit their
use, e.g. in connection with simulation toolkits like SimAgent. I
assume your system will not have that limitation?
Aaron
Aaron Sloman
> ....
> The machine I used this year [brief google] is probably around 3000
> VAX11/780 MIPS. (I once estimated, back in the 1980s, that 50 MIPS would
> be enough for a machine to pass the Turing Test, if only we knew how to
> program it.
The following is not a criticism of Jonathan, but of the majority of
people who refer to 'the Turing Test'.
The prediction that Turing actually made in his 1950 article came
true before the end of the last century. This is what he wrote:
It will simplify matters for the reader if I explain first my
own beliefs in the matter. Consider first the more accurate form
of the question. I believe that in about fifty years' time it
will be possible, to programme computers, with a storage
capacity of about 10^9, to make them play the imitation game so
well that an average interrogator will not have more than 70 per
cent chance of making the right identification after five
minutes of questioning. The original question, "Can machines
think?" I believe to be too meaningless to deserve discussion.
Available online in various places, e.g.:
http://www.abelard.org/turpap/turpap.htm
http://www.loebner.net/Prizef/TuringArticle.html
(has some OCR errors)
I assume he meant 10^9 bits = about 125Mbytes. Lots of PCs had more
than that by 2000.
I suspect something not very much more complex than the Pop-11 eliza
is needed to fool 70 per cent of 'average interrogators' for five
minutes. Of course, knowledgable people can insert probe sentence
sequences that will identify a current AI system as non-human within
a much shorter time than five minutes.
Many people mistakenly believe that Turing was proposing his
scenario as a serious *test* for whether a machine can think, or
whether a machine is intelligent. Turing rightly regarded the
question as too ill-defined for any test to be able to answer it.
He proposed the test only as a basis for considering and rebutting
objections to his claim that it would be passed within 50 years.
(1) The Theological Objection
(2) The "Heads in the Sand" Objection
(3) The Mathematical Objection
(4) The Argument from Consciousness
(5) Arguments from Various Disabilities
(6) Lady Lovelace's Objection
(7) Argument from Continuity in the Nervous System
(8) The Argument from Informality of Behaviour
(9) The Argument from Extrasensory Perception
Of course what an 'average interrogator' knows can change as the
general population becomes better educated. Right now the level of
education about computers and AI in the population at large is still
generally so poor that Turing's prediction is probably still true,
though it might become false if stated about some future time!
At present most people know only how to use computers for a few
information-manipulation tasks, and know nothing about how the
software works or how it can be tested, improved, etc.
I have a proposed a new test in this paper
http://www.cs.bham.ac.uk/research/projects/cosy/papers/#tr0705
COSY-TR-0705 (PDF)
Why Some Machines May Need Qualia and How They Can Have Them:
Including a Demanding New Turing Test for Robot Philosophers
Paper for AAAI Fall Symposium 2007
It's a test not for a specific machine but for an AI *design*. It
should be possible for the same design to be implemented in two
identical machines (young robots) that 'grow up' to hold opposed
views on *every* substantive philosophical question about minds and
bodies, minds and machines, etc.
E.g. the generic design should make it possible for a robot to
develop (possibly after going to school and university) so as to
hold philosophical views like Stevan Harnad, or David Chalmers, or
like Daniel Dennett (or even like me -- I disagree with all of
them). Likewise such a machine could be capable of growing up to be
like Richard Dawkins or like a creationist who believes in
'Intelligent Design'.
However, different external influences are allowed.
It need not be possible to predict how the machine will develop in
any particular environment, since some developmental trajectories
could be significantly influenced by decisions made partly on a
random basis, e.g. whether to go to hear an advertised lecture or go
to see a popular movie.
A presupposition of the paper is that machines with human-like
intelligence will be capable of getting into human-like muddles,
confusions, and wickedness -- as a side-effect of using their
intelligence to make sense of the world, including learning from
others, etc.
So the frequently quoted goal of producing 'human-level AI' may not
be a good engineering goal, if we can do better than that. However,
it is conceivable that products of biological evolution have already
hit some theoretical limit in intelligence. I doubt it.
Jonathan L Cunningham
> Jonathan L Cunningham wrote:
> > (2) I decided that the ability to port/support LIB rc_graphic was *not*
> > going to be a design input. Instead, it is a design requirement. This is
> > a subtle, but important, difference.
>
> LIB rc_graphic is a very small subset of what is currently
> available. If you meant LIB rclib that would be considerably
> more useful! rclib uses rc_graphic plus a lot of other things.
Yes, I meant rclib.
> > I am making design decisions
> > without any reference to any other libraries or graphic packages. I am,
> > of course, using *experience* gained with other packages - but this is
> > going to be a "clean" design.
>
> If you ever write a design requirements document it could be a good
> thing to circulate, for various reasons.
There will be a REF file - which could be used as the basis for a
software specification document (e.g. for porting to Linux or to sit on
other graphical/UI packages/platforms). And I'll think about whether I
can prepend an overview (about the objectives and purpose) which might
be used as a basis for a requirements doc too.
[Pre posting note: it occurs to me that, when I've written the
specification, it might be interesting to turn the problem around.
Instead of asking how rclib might be implemented on top of it, ask how
hard it would be to implement on top of rclib! Which is not to say that
would be the best way to do it, but it would be an interesting
question.]
> > (3b) There will be a (small) external library that will need to be
> > loaded (for the Windows version) - there are a couple of good reasons
> > for this: one reason is to make (4) simpler. The code in this external
> > library could, in theory, be mostly written as part of "core" poplog,
> > but I prefer to keep it separate: monolithic code is harder to maintain.
>
> This sounds like the Xpw philosophy. $usepop/pop/x/Xpw includes
> about 11,000 lines of C code + comments, file headers and footers,
> change notes, etc. in 15 .c files and 15 .ht files.
I don't anticipate getting anywhere near 11,000 lines. I still want to
do as much as possible in pop11. On Windows, it is all running in the
same address space anyway. I'm using C++ for a pretty low-level object
layer, and I'm using C as glue on both sides of this layer: on one side,
as the api to communicate with the pop11 procedures, and on the other to
interface to the OS.
Astute readers will notice that this implies I am not using Microsoft's
own object encapsulation of their UI (the MFC, Microsoft Foundation
Classes, in C++, nor the newer .NET stuff).
Most of the abstraction (for user programs) will be done in objectclass,
and written in pop11. There isn't even a 1-1 correspondence between
pop11 objects and the C++ objects in my external layer. (Because my
pop11 abstractions do not correspond closely to the Microsoft API, so
some translation is needed.) It remains to be seen how easy it is to
use, and how well it will work.
> There are still some problems when a graphical window associated
> with a simulation (for example) can be changed either by a mouse
> event or by a running program -- the program and the mouse can
> specify contradictory moves. This can happen in the sheepdog demo
> which uses SimAgent and rclib, shown in
> http://www.cs.bham.ac.uk/research/projects/poplog/figs/simagent/
This problem can be avoided my using the 2nd "polling mode" I referred
to in my previous post, instead of the 3rd polling mode.
I am assuming that it is a requirement that the user can interact with
and move objects around (with the mouse) while they, themselves, are
moving.
Specifically, instead of starting the timer-based polling, you put a
call to pg_process_pending_events() somewhere in your main loop, and you
do *not* cache the positions of objects, but only update from their
current position. I.e. [note: not all the function names are genuine --
this is just for illustration] do this:
;;; using polling mode 2
repeat
pg_process_pending_events();
pg_position(my_animated_object) -> (x, y);
new_position(x, y) -> pg_position(my_animated_object);
endrepeat;
and not this:
pg_position(my_animated_object) -> (x, y);
repeat
pg_process_pending_events();
new_position(x, y) -> (x, y);
(x, y) -> pg_position(my_animated_object);
endrepeat;
That should be obvious, I think!
Also, this is how you could do it if you don't have to worry about the
user moving the object independendly of program control:
;;; using polling mode 3
pg_timerpoll();
repeat
pg_position(my_animated_object) -> (x, y);
new_position(x, y) -> pg_position(my_animated_object);
endrepeat;
If the user does interact/move the object, this could cause problems if
the timer interrupt goes off after the first assignment and before the
second.
You can't use polling mode 4, either, unless you set some compiler flags
telling the compiler not to plant code to check for interrupts in the
body of your function "new_position" nor any function it calls. How to
do that is in the existing poplog documentation. If you do that, then
the polling mode 4 code is just:
;;; using polling mode 4
repeat
pg_position(my_animated_object) -> (x, y);
new_position(x, y) -> pg_position(my_animated_object);
endrepeat;
i.e. you don't need to start the timer, and you don't need to explicitly
call the process_pending_events function. However, modes 2 and 3 both
work fine (I've tested them), require only one line of extra code, and
are simple to understand. So implementing mode 4 is not a high priority.
> I can't remember enough about RCLIB's event handling to see clearly
> how it relates to your polling modes. I suspect I have a messy
> subset of your ideas, and redoing it in your framework might clean
> it up.
I included the above partly to explain how my polling modes work. I
think that should be fairly clear now. If not, please say so, as I will
need to make it clear in the documentation.
> It mostly works as users expect, but occasionally moving objects
> leave 'droppings' on the screen for reasons that I have not been
> able to track down. It will be interesting to see if you can provide
> a system that avoids that.
>
> It may be connected with my decision to use 'xor' to make arbitrary
I am not using 'xor' to erase objects at all. I'm aware of the problems
this causes.
The approach I am using is intrinsically safer and more robust.
When you update the position of an object, it tells the window that
everything in its old position needs to be redrawn, and everything in
its new position needs to be redrawn. At some convenient time, but
before the window is repainted, the OS erases both rectangles, which may
overlap, and then asks all the affected objects to redraw themselves.
This is more expensive than the old 'xor' approach - but machines are
much faster nowadays. The worst that will happen will be some
flickering, and I will eventually implement double-buffering (in the C++
layer) to prevent that. It won't be the first thing I do though! :-)
I'm assuming you won't have more than a few hundred simple shapes to be
drawn: this interface is not really targeted at implementing real time
3D games! :-) :-)
[And I have lisp code, which already does double-buffering at the Lisp
layer. [Note to self: I will need to make sure it is possible to do this
from pop11.] I hope to use this as test code even before I provide
"double-buffered" as an optional window attribute.]
Presumably, in RCLIB, it is possible to draw into an offscreen window or
buffer, and then "map" or copy that to a visible window in a single
operation? So double-buffering could be used there.
> pictures movable. You will have to decide whether to follow that or
> provide support for movable bitmaps, which will require a more
> complex system for remembering things that have been covered.
No, the approach I describe is very simple. Yes, it will be possible to
draw bitmaps, and move them - e.g. by dragging them around with the
mouse. The existing mechanisms should "just work" as soon as I implement
bitmap drawing. Hmmm. I think I'll do bitmaps today :-)
> Another serious cause of problems was the use of LIB rc_graphic,
> which allowed commands like rc_drawline, rc_drawto, etc. to make use
> of global variables (e.g. for origin, scale, current location,
> etc.). That was very useful for writing simple picture-drawing code,
> but getting all the context-switches right proved quite difficult.
It's easier to prototype that way: at first, anyway! :-) I do have a
couple of global variables, but they are for "global" things that won't
change on a per-window basis, and I hope I've not given myself extra
work. Most things hang off properties, or are encapsulated as objects.
Global variables are a danger I'm aware of.
> In the past I've noticed that many people design graphical toolkits
> mainly to support user controlled interactions, and without support
This is, also, one of my primary objectives.
> for program driven moving pictures. This can seriously limit their
> use, e.g. in connection with simulation toolkits like SimAgent. I
> assume your system will not have that limitation?
I want to be able to display moving/changing data, e.g. animations, and
animated "control" displays. I would think that, from the graphical
interface aspect, simulations are a subset of animations in general?
It's a requirement I'll keep in mind though, in case there is something
I'm overlooking.
Jonathan
Aaron Sloman
On Mon, 10 Mar 2008 Jonathan L Cunningham wrote:
> > > (2) I decided that the ability to port/support LIB rc_graphic was *not*
> > > going to be a design input. Instead, it is a design requirement. This is
> > > a subtle, but important, difference.
> >
> > LIB rc_graphic is a very small subset of what is currently
> > available. If you meant LIB rclib that would be considerably
> > more useful! rclib uses rc_graphic plus a lot of other things.
>
> Yes, I meant rclib.
I still have not taken in all the design information from your
message, because of pressures of other work. But I did notice one
point.
I wrote this about RCLIB
[AS]
> > It mostly works as users expect, but occasionally moving objects
> > leave 'droppings' on the screen for reasons that I have not been
> > able to track down. It will be interesting to see if you can provide
> > a system that avoids that.
> >
> > It may be connected with my decision to use 'xor' to make arbitrary
> > pictures movable.
[JLC]
> I am not using 'xor' to erase objects at all. I'm aware of the problems
> this causes.
>
> The approach I am using is intrinsically safer and more robust.
The main reason I used XOR was to allow movable pictures to have any
shape at all including having holes. This allows sliders to be
diagonal or circular for example, instead of being restricted to
horizontal and vertical. In fact, any 2-D picture that can be drawn
by a collection of RC_GRAPHIC commands can be made movable, though
movements can look odd when such a picture moves over something that
is not white, because of the use of XOR rather than re-painting bits
of the window.
[JLC]
> When you update the position of an object, it tells the window that
> everything in its old position needs to be redrawn, and everything in
> its new position needs to be redrawn. At some convenient time, but
> before the window is repainted, the OS erases both rectangles, which may
> overlap, and then asks all the affected objects to redraw themselves.
>
> This is more expensive than the old 'xor' approach - but machines are
> much faster nowadays. The worst that will happen will be some
> flickering, and I will eventually implement double-buffering (in the C++
> layer) to prevent that.
I could not tell from that whether you were assume that all picture
objects are rectangular or whether the rectangle is just the
bounding box of a picture.
If I have understood this correctly it requires the system to
remember which objects are behind which others, so that re-painting
is done in the right order. In RCLIB the objects in a window are
totally ordered (though the same object can be shown in more than
one window, and can be covered in one window and visible in
another.)
It might be better to have a partial ordering, so that, for example,
it is specified that the long horizontal object is in front of the
two shorter objects, but the relative depths of the two shorter
objects is not specified:
|--|
| | |--|
|-------------------------------|
|-------------------------------|
|--| |--|
I have not thought this through, however.
My main reason for using XOR was not efficiency but ease of
implementation. Various things proved easy that would otherwise have
been much more difficult.
> I'm assuming you won't have more than a few hundred simple shapes to be
> drawn: this interface is not really targeted at implementing real time
> 3D games! :-) :-)
At present RCLIB is mostly used with SimAgent which is intended to
allow simulated agents with arbitrarily complex individual
architectures, such as might one day be needed for human-like
robots. If a complex 3-D robotic environment with many moving
objects is to be simulated, it may be better to use an external
dedicated 3-D physical simulation package as some people have
already done. For his PhD work, Nick Hawes used an external
graphical game engine (Unreal Tournament) connected to pop11, with
SimAgent adding intellgience (planning capabilties) to the game
agents.
Mattheus Scheutz uses RCLIB with SimAgent in his Simworld package to
explore evolutionary processes. He typically has worlds with lots of
simple agents interacting with other things like food, obstacles,
etc. However, I think that so far he has used only 2-D worlds where
objects cannot move through or over other objects. I expect he has
never had more than a few hundred simple shapes because of the
resource requirements of evolutionary experiments. (He can run his
simulations with the graphics turned on or off.)
[JLC]
> Presumably, in RCLIB, it is possible to draw into an offscreen window or
> buffer, and then "map" or copy that to a visible window in a single
> operation? So double-buffering could be used there.
It is possible to draw into a window that is not visible either
because it is unmapped or because it is located beyond the visible
area of the screen. I assume the latter is a lot more expensive?
Mapping and unmapping are handled by the X window interface. I
pressume there are copying operations of the sorts that David
Young's popvision library uses, though I have not provided any
specific RCLIB tools to use them.
One of the things I have found extremely annoying on my wife's
Windows XP machine is that no window can be moved partly off the
screen to show more of a partly covered window. Is that because
microsoft programmers did not know how to implement that? I use that
all the time with the X window system on Linux: it is FAR more
convenient than having to resize a window to show more of another
window. But it does require being able to move a window when its
title bar is off the screen. I use ALT+Left button for that.
I hope your system will not inherit such MSwindows restrictions.
Jonathan L Cunningham
> Apologies for delayed response:
No need!
> On Mon, 10 Mar 2008 Jonathan L Cunningham wrote:
> [JLC]
> > I am not using 'xor' to erase objects at all. I'm aware of the problems
> > this causes.
> >
> > The approach I am using is intrinsically safer and more robust.
>
> The main reason I used XOR was to allow movable pictures to have any
> shape at all including having holes. This allows sliders to be
This is even simpler with my approach.
> diagonal or circular for example, instead of being restricted to
> horizontal and vertical. In fact, any 2-D picture that can be drawn
> by a collection of RC_GRAPHIC commands can be made movable, though
> movements can look odd when such a picture moves over something that
> is not white, because of the use of XOR rather than re-painting bits
> of the window.
I'm not sure why you think XOR makes this more possible. The approach I
am using is to draw, when something moves, everything in its new
position.
That is *everything*. It doesn't matter whether they have holes,
transparent interiors or what. If *everything* gets redrawn, then all
that matters is that objects can draw themselves.
Of course, you only need to redraw things that have changed -- but you
can err on the side of caution. If in doubt, draw every single thing
which is visible anywhere in the window.
Actually, most windowing systems have a "clipping region". Drawing an
object which is not inside the clipping region is very cheap. So you
*can* redraw everything, as a first approximation. That's (currently)
what I'm doing. It works fine. At some point, I will have to see how
many objects it takes to make it slow down enough to be noticeable.
> [JLC]
> > When you update the position of an object, it tells the window that
> > everything in its old position needs to be redrawn, and everything in
> > its new position needs to be redrawn. At some convenient time, but
> > before the window is repainted, the OS erases both rectangles, which may
> > overlap, and then asks all the affected objects to redraw themselves.
> >
> > This is more expensive than the old 'xor' approach - but machines are
> > much faster nowadays. The worst that will happen will be some
> > flickering, and I will eventually implement double-buffering (in the C++
> > layer) to prevent that.
>
> I could not tell from that whether you were assume that all picture
> objects are rectangular or whether the rectangle is just the
> bounding box of a picture.
The bounding box is what is passed to the OS as needing to be redrawn. I
could just mark the entire window as needing to be redrawn, but by using
the bounding rectangle, it becomes cheap to redraw things which are
outside the area affected (because the OS ignores drawing commands in
those areas, and only draws what needs to be drawn).
> If I have understood this correctly it requires the system to
> remember which objects are behind which others, so that re-painting
> is done in the right order.
I am not sure what you are saying here. Yes, the objects are drawn
oldest first, unless the Z-order is explicitly changed.
So if you want a rectangle around some text, then you must draw an
opaque rectangle before you draw the text. On the other hand, if you
draw a rectangle with a transparent interior, it doesn't matter which
order you draw the text and the rectangle.
> In RCLIB the objects in a window are
> totally ordered (though the same object can be shown in more than
> one window, and can be covered in one window and visible in
> another.)
Objects are totally ordered in my system, too.
> It might be better to have a partial ordering, so that, for example,
> it is specified that the long horizontal object is in front of the
> two shorter objects, but the relative depths of the two shorter
> objects is not specified:
>
> |--|
> | | |--|
> |-------------------------------|
> |-------------------------------|
> |--| |--|
>
> I have not thought this through, however.
A few years back, I was trying to write a game (in C++) with little
animated robots running around, and the test for whether one object was
in front of another (occluded it) was very complex. I forget the details
- I must dig out the old code and have another look at it, to remind
myself of the issues.
Since the objects moved about, the ordering needed to be updated as
they moved.
I do remember that I wrote a routine called "posort" (partial-order
sort). But I can't remember why it was necessary! Possibly because you
could have not(a < b) and not(b < a), yet (a < c < b). Or something.
(E.g. neither a occludes b, nor b occludes a, but a occludes c, and c
occludes b. So they must be drawn in the order c,b,a. Most sort routines
assume that you can directly compare a and b ...)
I must dig out the old code and have a look at the comments in it. :-)
> One of the things I have found extremely annoying on my wife's
> Windows XP machine is that no window can be moved partly off the
> screen to show more of a partly covered window.
Is that in some particular application? I've never experienced this
restriction with anything I've done in Windows XP.
> Is that because
> microsoft programmers did not know how to implement that? I use that
> all the time with the X window system on Linux: it is FAR more
> convenient than having to resize a window to show more of another
> window. But it does require being able to move a window when its
> title bar is off the screen. I use ALT+Left button for that.
>
> I hope your system will not inherit such MSwindows restrictions.
There is no such restriction that I'm aware of. You can't (I believe)
move a window *completely* off-screen. But that makes sense: how would
you get it back? (Yes, the Linux/unix multiple desktops is a much better
idea - but given a single desktop, it makes sense not to lose windows
permanently.)
It's possible that there is an "easy" or "beginner" mode preventing
windows being moved. If so, it can be turned off.
Jonathan
Aaron Sloman
> > diagonal or circular for example, instead of being restricted to
> > horizontal and vertical. In fact, any 2-D picture that can be drawn
> > by a collection of RC_GRAPHIC commands can be made movable, though
> > movements can look odd when such a picture moves over something that
> > is not white, because of the use of XOR rather than re-painting bits
> > of the window.
>
> I'm not sure why you think XOR makes this more possible.
I misunderstood your reference to rectangles, I think.
> The approach I
> am using is to draw, when something moves, everything in its new
> position.
You also have to deal with things where the object was before it
moved, to make sure hidden things reappear. That's not a problem if
you redraw the whole window, but if you redraw only parts near the
object you have to worry about this.
With XOR, I simply draw the previously drawn object, which removes
it from that part of the window, then draw it in the new location.
Nothing else has to be redrawn.
Admittedly the result can look odd, because of the effects of XOR
where things Overlap. So it is not a good method in general.
On reflection, I am not sure it copes properly with multiple objects
added and removed in arbitrary orders. I guess it must work, because
XOR is commutative, but it feels a bit like getting extra memory for
nothing, magically!
> That is *everything*. It doesn't matter whether they have holes,
> transparent interiors or what. If *everything* gets redrawn, then all
> that matters is that objects can draw themselves.
Yes. The difference is that when I move something nothing else has
to be re-drawn.
>...
> > In RCLIB the objects in a window are
> > totally ordered (though the same object can be shown in more than
> > one window, and can be covered in one window and visible in
> > another.)
>
> Objects are totally ordered in my system, too.
I forgot to add that selecting an object with the mouse, by default
brings it to the top. I assume that is true for you too.
I also found that when an object has been selected for dragging,
then during dragging (i.e. until the mouse key goes up) other
objects have to be ignored even if the mouse is over them, because
the dragged object doesn't always keep up with the mouse if it is
being moved quickly. I presume you also have to do that.
> ....
> > One of the things I have found extremely annoying on my wife's
> > Windows XP machine is that no window can be moved partly off the
> > screen to show more of a partly covered window.
>
> Is that in some particular application? I've never experienced this
> restriction with anything I've done in Windows XP.
Apologies -- my mistake, partly.
I don't use Windows very often and I had forgotten the precise
problem. It is true that you can move a window partly off the screen
sideways or downwards, but not upwards (except a little), because
the only way to move it (as far as I know) is by dragging the title
bar, and so when that gets to the top of screen, or at least the bit
the mouse is on, you cannot move it higher. So re-sizing is the only
way to reveal something lower down the screen. But often I only want
to see the lower portion of a window, so I want to move it upwards
off the screen.
Maybe there is a way to move a window in XP without dragging the top
bar. Nothing I tried worked.
> There is no such restriction that I'm aware of. You can't (I believe)
> move a window *completely* off-screen. But that makes sense: how would
> you get it back?
On linux I can give a pop11 command to move a pop11 window to a
location right off the screen and another to bring it back in view.
But I agree that in general you can't bring back someting that has
moved off.
> (Yes, the Linux/unix multiple desktops is a much better
> idea - but given a single desktop, it makes sense not to lose windows
> permanently.)
Someone told me that Windows has an optional multi-desktop package
on a microsoft games/toys (?) web site, but I have not tried it.
It's limited to 4 desktops. Currently I use up to 10 when I am
working.
Cheers.
Aaron
Jonathan L Cunningham
> Jonathan:
>
> > The approach I
> > am using is to draw, when something moves, everything in its new
> > position.
>
> You also have to deal with things where the object was before it
> moved, to make sure hidden things reappear. That's not a problem if
> you redraw the whole window, but if you redraw only parts near the
> object you have to worry about this.
It's not that difficult.
I think the crucial thing here is that you don't think of terms of
drawing objects, but in terms of areas that need to be drawn on.
So you don't draw *anything* when an object moves.
It's simpler to illustrate by writing some pseudo-code. Every object
has associated with it (usually it inherits) a method for marking the
area of its current location as "needs redrawing". Typically this will
be its bounding rectangle. Suppose this function is called "redraw()",
then the pseudo-code for moving an object is:
redraw();
moveto(new_position);
redraw();
This marks both the old and new positions as being in need of redrawing,
although no drawing takes place. By the time the first area gets
redrawn, the object will have moved, and only what is still there gets
drawn.
[Implementation note: this needs to be done in the "graphics" thread,
not the "application" thread, so if the application wants to move an
object, it passes a "move object" request to the graphics thread. A
minor detail, but important :-). This is handled by the pop11 side of
the library, of course. As far as user code is concerned, you just
assign to the updater of an object's position:
(23, 60) -> position(obj); ;;; move obj to window coords (23, 60)
]
Note that this works: (a) even if the object teleports half-way across
the screen, (b) does nothing if the window is not visible, (c) works
even if the object was off-screen and is moved onto the screen, (d) and
vice-versa, if a visible object is moved off-screen, (e) works if
objects are occluded by other objects, including objects with holes, or
the object being moved is between (in the Z direction) other objects,
(f) works even if the old position and the new position overlap.
Bottom-line: it works correctly in all cases. You don't even need to
think about special cases.
You need to be able to redraw (parts of) the window anyway (e.g. if the
user moves overlapping windows), so this approach requires, only about
four or five lines of code to define the redraw() function (most of that
is copying the coordinates of the bounding rectangle into a C data
structure, to pass them to a system call.)
As I said, it's conceptually very simple and robust, and not
particularly expensive, either. In fact, if you have a *lot* of objects
moving, it can even be cheaper than XOR etc. because nothing at all gets
drawn until the OS decides it can spare the time, and then it will
combine all the "update" regions and draw everything that needs to be
drawn only once.
So, for example, on a slow machine and objects with very slow drawing
procedures, instead of drawing every mouse move event, the machine might
only have time to draw every second or third mouse move -- this all
happens automatically -- and, crucially, *gets it right*. There can be
no bits left behind, and the application won't miss any mouse moves
(e.g. drawing a curve freehand with the mouse, you won't get straight
bits because of lost points).
> > That is *everything*. It doesn't matter whether they have holes,
> > transparent interiors or what. If *everything* gets redrawn, then all
> > that matters is that objects can draw themselves.
>
> Yes. The difference is that when I move something nothing else has
> to be re-drawn.
But this can actually work out more expensive if you are moving multiple
objects, because you have to draw and undraw objects, whereas the simple
approach I describe, as well as being more robust, only draws each
object (of those which need to be drawn) once. And only when the main
application can spare the time. (This would not be so easy to do in a
single-threaded application.)
But efficiency is not the main concern: simplicity and robustness are
the reasons for the design decision. I can be confident that there won't
ever be "bits" left behind, when an object is moved, as long as an
object's bounding rectangle is computed correctly (which is easy).
It's because, in the past, I've used the XOR approach to dragging
objects that I decided against it this time.
> > Objects are totally ordered in my system, too.
>
> I forgot to add that selecting an object with the mouse, by default
> brings it to the top. I assume that is true for you too.
Actually, it isn't.
Why should clicking on an object bring it to the front? (Genuine
question.)
As currently implemented, everything "draggable" under the mouse gets
dragged[*], passing in front of or behind other objects. So I could drag
an image of a man along behind an image of a fence, or behind a window
in a house, and it would work correctly. (See above re robustness.)
I suspect that dragging behind other objects would be tricky using the
old XOR approach...
[*] Which surprised me a little, when I tested it. I moved two objects
so that they overlapped, and then tried "dragging" from the overlapped
region to see which one moved -- and they both moved. After at least
two second's reflection, I decided this was correct. I was only testing
the drag code, and it is up to the application what it wants to happen.
If the application wants to let the user drag multiple objects
simultaneously, it can. Visually, it looks fine.
> I also found that when an object has been selected for dragging,
> then during dragging (i.e. until the mouse key goes up) other
> objects have to be ignored even if the mouse is over them, because
> the dragged object doesn't always keep up with the mouse if it is
> being moved quickly. I presume you also have to do that.
Um, no. But I'm not using X-Windows. Everything happens (so far) very
fast and smoothly (it's all in the same address space). But I am doing
the dragging in the C++ library: the pop11 code just specifies what can
be dragged (i.e. what should move when the mouse moves). I don't know
what delays there would be using X - but the approach is robust: it
should still work correctly.
I hasten to add that this is nowhere finished yet! Maybe it will end up
as 11,000 lines of C++ code after all (as well as associated pop11
libraries).
But I'm still happy with the way it is shaping up. :-)
Jonathan
Chris Dollin
(late to the ball, and an aside, but still ...)
> Why should clicking on an object bring it to the front? (Genuine
> question.)
It shouldn't.
(The important X customisations I do are (a) focus follows the
/mouse/, thanks and (b) click does /not/ raise a window, thanks,
and (c) /this/ and /that/ and /t'other/ keystrokes do window
raise/lower/iconise.
My Mac drives me nuts because there's no standard way to
turn off its sticky focus and click-raising.
Don't get me started on how much space is taken up by
window dressing in, say, Eclipse, which is bearable on
a big screen but incapacitating on an EEE PC ...
... and yes, I do remember the days of 24x80 VTnnn
terminals ...)
--
Far-Fetched Hedgehog
"Never ask that question!" Ambassador Kosh, /Babylon 5/
Jonathan L Cunningham
> On Monday 17 March 2008 14:11, Jonathan L Cunningham wrote:
>
> (late to the ball, and an aside, but still ...)
>
> > Why should clicking on an object bring it to the front? (Genuine
> > question.)
>
> It shouldn't.
>
> (The important X customisations I do are (a) focus follows the
> /mouse/, thanks and (b) click does /not/ raise a window, thanks,
> and (c) /this/ and /that/ and /t'other/ keystrokes do window
> raise/lower/iconise.
>
> My Mac drives me nuts because there's no standard way to
> turn off its sticky focus and click-raising.
Jeff Best bought a Mac for the Open Poplog project.
It has converted me from being a pro-Mac person to thinking that maybe
Windows isn't so bad after all ... :-)
And it's not being able to (easily) find out how to turn off or
customise things that are there for the beginners and "ordinary" users
that has done that.
The Mac implementors seem to have a clear vision for how things Should
Be Done - and that's good. But if you don't agree that This Is How It
Should Be Done, it's not so good.
I don't use the Open Poplog Mac enough to know if I could be persuaded
to change my attitude... but I suspect not. I'd rather spend my time
beating Windows into abject submission. At least I've got that OS
cornered: its defeat and surrender is inevitable.
Jonathan