misc/RFC: design of a GUI API
BGB / cr88192
will approach this.
basically, I am considering the issue of how I will approach adding GUI
facilities to my x86-interpreter-based world.
firstly, the core issues (as I see them):
there needs to be a separate protocol and GUI API (I am currently leaning
against the use an RPC-based design, nor am I feeling inclined to marshall a
huge number of API calls as system-calls).
I don't feel X11 is an ideal choice (although, I may look into it and see if
any of it can be used). in particular, since I am aiming at a higher level
of abstraction (Widgets), and for my uses will not be using a raster-based
backend, I am not sure if X11 is ideal (could include extra unecessary
baggage).
similarly, I WILL be writing the "server", this much I know already (X11
would not save me much here).
however, like X11 I may want to consider network transparency as a possible
goal.
it is likely to operate over local sockets, however, I want to keep this
"open" (such that new "channels" may be added, possibly to redirect the
command-stream back into the app), as well I want to be able to provide an
option so that shared-memory may be used (my current thinking is that
shared-memory would allow some tasks to be performed more efficiently than
would sockets).
likely the basic structure will be based arounds "messages", however, some
messages may contain "linear command streams" (basically, packed sequences
of commands expected to be done all at once).
likely, datagram sockets will be assumed (with 1 or more messages per
datagram), but stream sockets could be used by wrapping the datagrams in
headers (or sending them directly). potentially, very large messages could
be sent (such as containing an entire raster image), which could be handled
via multiple datagrams (in the datagram case, for example, over UDP), or by
cutting up the message (to help avoid plugging up the stream in the stream
case).
the protocol would likely require ordered reliable delivery, which would
require special handling in a network+UDP case (that or, for an actual
network, only TCP is used).
for the API, I am considering combining aspects I like from:
OpenGL, GDI, GTK, AWT, ...
in particular:
it will likely use abstract handles (like GDI) and abstract getters/setters
(like OpenGL);
it will be Widget-centric (like GDI, GTK, and AWK), but will likeky support
box-packing, ... (like GTK and AWT).
specifics:
I may use part of OpenGL as a template, and so likely the command stream
would marshall both GUI and OpenGL commands, and likewise, some OpenGL API
calls could be used for implementing some of the GUI API calls;
however, I don't want OpenGL to be a "required" component.
(even if not used, directly, I am thinking of using many of GL-like
abstractions for managing resources).
possibly, I may look into OpenGL ES in order to reduce the total amount
needed to be written (vs full OpenGL, which is, well, massive...).
other resources likely to be handled would include keyboard and mouse, and
likely also audio, ...
also of note:
audio will not be, necessarily, raw PCM, but may include a sample-based
mixer, and likely also MIDI (where a MIDI command-stream is likely to be
used to control the mixer). (hell, I built a partial TTS engine on top of
MIDI, so it should work OK for mixer control).
for parts outside of MIDI (such as sending audio samples, ...), likely
GL-like commands will be used (hmm, 'glTexImage1D'...).
MIDI sequences would likely be embedded in messages.
I "could" consider an extension to allow more channels and controllers (the
MIDI bytestream is natually limited here, but like with "patches", could be
expanded via "banking").
(note: I already have a MIDI-synth in my codebase, so I can reuse it
here...).
raw PCM may also be provided.
I would likely want to investigate more before settling on an exact set of
encodings, but current leaning:
Message {
u16le msg_tag;
u16le msg_opt;
u32le szdata;
byte data[szdata];
}
with msg_tag identifying the specific message type, msg_opt giving a
message-specific field, and data any message-specific data (a header, packed
command stream, ...).
likely, for packed command streams:
<tag:u16> <tag-specific-data*>
also possible is variable-length tags (likely UTF-8 / Matroska-like coding),
but I am not certain if it would be worth the added decoding overhead
(probably should be ok).
I may have to check to make sure there is not already a command-stream for
GL ES (something like this may well already exist).
probably messages would be end-to-end in a datagram (terminated with a NULL
message).
split messages would be handled in a special split-message message (likely
with 'opt' holding a multiplex-id and sequence number).
...
any comments?...
Rod Pemberton
"BGB / cr88192" <cr8...@hotmail.com> wrote in message
news:he6vf5$r18$1...@news.albasani.net...
> well, I am idly thinking of things, and starting to consider ideas for how
I
> will approach this.
>
> basically, I am considering the issue of how I will approach adding GUI
> facilities to my x86-interpreter-based world.
>
> I don't feel X11 is an ideal choice
>
I've mentioned Depui GUI a few times. I'm not sure if it's developed enough
for what you intend.
Depui
http://www.deleveld.dds.nl/depui33/depui.htm
His main page with a couple "typesafe" projects:
http://www.deleveld.dds.nl
I've considered the *idea* of combining the low-level functionality of an OS
with LibSDL for the higher level functionality. It's not in my current
plans. You could then choose one of many LibSDL games or OS-like
applications (DOSBox, DOSEMU) as your OS. Instead of multibooting Linux,
Windows, DOS, etc, you could multiboot DOOM, Quake4, MAME, DOSBox, etc.
Sweet? I think so...
http://en.wikipedia.org/wiki/List_of_games_using_SDL
Rod Pemberton
BGB / cr88192
"Rod Pemberton" <do_no...@nohavenot.cmm> wrote in message
news:he76cu$a0c$1...@aioe.org...
>
> "BGB / cr88192" <cr8...@hotmail.com> wrote in message
> news:he6vf5$r18$1...@news.albasani.net...
>> well, I am idly thinking of things, and starting to consider ideas for
>> how
> I
>> will approach this.
>>
>> basically, I am considering the issue of how I will approach adding GUI
>> facilities to my x86-interpreter-based world.
>>
>> [...]
>> I don't feel X11 is an ideal choice
>>
>
> I've mentioned Depui GUI a few times. I'm not sure if it's developed
> enough
> for what you intend.
>
> Depui
> http://www.deleveld.dds.nl/depui33/depui.htm
>
> His main page with a couple "typesafe" projects:
> http://www.deleveld.dds.nl
>
errm...
this does GUI drawing, granted...
but, I am not sure if it is how I wanted to approach GUI...
this would imply doing drawing in the interpreter (slow), and simply supply
raster graphics to the host app. this is lame...
>
> I've considered the *idea* of combining the low-level functionality of an
> OS
> with LibSDL for the higher level functionality. It's not in my current
> plans. You could then choose one of many LibSDL games or OS-like
> applications (DOSBox, DOSEMU) as your OS. Instead of multibooting Linux,
> Windows, DOS, etc, you could multiboot DOOM, Quake4, MAME, DOSBox, etc.
> Sweet? I think so...
>
> http://en.wikipedia.org/wiki/List_of_games_using_SDL
>
I really don't know what you are talking about here...
basically, I am wanting a protocol, of a vaguely similar nature to the X11
protocol...
I had been looking some at X11R6, but still note that it is not exactly what
I would want.
X11 by itself is mostly about windows and pixmaps, which would work, it is
just not "ideal" for an OpenGL based host (would likely involve a lot of
manual drawing into a framebuffer then putting this on the screen via GL,
which is kind of a waste of GL).
similarly X11 is not too much higher level than just giving a framebuffer...
(I could do this, it would be easy, it is just wasteful).
however, I wanted to base the drawing abstractions on GL and vector
graphics, rather than on raster-graphics.
similarly, I will probably do the GUI drawing in the host app, probably
using custom widgets drawing via OpenGL;
the client app, running in the interpreter, can then use a protocol to
communicate with the host GUI.
the issue then is mostly about where the exact "level of abstraction" should
be.
lower-level, but fairly powerful:
provide a decent OpenGL subset (such as OpenGL ES), which is then used to
"draw into" the host (probably either directly to screen, or via "render to
texture");
GUI could be done via GL commands.
high-level, but weaker:
provide a widgets-orientated API (all widget state/... is done in the host
app).
this is closer to what I was imagining initially.
possibly, I could end up needing both:
high-level widgets, as well as lower-level GL.
I had imagined that GL commands could be batched up into long chains of
commands (essentially, like display lists), since this would be more
efficient (simulating multiple contexts via lots of render-to-texture would
be slower...).
also possible would be to use a drawing API design more similar to that used
in Direct3D...
(but, alas, I would rather buffer commands...).
or such...
Rod Pemberton
>
> >
> > I've considered the *idea* of combining the low-level functionality of
an
> > OS
> > with LibSDL for the higher level functionality. It's not in my current
> > plans. You could then choose one of many LibSDL games or OS-like
> > applications (DOSBox, DOSEMU) as your OS. Instead of multibooting
Linux,
> > Windows, DOS, etc, you could multiboot DOOM, Quake4, MAME, DOSBox, etc.
> > Sweet? I think so...
> >
> > http://en.wikipedia.org/wiki/List_of_games_using_SDL
> >
>
> I really don't know what you are talking about here...
>
:-) I changed topics on you... Didn't you find it interesting? Did
anyone else find it interesting?
LibSDL seems powerful enough to me to implement an OS on top of it. A few
psuedo-OSes have been done: DOSBox and DOSEMU. Many modern games, such as
Quake 4 run on LibSDL. One of the most complicated game engines ever
created, MAME, which emulates nearly every _arcade_ video game in existence
runs on LibSDL also. MESS, similar to MAME, but which implements emulations
for nearly every early personal computer (464 listed), such as C64, Atari,
TRS-80, etc., also runs on LibSDL. So, LibSDL could be used as the
low-level "hardware" API for an OS. But, does an OS have to be an OS?
Couldn't a game be your OS - say if you're a Q3 Arena fanatic?
http://www.libsdl.org/
http://mamedev.org/
http://www.mess.org/
> basically, I am wanting a protocol, of a vaguely similar nature to the X11
> protocol...
>
I know what X11 is. But, I'm not familiar with it's protocol or
implementation methods. You are talking about the X11 protocol used to
communicate graphics over a link, yes? E.g., to implement a thin-client?
Is this needed on a single PC? The X11 client and server are both on the
same machine. The only place I've seen a separation of the X11 client and
server, by a network link, was in a University setting. IIRC, this was a
purely non-Linux, non-Unix setup. DEC Alpha's were on one side and DEC
VAXes were on the other side.
> the issue then is mostly about where the exact "level of abstraction"
should
> be.
That's always the issue with everything code related.
> provide a decent OpenGL subset (such as OpenGL ES)
Standards are good. They allow more stuff to work. But, they also usually
require more work and introduce restrictions.
Rod Pemberton
BGB / cr88192
"Rod Pemberton" <do_no...@nohavenot.cmm> wrote in message
yeah...
SDL is basically just a wrapper though:
wrap GL/GLW/GLX, wrap threads and mutexes, ...
so, what it saves:
the big globs of OS-specific init code...
>> basically, I am wanting a protocol, of a vaguely similar nature to the
>> X11
>> protocol...
>>
>
> I know what X11 is. But, I'm not familiar with it's protocol or
> implementation methods. You are talking about the X11 protocol used to
> communicate graphics over a link, yes? E.g., to implement a thin-client?
> Is this needed on a single PC? The X11 client and server are both on the
> same machine. The only place I've seen a separation of the X11 client and
> server, by a network link, was in a University setting. IIRC, this was a
> purely non-Linux, non-Unix setup. DEC Alpha's were on one side and DEC
> VAXes were on the other side.
>
in my case, it would mostly be in use within the same app, but a protocol is
needed as otherwise I would need some kind of RPC, since the client and
server will be within separate address spaces...
most of the message passing is then likely to be being done over local
(probably local datagram) sockets.
>> the issue then is mostly about where the exact "level of abstraction"
> should
>> be.
>
> That's always the issue with everything code related.
>
yeah.
it is also why I often prefer to reuse existing designs / file-formats / ...
since then I don't need to design, only implement.
but, X11 is not an ideal fit...
>> provide a decent OpenGL subset (such as OpenGL ES)
>
> Standards are good. They allow more stuff to work. But, they also
> usually
> require more work and introduce restrictions.
>
yep.
a partial downside is that there doesn't really seem to be a spec around for
point-to-point OpenGL, so I would have to role my own.
it is also an issue that I might have to consider introducing semantics
above-and-beyond those of normal GL, such as separating draw-time and
non-draw-time operations, limit what can be done and where, ... this would
be not entirely "authentic".
all this would be mostly an artifact of the implementation (the interpreter
will not run lock-step with the host app).
raw GL might not be the best option.
I would probably need to, instead, operate at the level of geometry and
objects, and leave it more flexible as to "when" and "in what order" things
go onto screen.
however, this is not all bad:
this provides a way to more easily do things like real-time lighting,
shadowing, and object culling.
however, it is also, clearly, a piece of 3D engine technology (and, infact,
a superset of a prior piece of 3D engine technology I had developed to allow
more generically applying shader effects to arbitrary geometry...).
"render to object"...
it may have value, but it is in an entirely different part of the project
codebase.
but, hell, I have not worked much on 3D stuff in a while...
>
> Rod Pemberton
>
>
Rod Pemberton
> wrap GL/GLW/GLX, wrap threads and mutexes, ...
>
It's a wrapper for currently existing OSes. For a brand new OS, the LibSDL
API could be implemented as the OS' syscall API.
> so, what it saves:
> the big globs of OS-specific init code...
What it'd allow is the user to choose one or many games or OSes, written for
LibSDL, as their default computing platform. As more of these projects are
written for LibSDL, more choices of games and OSes will become available.
UML (User Mode Linux) is already abstracted from Linux. It the entire Linux
kernel, minus native hardware support. AIUI, it has it's own interface in
the 2.6 kernels. But, someone could port it from that interface to LibSDL
too. If it's capable of supporting all that stuff above, esp. MAME, I see
no reason why a fully functional C library couldn't be written against
LibSDL. At a minimum, it takes eighteen to twenty host OS functions to
implement an ANSI C library. POSIX need more. One you've a C compiler and
library for a platform, everything becomes available.
Rod Pemberton
BGB / cr88192
"Rod Pemberton" <do_no...@nohavenot.cmm> wrote in message
oh, ok...
I may look into this...
>> so, what it saves:
>> the big globs of OS-specific init code...
>
> What it'd allow is the user to choose one or many games or OSes, written
> for
> LibSDL, as their default computing platform. As more of these projects
> are
> written for LibSDL, more choices of games and OSes will become available.
> UML (User Mode Linux) is already abstracted from Linux. It the entire
> Linux
> kernel, minus native hardware support. AIUI, it has it's own interface in
> the 2.6 kernels. But, someone could port it from that interface to LibSDL
> too. If it's capable of supporting all that stuff above, esp. MAME, I see
> no reason why a fully functional C library couldn't be written against
> LibSDL. At a minimum, it takes eighteen to twenty host OS functions to
> implement an ANSI C library. POSIX need more. One you've a C compiler
> and
> library for a platform, everything becomes available.
>
yeah...
as is, for targetting my interpreter, I am using GCC...
I have a C library and many chunks of POSIX, but not much beyond this
(graphical stuff, ...).
so, I am mostly left uncertain as to the internal details of how I will
marshall all this stuff...
the use of "protocols" is one means I currently have available, and is
likely less effort than marshalling API calls, but alas, a protocol needs to
be designed, ...
(I would need to figure out what sorts of information will be sent and how
it will be represented, ...). luckily, I know at least that I have
essentially no latency or bandwidth issues (since things will be generally
moved around locally via buffers, and shared memory is also an option).
or such...