But it seems to me that what seldom gets asked is: why is it
Python's concern which package to use for those purposes? It seems
to me that a proper approach from the viewpoint of the language
itself is to provide an interface, and for the package developers
to provide an implementation of that interface. A Python program
would thus address its api only, leaving it up to the package to
interpret that properly. In the ideal world, this would mean that
any Python program could swap the gui package (or the db package)
without changing the logic of the program. Python-the-language
(rather than Python-the-package) would theoretically be a more
compact download, and the hooked-in parts could be installed
separately.

Anygui and anydb have been attempts to reach the ideal interface
(and there are others, I think), but each has its problems and
neither is supported or developed as fully as the idealized
interface would require. I don't see much evidence that the various
packages are designed to conform to those designs (and I don't
contend that they should be required to until and unless the
generic interfaces are in a well-developed state). But the result
of all this is that there continues to be confusion to newbies and
even veterans who are new to a given area or platform. Shouldn't it
be a priority to eliminate the source of that confusion?

Now, on the grounds that practicality beats purity, it's certainly
reasonable to argue that the current system works better. If we
could just jump into a state where there was a perfectly-
functioning anydb and anygui, then we'd already have been there,
but we can't and so we won't try to go in that direction at all.
Python-the-package marches on; it can't wait for some ideal to
coalesce before it advances. It's hard to argue too strongly
against that view.

Then, too, it's hard to see how an anygui or (more especially, I
think) an anydb could possibly work in the real world. The various
approaches to windowing and databases are too different for a
single interface to encompass them all. Some capabilities would be
unforeseen or ignored in any such interface. That argument carries
a lot of force. Certainly it is true that adding parameters to a
function call to accommodate some new feature gets real old real
soon. But while it is a strong practical argument, it is not one
that truly wins in the long run, I think. It is an argument in
favor of changing the way arguments are passed, so that it becomes
possible to handle the various pieces of information the lower-
level package needs. The interface should be agnostic as much as
possible. The package should be able to get the information it
needs from what is passed by the api. Both of these things must be
true.

Back when I was programming in DEC C, I saw how their
implementation of functions like printf departed from the C
standard by adding optional keyword parameters. Using those
parameters, DEC's more-sophisticated filesystem could be used
productively without going deeply into the filesystem's api.

Of course, a program written that way wouldn't work if transferred
unchanged to another system, but it *could* have worked if the
other system could also either handle the keywords (or some of
them) or ignore them. The results may not have been exactly as
desired, but there would have been results. Similarly, an elaborate
call that allows access to a sophisticated gui should also produce
results if the gui is simpler. Supposing that a given parameter
required by the second system has not been included in the first
place, it should be possible to add that parameter without
affecting the first system one bit. The first one would still not
be looking for the parameter, while the second would now find it.

That's all I'm talking about here. To be able to write a Python
application that can ultimately be displayed using wxWidgets or Qt
or Tkinter or Curses. Then, without having to do more than to
change which interface package is imported, to expect to see the
result displayed. To have it just happen, and to have expectations
when using Python that it *will* just happen that way.

So what do you think? What's wrong with the picture? Why isn't
there a greater priority to work in this direction?

http://www.joelonsoftware.com/articles/LeakyAbstractions.html

As he says, "All non-trivial abstractions, to some degree, are leaky."
GUI abstractions tend to leak badly.  As I write this email in
Thunderbird, my initial impression is that most of the controls on the
screen are pretty standard and could be done with virtually any
toolkit... or are they?  Looking again, the "from" field has both a
place for me to type and some grayed out text that reminds me I'm
posting from a different account; the "to", "cc", and "newsgroup" fields
are not quite combo boxes; the toolbar has pulldown buttons.

So it turns out that Thunderbird is actually tied to a particular
toolkit, and for good reason: GUI development has not stagnated; toolkit
developers continue to invent new, interesting widgets and variations,
leading to better UIs.  Application developers want to take advantage of
those innovations.

Even anydbm is a pretty leaky abstraction.  I recently tested the
behavior of 5 Python dbm implementations when they run out of disk
space.  Most left the database in an inconsistent state, some swallowed
the exception, some segfaulted, and one deadlocked.  If I'm going to
rely on a dbm module, I really need to know what it's going to do when
it hits an exceptional condition.  Thus anydbm seemed like a bad choice.

Just one teeny little item.

The Python world lacks the phenomenally successful development models
enjoyed by the now ancient Turbo Pascal, Delphi and <gasp> Visual Basic.
    AND
If the likes of Visual Basic can have it, then it becomes really, *really*
hard to convince the world that Python is a serious, professional system.

At some point, one has to break out of theory and produce!
Or challenge the theory with some hard questions.

Just one teeny little item.

The Python world lacks the phenomenally successful development models
enjoyed by the now ancient Turbo Pascal, Delphi and <gasp> Visual Basic.
    AND
If the likes of Visual Basic can have it, then it becomes really, *really*
hard to convince the world that Python is a serious, professional system.

At some point, one has to break out of theory and produce!
Or challenge the theory with some hard questions.

    Thomas> The Python world lacks the phenomenally successful development
    Thomas> models enjoyed by the now ancient Turbo Pascal, Delphi and
    Thomas> <gasp> Visual Basic.
    Thomas>     AND
    Thomas> If the likes of Visual Basic can have it, then it becomes
    Thomas> really, *really* hard to convince the world that Python is a
    Thomas> serious, professional system.

Remember, in the open source community we all pretty much just scratch our
itches.  Given that for the most part nobody in the Python community has a
handle on any other Python person's paycheck, it's unlikely that enough of
the community can be convinced that a VB-like development environment would
be a "killer app" for Python and thus motivated to go produce one.
Apparently, most Python people feel productive enough without such a tool.

I think we need to get over this "killer app" jazz.  How about plain old
productivity?  How do we enjoy the rapid and effective algorithm production
that Python affords us if one must pay back that productivity on an
effective and tediously acquired user interface. I rather think reducing the
code needed to produce a useful user interface frees up considerable effort
better spent on the programming problems we use Python for.

I happen to be one - and I *know* I'm not alone - who thinks that building
user interfaces is way too difficult and way too important.  It is
particularly frustrating in that we do seem to be moving backwards in this
department.

Am I the only one who notices that we are in the dark ages when it comes to
producing (new) software that people can use.  Was MS Office 2000 the be all
and end all of software development.  Why has it all come to a halt?  My
explanation is a:
      Microsoft entering senility
      a programming community unable to recognize a developing vacuum
      The great html plague
And -

But there is a common need for production of systems where data-
entry and data views are required. Typically, the back end is a
database, though it need not be. It's not that hard to build such a
system using any one of the packages, particularly once you acquire
expertise with that package.

But should you want to switch to another package, you find that a
*lot* has to change in your applications, no matter how hard you've
tried to separate the presentation from the logic, just because the
requirements of the various systems differ so much. It might be
that you have to reorder parameter lists, or subtract parameters,
or insert some. Almost never can you simply use a different import
statement alone to change the underlying windows handler. If you
need to add new parameters, almost never can you simply append them
to an existing parameter list and expect the thing to work.

And if you do change to the new system, your code can no longer
work using the old system. A lot of this is just because of the way
interfaces are specified; for some call on system A you need, let's
say: (a)a label, (b)a size (which must be a tuple, not a list), (c)
a boolean value. Now you switch to system B, which does not name
the equivalent call the same, nor does it pass in the same
parameters, or not in the same sequence. Multiply this by all the
api calls you have, and you may as well start over with each
package as try to make it all work.

It seems to me that Python (the language, not the package) is
intended in part to make tasks like that easier. It would be better
if the call for a given function were named the same (in your
application code), and if you could simply append some new
parameters to make the new system function (knowing that its
interface would simply ignore the old parameters it did not
understand), well, that's not too hard, and it doesn't break the
old system. So you add what you need to (if you need to add
anything), and now you have a system that runs on two packages. All
you have to do is to import the one you have, or want to use.

It means the package writer has to create an API bridge, one that
maps the underlying package's api to the Python standard api. The
application writer doesn't or shouldn't have to worry about that
mapping. And it probably means that dictionaries (or something
similar) are used to pass most parameters.

This sounds chaotic, as though interfaces would suddenly be
impossible to document, but it need not be so. If a package has a
feature that cannot be mapped reasonably to the Python api, then
add to the Python api, or (as part of the Python api) allow a
system pass-through the package's api can interpret. If the package
doesn't get the information it needs, it will complain, just as it
does now. Its requirements will be documented just as they are now.

It's not that all applications would run on all systems without
change, but that essentially all *basic* applications would run on
any system that supported the same basic operations without change.
There are many widgets, and as windowing packages become slicker,
there are more options; eventually, as the options become universal
(or at least common) they would be added to the Python api.

Whether or not a D&D GDI would make this better would still be up
to the individual programmer, I'd think. It would be a heck of a
lot easier to *write* a GUI if you weren't overly concerned with
the underlying graphics package.

That said, you're right:

    >> Apparently, most Python people feel productive enough without such a
    >> tool.

    Thomas> As did many with Fortran. Or Cobol. Are we now so productive
    Thomas> that there is no longer an unmet need for new/better software?
    Thomas> Do we stop here?  Is Python a comfortable place for the Luddites
    Thomas> hang out.

There's a difference.  When I began programming, the only tools I had were
the IBM card punch machine (basically a typewriter to enter my program) and
the printout I got from the operator via my drop box (to show me compilation
or run-time errors or my program's output on those rare instances when the
PL/1 compiler didn't barf on my deck).  In that situation, switching to a
better set of tools (an ed(1)-like editor displayed on a CRT hooked up to
our school's CDC timesharing system) was a no-brainer.  It wasn't all that
big a jump to switch to PR1ME computer's full-screen editor (probably
something like vi, though it's been so long I can't remember anymore).
After leaving school and moving on to VAX/VMS I looked around for another
full-screen editor, farted around with EDT for a bit, but eventually
stumbled upon Gosling's Emacs (on a DECUS tape I think), then GNU Emacs
awhile later, then in the past three or four years settled on XEmacs.  (I no
longer remember how I edited BASIC code on the IMSAI 8080.)

About 25 years have elapsed since my first exposure to Gosmacs.  As you
might imagine, I now have a fair amount of cortical area devoted to
Emacs-fu.  Based on my inability to help my son with his geometry homework
this year I suspect most of the neurons that had previously been devoted to
Euclidean geometry have been taken over by Emacs.  Here's a quick summary of
(some of) the functionality available to me from within Emacs that relates
more-or-less directly to programming Python:

* I do all my basic text editing and software development in Emacs, both
  local and remote files, but I also read mail there, including this mailing
  list.  If I had more of a Usenet mindset I could read it in Emacs as well.

* Python mode (thanks Tim & Barry) gives me access to inline help for
  attributes of many builtin objects, it does a good job of colorizing my
  code, supporting block moves and block comments and alerts me to runaway
  strings.

* With pycomplete hooked up to Pymacs, it can do a pretty decent job
  completing partial expressions.

* Using Emacs' builtin compilation facility I run pychecker over my code and
  step through the messages it emits.  (I suppose with a little thought I
  could modify the compilation facility to define pychecker suppressions
  on-the-fly as I pick through pychecker's output.)

* Emacs' tags facility lets me effortlessly jump to class and function
  definitions in my code.

* I can get Python tracebacks when sitting at a gdb prompt using a little
  snippet of gdb's command language.  If I wanted to I could use pdb to set
  breakpoints and execute my code under control of the pdb debugger but I
  generally just insert print statements.

* I've also messed around a little with Bicycle Repair Man, a refactoring
  editor written in Python that integrates into Emacs via Pymacs.

* I can also edit C and C++ code in the same editing/compilation/debugging
  environment when the need arises.

* I can transparently use CVS or SCCS (and I suspect Clearcase and
  Subversion) from within Emacs, without being aware of the different
  packages' syntaxes.

* If I need to work from home I can ssh into my desktop computer at work and
  using gnuclient, connect up to a running Emacs session that's got all the
  state I built up while at work.

The learning curve for me to switch to something other than Emacs for
software development at this point just isn't worth it.  I'm sure the same
would be true of long-term vi users or people accustomed to other
full-featured editors.  (Both Emacs and vi have the advantage that they are
available on all major computing platforms.)

Maybe I'm arguing for a tight integration between an IDE and an external
editor and just don't realize it.  I suppose it can be done, but I suspect
it would be a major challenge.  If not, it would have been done well already
by other tools.  Instead, my limited experience in this area suggests that
most development environments just punt and use whatever feeble editing
widget is available in the author's chosen widget set.  They all seem feeble
and slow to me, mostly because they aren't optimized for editing fixed width
text.

OTOH, maybe I'm arguing for a better integration of other tools/packages
with Emacs and more complete documentation of what's already available.
Should anyone be interested in helping out, a good place to tackle that
would be in the python-mode project on SF
<http://sourceforge.net/projects/python-mode/>.  I do know from past
experience with other tools that "providing basic Emacs keybindings" just
doesn't cut it.  Where are the rectangle and register commands?  What about
macros?  (How would I live without macros?)  At this point it's not just
basic typing skills I would have to transfer to another tool to be able to
switch and be as productive as I am with Emacs today.  Sure, you can laugh
at the kitchen sink that Emacs has become over the years, but it supports a
lot of functionality.  Think of it as an IDE framework & tool chest if you
like.  I'm sure I've only scratched the surface.

In short, just because I still program in Emacs, don't assume I write my
code on stone with hammer and chisel.

    >> Are we talking about a drag-and-drop GUI builder?
    Thomas> I am!

    >> What about Glade?
    Thomas> Dying to try that.  Ask me again in a week.

I use Glade at work on a regular basis.  Took me a few days to get
proficient, and it has some warts, but it sure beats the heck out of
manually creating all those widgets in my Python code.

My reference to Visual Basic was meant to be a poke in the eye.  The
language itself stinks.  BUT - having a robust IDE with keyword tooltip
prompts, built in language documentation, robust built in debugging, and
most of all - a stable GUI , a standard suite of widgets for interacting
with those pesky, error prone humans, the ability to draw these on a form
with a mouse and have all that running in minutes! It no longer matters that
the language itself smells like 4 day old fish!  No amount of linguistic or
structural elegance can have as much impact on productivity as the IDE/GUI.

It drives me crazy when someone suggests that it might amount to no more
than a bit of fluff hardly worth bothering with because when it comes to
programming -

    Thomas> My reference to Visual Basic was meant to be a poke in the eye.
    Thomas> The language itself stinks.  BUT - having a robust IDE with
    Thomas> keyword tooltip prompts, built in language documentation, robust
    Thomas> built in debugging, and most of all - a stable GUI , a standard
    Thomas> suite of widgets for interacting with those pesky, error prone
    Thomas> humans, the ability to draw these on a form with a mouse and
    Thomas> have all that running in minutes! It no longer matters that the
    Thomas> language itself smells like 4 day old fish!  No amount of
    Thomas> linguistic or structural elegance can have as much impact on
    Thomas> productivity as the IDE/GUI.

    Thomas> It drives me crazy when someone suggests that it might amount to
    Thomas> no more than a bit of fluff hardly worth bothering with because
    Thomas> when it comes to programming -

    Thomas> An excellent IDE/GUI just trumps everything.

Glade is fine for building Gtk user interfaces.  I have no idea if there are
similar tools for other widget sets, though I wouldn't be surprised if such
tools existed.  Once the GUI is fairly stable, most of the development after
that occurs in the underlying functional part of the code (at least in my
recent experience).  For that, no amount of Glade slinging will help.

http://wxglade.sourceforge.net/
http://boa-constructor.sourceforge.net/
http://spe.pycs.net/
http://www.roebling.de/

http://thekompany.com/products/blackadder/

Now, I'm not an expert on GUIs, GUI toolkits, or GUI builders. I've
used some of each, and draw my conclusions from that admittedly small
sampling. Possibly there are other samples that don't have the
problems I describe. Part of the point of posting this is to expose
these thoughts to others, and find out what I've overlooked. I'll
return to these points where appropriate.

What the user deserves.

I'm going to take a side trip into what constitutes a robust, flexible
user interface - which is what I believe the user deserves. Please
bear with me. By robust, I mean the user interface should adopt to the
environment it's being run in. Windows that don't fit on the screen
are simply unacceptable - but all to common if you use a very small
screen resolution. By flexible, I man the user should be able to
adjust the UI to suit their viewing conditions - the DPI on the
screen, the viewing distance, and their possibly aging eyesight.

I use two systems with graphical interfaces, and both are a bit out of
the ordinary. You might even call them extreme. One is a 1600x1200
desktop on a 21" monitor viewed from about 18" away. The other is a
640x480 desktop on a 53" monitor viewed from about 10' away. Even
those these are on opposite sides of "normal" users systems, they both
suffer from the same problem - out of the box applications have UI
elements that are unreadably small.

The obvious solution would be for the system to detect all these
environmental factors, and scale the applications
accordingly. However, things like viewing distance and the quality of
my eyesight are hard to detect automatically, and it would be a pain
to have to enter all those things manually. Since the end result of
all these is a single factor - a UI scale factor - a system wide knob
to scale applications would seem to be the solution.

Note that this is *not* an easy thing to do. Anyone who's tried
scaling bit-mapped fonts will tell you you can't simply scale something
and expect it to look good. Scalable fonts have "hints" and other
goodies in them so they look good as you scale the fonts up/down. This
may be why, but most windowing systems don't seem to do provide a
global UI scale knob. X certainly doesn't. Windows has a limited
capability to scale fonts system-wide, but it doesn't work very
well. OSX seems to have a system-wide display DPI setting one can play
with, but it's not clear how well that works. NeWS and NeXT could
probably do this, but would seem to have a negligible to nonexistent
user bases. rio seems capable, but the typical rio UI is ugly even by
my admittedly low standards.

Failing the single system-wide knob, a flexible application should
have allow the user to scale the application with a single knob. This
is hard to do with modern GUI toolkits. I do it by using text for
imagery in the UI, and having a scaling factor that is applied to the
font sizes. While I prefer text to graphics for UI elements, that's a
rant for a different time. I'll concede that such an interface is
probably unacceptable on a modern commercial application - users now
expect graphics. But how many GUI toolkits let you use a scalable
graphics format (SVG, PS, etc) for images in the UI? My predilections
have kept me from looking closely, but I certainly don't recall seeing
any in the GUI toolkits I've looked at.

I claim it's obvious from this what's wrong with GUI builders. But I
want to look at what I, as a developer, want from GUI development
tools before going into details.

What the developer wants.

Ok, what I mean is what *I* want. But I don't think I'm that unusual,
so I'm going to generalize to other developers as well. I'll point out
what has to be different for this generalization to fail.

First, I'm not a graphics designer. While I've studied page layout and
typographical design, and read Tufte's books on design, I'm a *long*
way from being a good graphics designer. I know even less about the
field of human-computer interactions. Hopefully, a good GUI library
will have had people on the development team who do know something
about these fields. In this - the desirable - case, the library should
have more knowledge about good UI design than I happen to posses.

Given that, the more I can depend on the library to do for me - the
less I have to specify - the more likely I am to produce a good UI
with that library. Conversely, the more I have to specify in the GUI
development process, the more likely I am to screw things up. I claim
this statement should hold for most developers, unless they happen to
be experts in the fields of graphics design or human-computer
interactions.

What's wrong with GUI builders.

Ok, *now* we're ready to deal with GUI builders.

The first, and most obvious, thing that GUI builders do is force the
developer to specify an exact position - if not size - for the
graphical elements of the UI. Better GUI libraries don't do
that. Instead they let the developer specify the position as part of
the overall structure, say as "the third button in the second toolbar
from the top", or "one of two buttons in a toolbar running across the
bottom of the window". This latter allows the knowledge of the library
designers to dominate the design, hopefully producing a better UI than
the you get from your typical software developer.

The GUI builder might provide an advantage if it allowed the GUI
design to be cleanly separated from the underlying code, so that a
specialist in UI design could do the graphics design, and then the
developer could come and add the underlying code. However, the GUI
builders I've dealt with require specifying names for the various
elements of the UI, which names should make sense in both
environments. This would seem to defeat such a split design, but I've
never tried it, so I can't say for sure.

Further, look at the kind of interface the GUI builder gives you. The
position of all the elements are nailed down. This makes it hard for
the GUI to adopt to the extremes of display environments that you find
in the wild. The GUI builders I've used made it hard, if not
impossible, to build an application that can scale the GUI to meet the
users needs. In other words, the GUI builder helps build applications
without flexible, robust user interfaces that are inferior to what can
be built using a GUI library.

Typically, I don't find a GUI builder that much faster than a good GUI
library. If I have to specify pixel positing for elements, the builder
will be a win, but if not, then not. So I haven't examined all the GUI
builders in depth. Maybe there's one out there that will let the
application developer scale the GUI when the application is invoked,
and will re-arrange elements of the GUI so they fit on the screen
properly. If so, I'd love to hear about it.

Certainly some---or even most---builders work like this.  But there's no
reason that a GUI GUI builder can't work in terms of the more
sophisticated layout algorithms that are available in most modern GUI
toolkits.

I've written a GUI builder for Tcl/Tk (some old versions live at
http://unpy.net/~jepler/nf/ but they're very out of date).  The core of
the application is the part that can write Tcl source code to regenerate
the screen as it's currently displayed.

The user can either use graphical tools like "insert widget", "show
properties", "pack earlier/later", "automatically add accelerator keys",
or script the thing by typing in Tcl.  Want to use the grid manager?
Fine, go ahead.  The graphical interface to the grid command may be
poor, but the commandline interface works great.  When things are as you
want them, just "save", and you can trivially "source" the resulting Tcl
code from your Tcl/Tk app.

I'm not trying to evangelize Tcl/Tk above any other language/toolkit,
but I am saying that by pairing an interpreted language with a GUI
toolkit, you can get a powerful GUI builder that instantly has top-notch
scrptability and also the guarantee that you can use the powerful
features of the GUI toolkit.  

I didn't do a survey of existing software before writing mine, but I
don't know of another builder that takes this kind of approach.  I
wonder why not.

I seem to remember that the GUI builder in Visual Studio didn't seem
to be particularly sane (I couldn't seem to work out how to get a
dialog to resize appropriately to accommodate different font sizes,
for example), but I assumed that was just me failing to understand
something.

Swing(for Java), GTK, Qt, and wxWidgets(to a lesser degree) all use a
sizer metaphore.  You lay out he sizers and put your widgets in
various sizer layouts.  This means that the scaling you speak of
above, just works for you.  Accordingly GUI builders for these
toolkits work within that model(look at Glade for GTK and wxGlade for
wxWidgets).

I think you need to step out of the age of Motif and MFCs and look at
what modern toolkits and GUI designers have to offer before you start
in on a rant.

-Chris

On 6/5/05, Mike Meyer <m...@mired.org> wrote:

    The toolkit I am most familiar that does try to be magnifiable is
Windows.Forms and it is also my least favourite currently having
particularly ugly text. To achieve resolution independence Windows.Forms
(and the underlying GDI+) uses sub-pixel positioning as do other recent
rendering libraries. This leads to the possibility of each render of a
particular character being different. For example, in a text editor, if
the font height is 10.5 pixels then two assignment lines
x = 1
y = 0
    may display the first '=' legibly with one black line, one white
line and another black line but the second line, offset by .5 pixels
will show a 4 pixel high grey rectangle. Forcing the '=' onto the pixel
grid will distort the text appearance.