Image based development

[Blake McBride]

A popular development scenario that I term "disk based" looks like the
following:

A. A program is created / edited with a text editor (like emacs)

B. The code is saved to a disk file (like myprogram.lisp )

C. A language interpreter / compiler loads (from disk or a socket)
and runs the code where the programmer evaluates its performance

D. Go back to A.

---

There is another development scenario that is old, but may be worth
looking into. I term this other method "image based" and it looks like
the following:

1. Code is added and edited right in the environment where it is being
used (like CCL, SBCL, CLISP, etc.). Editing the code could be done
with an editor like a structure editor written in Lisp rather than a
text editor.

2. Run the code right in the environment it was created in.

3. Go to step 1 in order to continue development.

Also:

4. The state of the project can be saved as an image file
representing the entire project when necessary (i.e. CLISP .mem file,
SBCL and CMUCL .core file, CCL image file, etc.).

5. A disk / text form of the application can also be created for
reference or porting whenever needed (like mypackage.lisp ).

Languages that use this latter model include Smalltalk and InterLisp.

---

In order to do image based development in Common Lisp, certain tools
are needed:

a. A way of editing a program, like a structure editor

b. A way of writing a text form of the application when desired (item 5
above)

c. The interpreter / compiler should be able to save / load an image

I have completed item "a" (to be released shortly). I will write item
"b" shortly. Item "c" is already included in many Lisp interpreters /
compilers. Those without it can save/load a text form that will be
provided by item "b", although this is somewhat clunky.

I been interested in image based development for some time. I thought
this would be a good road to investigate. Your input is appreciated.

Thanks.

Blake McBride

[Pascal J. Bourguignon]

Blake McBride <blak...@gmail.com> writes:

> I been interested in image based development for some time. I thought
> this would be a good road to investigate. Your input is appreciated.

http://www.informatimago.com/develop/lisp/com/informatimago/small-cl-pgms/ibcl/
http://www.informatimago.com/develop/lisp/com/informatimago/small-cl-pgms/sedit/

--
__Pascal Bourguignon__ http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

[Blake McBride]

On 08/28/2015 12:46 PM, Pascal J. Bourguignon wrote:
> Blake McBride <blak...@gmail.com> writes:
>
>> I been interested in image based development for some time. I thought
>> this would be a good road to investigate. Your input is appreciated.
>
> http://www.informatimago.com/develop/lisp/com/informatimago/small-cl-pgms/ibcl/
> http://www.informatimago.com/develop/lisp/com/informatimago/small-cl-pgms/sedit/
>

Thanks. I knew about sedit. The structure editor I will be releasing
is quite a bit more substantial.

Basically, my LISPF4 InterLisp clone contains a clone of a substantial
subset of the standard InterLisp structure editor. I ported that to
Common Lisp. A copy of the manual appears at the end of this message.

I will look into your ibcl. That may make it unnecessary for my next step.

Thanks!

Blake


-----

EDIT - Structure editor for Common Lisp

This code was taken from the LISPF4 project and was enhanced and
converted to Common Lisp by Blake McBride. The LISPF4 editor is an
implementation of the structure editor that came with InterLisp.

In order for editing of functions to work, they must have been defined
using the "defun" or "defmacro" defined in this package.


Several edit functions are implemented:

(EDITF fn edcom) edit a function or macro. Value = NIL.
(EDITS s edcom) edit any s-expr. Value = s
(EDITV v edcom) edit variable v
(EDITP s edcom) edit the property list associated with s
edcom = list of edit commands
(or NIL). If edcom is non-NIL the commands
will be executed and the editor will exit.

In what follows cexpr is the current expression.

The following commands are implemented.

OK Saves the changes and leaves the editor
STOP Exits the editor without saving
SAVE Saves edit for future edit session (only EDITF)

Display:

P Print to level 2
PP PrettyPrint to level 2
? Print to level 100
?? PrettyPrint to level 100

Positioning the cexpr:

! sets cexpr to top level expression
n Set cexpr to the n'th element of cexpr.
-n Set cexpr to the n'th element from the end of
cexpr.
0 ascend one level.
NX next expression
(descend to the next element one level above)
UP Ascend one level but only display elements to the
right of the original cexpr.
F expr searches the first occurrence of expr
in the cexpr regardless of level

(MARK atm) Set atom atm to the current edit position.
(/ atm) Go to position marked by atom atm

Expression editing:

Adds:
(-n e1 ...) inserts e1 ... before the n'th element.
(N e1 ...) adds e1 ... after the last element within cexpr.
(A e1 ...) Adds e1 ... AFTER cexpr.
(B e1 ...) Adds e1 ... BEFORE cexpr.
Replacing:
(n e1 ...) n >= 1 replaces the n'th expression by e1 ...
(: e1 ...) Replaces the entire current expression by e1 ...
(R x y) All occurrences of x are replaced by y in cexpr.
(MBD e1 ...) Replace cexpr with e1 ... and allow * to represent
the original expression.
Ex.: We have (PRINT X) and we want
(COND ((NULL L) (PRINT X) NIL)
(T (PRINT X) (GO LOP)))
we do
(MBD (COND ((NULL L) * NIL)
(T * (GO LOP))))
(XTR e1 ...) Ex.: We have (COND ((NULL L) NIL)
(T (PRINT L))
and we want (PRINT L)
we do
(XTR 3 2), (XTR (PRINT L)) or
(XTR PRINT)

Deletions:
(n) n >= 1 deletes the n'th expression of cexpr
(:) Delete the current expression.

Global editing:

S x Set x to the current expression.
(US x cmds) Execute edit commands cmds with the ability to
utilize the expression in atom x
S and US can be used in different edit sessions.

Ex.: Move the PROG expression of FOO to be the PROG expression of
another function FII.

(EDITF FOO)
F PROG S DEF OK
(EDITF FII)
(US DEF (3 DEF)) OK

The 3'rd element (the prog expression of FII) is replaced by the one
stored in DEF.

Parenthesis manipulation:

(BI n m) Both In. A left parenthesis is inserted before
the n'th element and a right parenthesis is
inserted after the m'th element.
(BI n) insert parenthesis around the n'th element
(BO n) Both Out. Removes both parenthesis from the
n'th element.
(LI n) Left In. Inserts a left parenthesis before the
n'th element and a corresponding right at the end
(LO n) Left Out. Removes the left parenthesis from the
n'th element. All elements after the n'th element
are deleted.
(RI n m) Right In. Move the parenthesis at the end of the
n'th element in to after the m'th element inside
the n'th element.
(RO n) Right Out. Move the right parenthesis of the n'th
element to the end of the current expression. All
elements following the n'th element are moved
inside the n'th element.

Evaluation:

E expr Evaluate expression expr.

(ESET x c1...) Sets atom x to the edit commands c1...
x Executes the edit commands associates with atom x.
(ESET x) Disassociates all edit commands from atom x.

[Kaz Kylheku]

On 2015-08-28, Blake McBride <blak...@gmail.com> wrote:
>
> A popular development scenario that I term "disk based" looks like the
> following:
>
> A. A program is created / edited with a text editor (like emacs)
>
> B. The code is saved to a disk file (like myprogram.lisp )
>
> C. A language interpreter / compiler loads (from disk or a socket)
> and runs the code where the programmer evaluates its performance
>
> D. Go back to A.
>
> ---
>
> There is another development scenario that is old, but may be worth
> looking into. I term this other method "image based" and it looks like
> the following:
>
> 1. Code is added and edited right in the environment where it is being
> used (like CCL, SBCL, CLISP, etc.). Editing the code could be done
> with an editor like a structure editor written in Lisp rather than a
> text editor.
>
> 2. Run the code right in the environment it was created in.
>
> 3. Go to step 1 in order to continue development.

Purely image based is stupid, because the image is some binary cruft
that you canno version control, or port to other systems.
In some Lisps, images require a separate executale which loads them,
and sometimes newer versions of that executable wont' load old image
sor vice versa.

The hybrid approach is this:

1. Run one image.
2. Edit important code in a text editor, saving to a file.
3. Load modified files into the image to replace definitions with
new ones. (Perhaps using a build system like ASDF).
4. Directly, in the image, write only throwaway functions for testing,
and use the REPL for exploring and debugging. If anything is
to be retained, paste it into a file.
5. Optionally save the image to checkpoint the state or other reasons.

Lisp environments like Slime try to make it mostly seamless, because
you can evaluate code into the image out of the editor.

Speaking of image saving, it is important even if you don't use it
for development, because it's one of the ways of creating standalone
applications.

Also, your Lisp probably loads fast, even though it is loaded with functions,
because it's actually restoring an image, and not actually evaluating hundreds
of defuns and defmethods.

Emacs starts from an image, using a low-level C technique of
dumping memory and then reloading it.

I worked on a project which had a Make rule tree that too 30 seconds to read
and process. So even the tiniest incremental build took 30 seconds from the
time you typed "make" to the one-file compile, and link (which themselves took
a fraction of a second)!

I got sick of it and ripped the undumping code out of GNU Emacs, transplanting
it into GNU Make. I then had something like "make --dump file" which produced
a core dump in file, and a "make --restore file" (load the dump and build
without reading Makefile).

It became virtually instant to do incremental builds. Of course, if you
changed the rules, you had to re-do the dump: 30 seconds once in a while
was better better than 30 seconds every time.

[Blake McBride]

I understand and sympathize with what you say. On the other hand, I
wonder if editing large functions in a structure editor may be easier.
How often do people edit a large function, accidentally insert or leave
out one left or right paren only to find the whole function a mess?

I understand that emacs has a lot of facilities to help avoid that, but
it still can and does happen. Making this kind of mistake is much less
likely with a structure editor. With a structure editor, you are always
working in one scope at a time. Paren or sexp level errors are much
less likely.

Blake

[Pascal J. Bourguignon]

Despite the problems with pure image based development, it is not
stupid.

You want to be able to:

- ensure persistance,

- have reproduceable "builds". Ie. you need a source "script" that will
let you recreate the current state of your image starting from a
virgin image.

- manage revisions.

- exchange source "scripts" between images.

- import and export source "script" in a form readable and writable by
other systems for porting purposes (this could imply a textual
serialization into ASCII files or some such).

All these features can and have been provided by purely image based
development systems, notably in the Smalltalk world.



In particular, all but the last feature, DO NOT require a text based
source format! And you need a text based source format only because
that's the Code that's specified for Information Interchange by the
American Standard (ASCII = American Stadnard Code for Information
Interchange).


This text format is neither needed, nor the best choice, for features
such as revision control, source editing, "build" reproduction, or
persistance.

- Persistance is provided by the image saving feature, so we consider it a
solved problem. (It is not specified by the Common Lisp Standard, but
most implementations provide a save image feature).

- Reproduceable "builds" only require that you can recover the source
form from the image. This is not a feature mandated by the Common
Lisp standard, but FUNCTION-LAMBDA-EXPRESSION is specified if
implementations care to provide it. It is insufficient, since it
concerns only functions, and not all the toplevel forms possible.
This is what ibcl addresses. Notice that (source 'f :function)
does NOT return a string (text), but a sexp!

- Smalltalk has image based revision control systems, it would be
trivial enough to implement one for bunches of lisp sexps. (cf. the
various tree diff functions floating around). Granted, you might want
to integrate git, so check the last feature.

- Exchanging code between images can be done directly with sexps, (for
example, you can memory-map a shared memory block, and copy there the
sexp lisp objects, cf. com.informatimago.common-lisp.heap.heap).
And you can send this memory block thru the network to a remote image.

- finally, lisp alread provides PRINT and READ as serialization and
deserialization of lisp sources, so the last feature is free.

But before you save the text into external files, you can also consider
keeping it in the image, and using a text editor inside the image, if
you fancy reader macros, manual indentation and comments.


In any case, it's not dumber than file based development, where you will
have the same difficulties in transmitting files across different
systems (character encodings, line encodings, byte sizes, storage
devices or network protocols, etc). You seem to be forgetting that not
all computers are linux running on PC with the Internet.

[Pascal J. Bourguignon]

Blake McBride <blak...@gmail.com> writes:

> I understand and sympathize with what you say. On the other hand, I
> wonder if editing large functions in a structure editor may be
> easier. How often do people edit a large function, accidentally insert
> or leave out one left or right paren only to find the whole function a
> mess?
>
> I understand that emacs has a lot of facilities to help avoid that,
> but it still can and does happen. Making this kind of mistake is much
> less likely with a structure editor. With a structure editor, you are
> always working in one scope at a time. Paren or sexp level errors are
> much less likely.

Emacs with paredit (and even without to some level, it already has some
basic sexp-navigation and editing features), provides a hybrid view,
where you can perform structural editing as well as text-level editing.

In emacs with paredit-mode activated:

( is a structural editing command (insert a new list)

a is a text-level editing command (insert a #\a).

C-d provides a text-level editing function (deletes a single character)
when activated over consitutent characters, but it provides a
structural editing command, when activated over a parenthesis (left or
right).

and so on.


This hybrid approach has a big advantage from the ergonomic point of
view, since it allows for very modeless editing. (Despite having
major mods and minor modes, a lot of modes, emacs is basically a
modeless editor, contrarily to eg. vi, which clearly distinguishes a
command mode from an insertion mode; (but I hear vim is also able to
fuzzy the distinction).

So far, purely structural editors were rather inconvenient to use, since
even to edit a simple string or symbol name, you would have to activate
very specific commands in a very modal way.

paredit still allows to break the structure (and restore it too) of the
buffer. (For example, C-w will let you remove unbalanced parentheses,
and C-y let you add unbalanced parentheses). But it would be possible
to lock down all those cases in emacs, and to provide a mode that'd
preserve always the structure of the buffer, but that would basically
still be as easy to use as emacs+paredit, that is, basically, where you
give editing commands that look like they're working on the text level,
but that are actually structure level editing commands.

For example, instead of typing (BI n) to insert a list, you just type (
to insert an empty list, or M-( to wrap an element in a new list.
C-u N M-( to wrap N elements. Etc.

So while you are typing what seem to be normal text, your text is
actually interpreted by the editor as structural editing commands (and
validated as such), to produce a structured buffer. Ok, in the case
of emacs+paredit, the buffer actually contains text, and emacs+paredit
doesn't maintain a sexp representation, but a structural editor could be
developed providing the same emacs-like UI (with key bindings, etc) and
text-level editing features while actually editing a sexp structure
instead of text. This would be the most ergonomic.

[Barry Margolin]

In article <HY-dneQ1sqHaCX3I...@supernews.com>,

Blake McBride <blak...@gmail.com> wrote:

> I been interested in image based development for some time. I thought
> this would be a good road to investigate. Your input is appreciated.

I remember reading papers in SIGPLAN or LCS Technical Reports 30+ years
ago about this. Some people theorized that the file-based model would go
away within a decade, as GUI programming languages emerged. They seemed
to think that non-programmers would be able to do real programming with
drag-and-drop.

As we can see, this never happened. The closest we have are web
applications that make it easy for people to make simple web sites. The
grand expectations reminded me of COBOL -- its syntax is very
English-like, and the idea was that it would be understandable to
management. But I wonder how many managers ever really looked at the
code under the hood.

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Pascal J. Bourguignon]

Barry Margolin <bar...@alum.mit.edu> writes:

> In article <HY-dneQ1sqHaCX3I...@supernews.com>,
> Blake McBride <blak...@gmail.com> wrote:
>
>> I been interested in image based development for some time. I thought
>> this would be a good road to investigate. Your input is appreciated.
>
> I remember reading papers in SIGPLAN or LCS Technical Reports 30+ years
> ago about this. Some people theorized that the file-based model would go
> away within a decade, as GUI programming languages emerged. They seemed
> to think that non-programmers would be able to do real programming with
> drag-and-drop.
>
> As we can see, this never happened.

Graphical programming languages exist, here are a few examples:

http://www.dangermouse.net/esoteric/piet.html
http://code.google.com/p/blockly
http://drakon-editor.sourceforge.net/language.html

What doesn't happen though is non-programmers doing real programming.

> The closest we have are web
> applications that make it easy for people to make simple web sites. The
> grand expectations reminded me of COBOL -- its syntax is very
> English-like, and the idea was that it would be understandable to
> management. But I wonder how many managers ever really looked at the
> code under the hood.

None unfortunately.

[paul wallich]

On 8/28/15 3:50 PM, Pascal J. Bourguignon wrote:
[...]

> But it would be possible
> to lock down all those cases in emacs, and to provide a mode that'd
> preserve always the structure of the buffer, but that would basically
> still be as easy to use as emacs+paredit, that is, basically, where you
> give editing commands that look like they're working on the text level,
> but that are actually structure level editing commands.
>
> For example, instead of typing (BI n) to insert a list, you just type (
> to insert an empty list, or M-( to wrap an element in a new list.
> C-u N M-( to wrap N elements. Etc.
>
> So while you are typing what seem to be normal text, your text is
> actually interpreted by the editor as structural editing commands (and
> validated as such), to produce a structured buffer. Ok, in the case
> of emacs+paredit, the buffer actually contains text, and emacs+paredit
> doesn't maintain a sexp representation, but a structural editor could be
> developed providing the same emacs-like UI (with key bindings, etc) and
> text-level editing features while actually editing a sexp structure
> instead of text. This would be the most ergonomic.

My dim recollections of editing in Interlisp on D-machines was that they
did pretty much this. Cursor movements and insertions/deletions were
interpreted as structural movements behind the scenes, and did "the
right thing" depending on where you were. The result was for the most
part indistinguishable from a text-based editor except that you avoided
a lot of the screwups. (There's a huge jump in cognitive support going
from something that simply shows you where your matching parens are to
something that actually rebalances the text as you go.)

A structure-based editor also makes really good use of the old
square-bracket convention, because you really can use open and close
square brackets to delineate chunks of code, to get displayed code
folded where you want it and so forth.

Given today's computing power and space, there's nothing stopping people
from integrating structural editing with file-based representations for
permanent storage. (Because my recollection is that that was still
something of a development nightmare: you had to have the right kind of
bookkeeping mind to keep track of just what had been defined, redefined,
rolled back and so forth in an image. And you could get into states
where the would you wanted was very close textually to what you had but
not easily reachable in terms of deltas that could be applied.)

paul

[Pascal J. Bourguignon]

Barry Margolin <bar...@alum.mit.edu> writes:

> In article <HY-dneQ1sqHaCX3I...@supernews.com>,
> Blake McBride <blak...@gmail.com> wrote:
>
>> I been interested in image based development for some time. I thought
>> this would be a good road to investigate. Your input is appreciated.
>
> I remember reading papers in SIGPLAN or LCS Technical Reports 30+ years
> ago about this. Some people theorized that the file-based model would go
> away within a decade, as GUI programming languages emerged. They seemed
> to think that non-programmers would be able to do real programming with
> drag-and-drop.

Well first point is completed: iOS and Android essentially don't have a
(user visible) file system anymore.

> As we can see, this never happened.

Look better.

[Kaz Kylheku]

On 2015-08-28, Barry Margolin <bar...@alum.mit.edu> wrote:
> In article <HY-dneQ1sqHaCX3I...@supernews.com>,
> Blake McBride <blak...@gmail.com> wrote:
>
>> I been interested in image based development for some time. I thought
>> this would be a good road to investigate. Your input is appreciated.
>
> I remember reading papers in SIGPLAN or LCS Technical Reports 30+ years
> ago about this. Some people theorized that the file-based model would go
> away within a decade, as GUI programming languages emerged. They seemed
> to think that non-programmers would be able to do real programming with
> drag-and-drop.

What actually happened is that *applications* did away with the file-based
model.

The image-based model is what you see in office applications: word
processors, spreadsheets and so on.

A binary word-processor document is monolithic image of objects dumped to disk.

A programmer wants to use a markup language, in which content like graphical
images and style sheets are referenced as separate files.

[Barry Margolin]

In article <87bndqz...@kuiper.lan.informatimago.com>,

"Pascal J. Bourguignon" <p...@informatimago.com> wrote:

> Barry Margolin <bar...@alum.mit.edu> writes:
>
> > In article <HY-dneQ1sqHaCX3I...@supernews.com>,
> > Blake McBride <blak...@gmail.com> wrote:
> >
> >> I been interested in image based development for some time. I thought
> >> this would be a good road to investigate. Your input is appreciated.
> >
> > I remember reading papers in SIGPLAN or LCS Technical Reports 30+ years
> > ago about this. Some people theorized that the file-based model would go
> > away within a decade, as GUI programming languages emerged. They seemed
> > to think that non-programmers would be able to do real programming with
> > drag-and-drop.
>
> Well first point is completed: iOS and Android essentially don't have a
> (user visible) file system anymore.

They also don't have editors for writing programs on the devices, so
that's a non-sequitor.

When writing applications for these devices, you use editors on personal
computers, and they still use the file model.

[Pascal J. Bourguignon]

Barry Margolin <bar...@alum.mit.edu> writes:

> In article <87bndqz...@kuiper.lan.informatimago.com>,
> "Pascal J. Bourguignon" <p...@informatimago.com> wrote:
>
>> Barry Margolin <bar...@alum.mit.edu> writes:
>>
>> > In article <HY-dneQ1sqHaCX3I...@supernews.com>,
>> > Blake McBride <blak...@gmail.com> wrote:
>> >
>> >> I been interested in image based development for some time. I thought
>> >> this would be a good road to investigate. Your input is appreciated.
>> >
>> > I remember reading papers in SIGPLAN or LCS Technical Reports 30+ years
>> > ago about this. Some people theorized that the file-based model would go
>> > away within a decade, as GUI programming languages emerged. They seemed
>> > to think that non-programmers would be able to do real programming with
>> > drag-and-drop.
>>
>> Well first point is completed: iOS and Android essentially don't have a
>> (user visible) file system anymore.
>
> They also don't have editors for writing programs on the devices, so
> that's a non-sequitor.
>
> When writing applications for these devices, you use editors on personal
> computers, and they still use the file model.

Actually, there are several editors both on iOS and Android.

On Android there are even complete stand alone IDE allowing you to
develop locally Android applications.

But still, despite the use of unix kernels in those systems, they try
very hard, with sandboxes and restrictions, to prevent two applications
to process the same data files. This in effects nullify the notion of
file for the user, even if some tools like IDE and "file managers" on
Android still let programmers see them. The end user can only duplicate
the data by "sharing", ie., sending a copy, o ironia tempore, to a
different application sandbox.


Clearly, those systems are application centric, because this is where
Apple and Google can make money, selling applications.

But a computer system for users would be file centric, since this is
where the data of the users, owned by the users, and relevant to the
users, is stored. Applications are irrelevant to users, mere tools to
process their data. Even for games! As soon as the game is not
trivial, playing with it the user creates a data structure that is his
own, and that he might want to process with other tools than the game
itself. But he doesn't have those freedoms, since the system is here to
extract money from the user, not to serve his purposes.


Some people wonder why GNU/Linux is not established on the "desktop" yet.
They should not. It will never (at least not while still remaining GNU;
there's not much GNU stuff in Android or iOS). Not because of license,
but because of this fundamental philosophical difference. Desktops
(MS-Windows, MacOSX) are too strongly stressing the application side of
the duality, even if files are still a notion there. For example, files
are marked by the application signature, so that by default, it is
always processed by the same application.


On a unix (GNU/linux) system:

$ ls ~/Desktop/*.txt
/home/pjb/Desktop/20140308--apple--commande.txt
/home/pjb/Desktop/220446114-At-Last-Why-Sugar-Kills.txt
/home/pjb/Desktop/DOC-488531.txt
/home/pjb/Desktop/out.txt
/home/pjb/Desktop/pjb-notes.txt

files are not assigned to an application. You can process them with any
tool you want:

$ grep -A1 deadline ~/Desktop/pjb-notes.txt
| I love deadlines. I like the whooshing sound they make as they fly by.
| -- Douglas Adams
$ emacs ~/Desktop/pjb-notes.txt
...

(Don't be blinded by the text file format, the same works with any file
format, for example JPG, AVI, whatever. Perhaps not with proprietary
file formats, but nowadays even Microsoft Word writes xml files).


If you want to make a difference and establish GNU/Linux "on the
desktop", instead of building an application centric system that
encloses the users in a money extracting machinery, provide a file
centric GUI that lets user process their files with any program.
Including programs (scripts) they'll write themselves!


Of course, on iOS and Android, you have to go thru the filter of
capitalistic corporations whose only purpose is to amass astronomical
and meaningless amounts of money, so they won't let you do that, since
there's no money to be extracted from user data stored in user files on
user hard disks or sdcards.

[Ivan Kanis]

August, 28 at 15:58 Barry Margolin wrote:

> The grand expectations reminded me of COBOL -- its syntax is very
> English-like, and the idea was that it would be understandable to
> management. But I wonder how many managers ever really looked at the
> code under the hood.

And now we have Python... At least Pythonists are not trying to sell it
to managers.

Ivan

--
The price of wisdom is above the rubies.
-- Job ch. 30, v. 18

[Marco Antoniotti]

+1

--
MA

[Blake McBride]

Using Linux every day, and having been using Linux for many years (and
Unix before that), I strongly disagree with your assessment of why Linux
hasn't succeeded on the desktop. I believe the causes are as follows:

1. Successful cross-platform software vendors are purchased by MS or
Apple only to immediately drop Linux support thus eliminating popular
software from Linux.

2. MS and Apple tell hardware vendors that if they support Linux, MS &
Apple will no longer support them.

3. MS provides computer vendors with one price if they support MS only,
and a much higher price if they support Linux too.


Item 1 reduces Linux usefulness to users.

Item 2 reduces hardware driver support from Linux.

Item 3 reduces computer vendor support and promotion of Linux.

Just as MS kept the world in 640K needlessly for years and years, these
three things have very significantly reduced advancement in the computer
world.

How many years did MS practically give away Excel until they killed
Lotus? How many years did MS practically give away Word until they
killed Word Perfect? How many years did MS practically give away their
C compiler until they killed Borland?

The great God Steve Jobs killed Flash because Adobe made Windows their
primary platform for their publishing software.

It is a vicious world out there. Not a world in which ideas win. If
ideas did win, we'd all be using lisp.

Blake McBride

[Blake McBride]

On 08/29/2015 09:02 AM, Ivan Kanis wrote:
> August, 28 at 15:58 Barry Margolin wrote:
>
>> The grand expectations reminded me of COBOL -- its syntax is very
>> English-like, and the idea was that it would be understandable to
>> management. But I wonder how many managers ever really looked at the
>> code under the hood.
>
> And now we have Python... At least Pythonists are not trying to sell it
> to managers.
>
> Ivan
>

Imagine a language in which invisible characters determine program
logic! It is also a language that no pretty printer can be created for.

Blake McBride

[Blake McBride]

I didn't know about paredit. I just downloaded it and started playing
with it. It is very, very nice!!!

I wonder how many lisp programmers don't know about it?

Thanks.

Blake

[Pascal J. Bourguignon]

Only those who don't read cll and #lisp ;-)

[Marco Antoniotti]

You obviously have not heard of the programming language Whitespace :)

Cheers
--
MA

[Blake McBride]

No, I haven't. Whitespace is a joke language, I wish python was.

I had a project I had to do in Python. I bought a kindle book on
Python. The kindle formatting eliminated the space characters on the
left of the examples to make the examples fit better on the small
screen. Of course the programs were rendered utterly meaningless.

People complain about parenthesis but support invisible characters?
Amazing. I suppose an even better program is a totally blank screen.
You can imagine anything!

Perhaps the only language worse than Python is Whitespace!

Blake

[Pascal J. Bourguignon]

Blake McBride <blak...@gmail.com> writes:

> No, I haven't. Whitespace is a joke language, I wish python was.
>
> I had a project I had to do in Python. I bought a kindle book on
> Python. The kindle formatting eliminated the space characters on the
> left of the examples to make the examples fit better on the small
> screen. Of course the programs were rendered utterly meaningless.

LOL

> People complain about parenthesis but support invisible characters?
> Amazing. I suppose an even better program is a totally blank
> screen. You can imagine anything!
>
> Perhaps the only language worse than Python is Whitespace!

Yes.

[Rainer Joswig]

On 2015-08-30 08:54:17 +0000, Blake McBride said:

> The great God Steve Jobs killed Flash because Adobe made Windows their
> primary platform for their publishing software.

Steve Jobs made Flash unwanted on iOS because Flash is shit. An amazing
piece of shit.
Adobe Flash is among the worst pieces of software ever invented.
It has an endless stream of bugs, massive security problems,
inefficiency on mobile hardware, ...

[Rainer Joswig]

On 2015-08-30 07:02:42 +0000, Pascal J. Bourguignon said:

...

> On a unix (GNU/linux) system:
>
> $ ls ~/Desktop/*.txt
> /home/pjb/Desktop/20140308--apple--commande.txt
> /home/pjb/Desktop/220446114-At-Last-Why-Sugar-Kills.txt
> /home/pjb/Desktop/DOC-488531.txt
> /home/pjb/Desktop/out.txt
> /home/pjb/Desktop/pjb-notes.txt
>
> files are not assigned to an application. You can process them with any
> tool you want:
>
> $ grep -A1 deadline ~/Desktop/pjb-notes.txt
> | I love deadlines. I like the whooshing sound they make as they fly by.
> | -- Douglas Adams
> $ emacs ~/Desktop/pjb-notes.txt
> ...

If you had a Lisp Machine, you would even have sane commands and a better UI.
You could use an "edit file" command which opens the file in Zmacs,
instead you have to use strange application names under GNU/Linux.

You got the shitty Unix/Linux text based user interface, using an
editor with insane keyboard bindings and a shell with a strange command
language.

I still have my Unix Haters barf bag and it is as relevant today as it
was twenty years ago.

Personally I prefer GUI user interfaces for my personal machines

Even under GNU/Linux there is an actual GUI based UI, where you can
open files and there is a default to open them.

[Burp Reynalds]

On 2015-08-30, Rainer Joswig <jos...@lisp.de> wrote:

> Steve Jobs made Flash unwanted on iOS because Flash is shit. An amazing
> piece of shit.

True.

> Adobe Flash is among the worst pieces of software ever invented.

There are so many it's hard to rank them.

> It has an endless stream of bugs, massive security problems,
> inefficiency on mobile hardware, ...

Why single out Flash? Windows, Android, and even Linux all fall into the
above category.

Burp

[Blake McBride]

A. I see you too are a fan of the great one. Did you have to pay money
to buy into the spiel?

B. Flash is still the only platform that can produce a consistent UI
across everything.

C. Security holes and inefficiencies, yes. We can all use improvement
when we have so much else to offer.

D. Windows is the most popular software in the world. To quote a sage:

"Bill Gates made Windows unwanted on iOS because Winodws is shit. An
amazing piece of shit.
MS Windows is among the worst pieces of software ever invented.

It has an endless stream of bugs, massive security problems,
inefficiency on mobile hardware, ..."

Blake

[Rainer Joswig]

On 2015-08-30 17:25:14 +0000, Blake McBride said:

> On 08/30/2015 09:47 AM, Rainer Joswig wrote:
>> On 2015-08-30 08:54:17 +0000, Blake McBride said:
>>
>>> The great God Steve Jobs killed Flash because Adobe made Windows their
>>> primary platform for their publishing software.
>>
>> Steve Jobs made Flash unwanted on iOS because Flash is shit. An amazing
>> piece of shit.
>> Adobe Flash is among the worst pieces of software ever invented.
>> It has an endless stream of bugs, massive security problems,
>> inefficiency on mobile hardware, ...
>>
>
> A. I see you too are a fan of the great one.

Yes, I'm a fan of Steve Jobs. No apologies.

> Did you have to pay money to buy into the spiel?

WTF an idiot are you?

> B. Flash is still the only platform that can produce a consistent UI
> across everything.

Nobody cares. Flash was used for annoying shit mostly (ads) and for
stuff which should never been developed with Flash in the first place
(video).

> C. Security holes and inefficiencies, yes. We can all use improvement
> when we have so much else to offer.

Adobe was asked again and again and they constantly failed. Failed and
failed again. It got worse and worse. There is no way I would trust
them to provide a working, efficient and secure application framework.

> D. Windows is the most popular software in the world.

I don't care. I can manage my life without them.

'Did you have to pay money to buy into the spiel?'

[Rainer Joswig]

On 2015-08-30 17:25:14 +0000, Blake McBride said:

Blake, you don't need to send me this great content, slightly shorter,
additionally as an email.
Please don't. ;-)
Thanks.

[Blake McBride]

On 08/30/2015 01:09 PM, Rainer Joswig wrote:
> On 2015-08-30 17:25:14 +0000, Blake McBride said:
>
>> On 08/30/2015 09:47 AM, Rainer Joswig wrote:
>>> On 2015-08-30 08:54:17 +0000, Blake McBride said:
>>>
>>>> The great God Steve Jobs killed Flash because Adobe made Windows their
>>>> primary platform for their publishing software.
>>>
>>> Steve Jobs made Flash unwanted on iOS because Flash is shit. An amazing
>>> piece of shit.
>>> Adobe Flash is among the worst pieces of software ever invented.
>>> It has an endless stream of bugs, massive security problems,
>>> inefficiency on mobile hardware, ...
>>>
>>
>> A. I see you too are a fan of the great one.
>
> Yes, I'm a fan of Steve Jobs. No apologies.
>
>> Did you have to pay money to buy into the spiel?
>
> WTF an idiot are you?
>
>> B. Flash is still the only platform that can produce a consistent UI
>> across everything.
>
> Nobody cares. Flash was used for annoying shit mostly (ads) and for
> stuff which should never been developed with Flash in the first place
> (video).

Actually, I have a web-app with 450+ Flash GUI screens. It appears
perfectly on all Windows, Macs, Linux, and tablets (tablets with Puffin
browser or any other that supports Flash). It also appears and operates
totally consistently on all versions of Chrome, IE, Firefox, Safari,
etc. There is not one line of target specific code! Can you say that
about HTML5?

>
>> C. Security holes and inefficiencies, yes. We can all use
>> improvement when we have so much else to offer.
>
> Adobe was asked again and again and they constantly failed. Failed and
> failed again. It got worse and worse. There is no way I would trust them
> to provide a working, efficient and secure application framework.

I am not a fan of Adobe. Quite the contrary.

>
>> D. Windows is the most popular software in the world.
>
> I don't care. I can manage my life without them.
>
> 'Did you have to pay money to buy into the spiel?'
>

Didn't pay, didn't buy.

Blake

[Blake McBride]

On 08/30/2015 01:58 PM, Rainer Joswig wrote:
> On 2015-08-30 17:25:14 +0000, Blake McBride said:
>
> Blake, you don't need to send me this great content, slightly shorter,
> additionally as an email.
> Please don't. ;-)
> Thanks.
>

Sorry, in that case I accidentally hit "Reply" instead of "Followup" on
Thunderbird.

[Rainer Joswig]

On 2015-08-30 19:08:27 +0000, Blake McBride said:

> Actually, I have a web-app with 450+ Flash GUI screens.

A 'web-app' in Flash. OMG.

[Blake McBride]

Yea. Started it in '06. Communicates over SOAP web services. Was the
best idea at the time. Still works very, very well.

[Rainer Joswig]

On 2015-08-30 19:08:27 +0000, Blake McBride said:

> Actually, I have a web-app with 450+ Flash GUI screens. It appears
> perfectly on all Windows, Macs, Linux, and tablets (tablets with Puffin
> browser or any other that supports Flash). It also appears and
> operates totally consistently on all versions of Chrome, IE, Firefox,
> Safari, etc. There is not one line of target specific code! Can you
> say that about HTML5?

I hope you are the only user...

http://www.computerworld.com/article/2971721/security/stop-the-flash-madness-5-bugs-a-week.html


> On August 11, 2015 Adobe fixed 34 bugs in their Flash Player
software. Many of the bugs enabled a computer to get infected with
malicious software simply by viewing a web page.
...
> This adds up to 164 bug fixes so far in 2015, a year that was 223
days old when the last group of patches were released on the 11th. In
round numbers, this comes out to 1 bug fix every 33 hours for 2015.

Good luck with that.

Advice from ComputerWorld:

> Un-install the Flash Player. Do the world a favor.

Do it now.

[Kaz Kylheku]

On 2015-08-30, Burp Reynalds <br...@miami-vice.com> wrote:
> On 2015-08-30, Rainer Joswig <jos...@lisp.de> wrote:
>> Adobe Flash is among the worst pieces of software ever invented.
>
> There are so many it's hard to rank them.

And that's before we look beyond Adobe.

[Joost Kremers]

Blake McBride wrote:
> I didn't know about paredit. I just downloaded it and started playing
> with it. It is very, very nice!!!

Note that smartparens[1] provides much the same functionality, some of
it even for languages other than Lisp. (You'll want to set
`(smartparens-strict-mode 1)` in your Lisp modes, though, to make sure
you cannot (easily) delete parens).

Another interesting take is the `lispy` package.[2] It binds
structure-based editing and movement commands to simple keys but only
activates them when point is somewhere where they make sense (and where
inserting a character doesn't make sense). Basically when point is at an
opening or closing paren.


[1] https://github.com/Fuco1/smartparens

[2] https://github.com/abo-abo/lispy

--
Joost Kremers joostk...@fastmail.fm
Selbst in die Unterwelt dringt durch Spalten Licht
EN:SiS(9)

[Marco Antoniotti]

On Sunday, August 30, 2015 at 5:07:47 PM UTC+3, informatimago wrote:
> Blake McBride <blak...@gmail.com> writes:
>
> > No, I haven't. Whitespace is a joke language, I wish python was.
> >
> > I had a project I had to do in Python. I bought a kindle book on
> > Python. The kindle formatting eliminated the space characters on the
> > left of the examples to make the examples fit better on the small
> > screen. Of course the programs were rendered utterly meaningless.
>
> LOL
>
> > People complain about parenthesis but support invisible characters?
> > Amazing. I suppose an even better program is a totally blank
> > screen. You can imagine anything!
> >
> > Perhaps the only language worse than Python is Whitespace!
>
> Yes.

Nope. Python code and Whitespace programs are affected in the same way by the following command (on Linux)

$ sudo irsabol -r / -pattern '*.{py,ws}'

Cheers
--
MA

[WJ]

Pascal J. Bourguignon wrote:

> > I remember reading papers in SIGPLAN or LCS Technical Reports 30+ years
> > ago about this. Some people theorized that the file-based model would go
> > away within a decade, as GUI programming languages emerged. They seemed
> > to think that non-programmers would be able to do real programming with
> > drag-and-drop.
> >

> > As we can see, this never happened.
>
> Graphical programming languages exist, here are a few examples:
>
> http://www.dangermouse.net/esoteric/piet.html
> http://code.google.com/p/blockly
> http://drakon-editor.sourceforge.net/language.html
>
> What doesn't happen though is non-programmers doing real programming.

Gauche Scheme:

(define word-lists
(map
(lambda (str) (filter #/./ (string-split str #/[^[:alpha:]'-]+/)))
(list

"They seemed to think that non-programmers would be able to do
real programming with drag-and-drop."

"What doesn't happen though is non-programmers doing real
programming.")))

(use util.lcs) ;; longest common subsequence

(apply lcs word-lists)
===>
("non-programmers" "real" "programming")

--
[W]e had enough employees who made more than 85 to fill all the openings. The
highest score that any of the blacks scored on the test was 11. The lowest
score that any black made on the test was 4. All four of those blacks went
into skilled-trades training.
https://archive.org/download/TheOldmanArchives/oldman29-081003.mp3

[budden]

Hi Blake!
> A popular development scenario that I term "disk based" looks like the
> following:
>
> A. A program is created / edited with a text editor (like emacs)
>
> B. The code is saved to a disk file (like myprogram.lisp )
>
> C. A language interpreter / compiler loads (from disk or a socket)
> and runs the code where the programmer evaluates its performance
>
> D. Go back to A.

You forgot one subtlety which is special for common lisp:

C. One function, type, macro, or class source code is being changed in the already running lisp program.
D. Source code of only that specific object is recompiled (without stopping the program).
E. Go back to C.

Image-based development is used nowadays. Most extensive example is SQL database. Also there is, say, Microsoft Access.

In practice when you support large application, you have the following issues:
1. Multiple developers can work on the same project simultaneously.
2. Control of changes must be done as complex things are fragile.
3. Recording history of changes is required for stable operation.

This is usually solved well with keeping _sources_. For SQL databases, I usually keep textual representation in a bunch of *.sql files. That is, I move from image-based to source-based scenario.

Anyway, to support application you need to implement some operation on images:
1. Dump. Present any object in the image in the some human-readable (e.g. textual) form.
2. Compare. Compare two dumps or two images to see which objects have changed.
3. Coerce. Alter one (production) image to the state of other (test) image with minimal effort. E.g. without full rebuilding of production image and with pre-testing. This can not be done automatically in general case. E.g. when data type change you might need to update instances of objects by some data from the outside.
4. Merge. Move changes from two dumps into a new dump (or from two images into a new image, without dumping). Merge must allow per-change control by user.
5. RCS Check in/Check out - save dump to revision control system or load dump from it. Revision control system is a crucial productivity tool in long-term development and is commonly used today for all software projects.
6. (Optional) Rebuild - build image from the dump.

All SQL databases I know allow support for most of these operations via SQL dumps and SQL scripting. SQL script (dump) of database can usually be generated for database out of the box, and database structure can be rebild from it. Data can also be loaded or unloaded.

SQL is a simple language. All objects have names, packed to enumeratable namespaces and refer to each other by names. Common lisp have uninterned symbols, closures, unprintable objects, eval-read. Essential effort is required to dump lisp image in such a way which supports operations 1-6 and there is no standard way to dump lisp image in a human-readable form. Even if dumped, dump size can be enormous. There are also complexities in defining "state" of lisp images as we have linked FFI code, opened streams, GUI resources and other links to state of the operating system which can not be saved together with the image. So save/load operation of image is always lossy and it imposes restrictions to software of way of its management.

My opinion (based on rather extensive SQL and Lisp experience) - modern way suggested by SLIME, when you have both sources and image in coordinated evolution is much better than pure image-based development.

[Pascal J. Bourguignon]

budden <budde...@mail.ru> writes:

> This is usually solved well with keeping _sources_. For SQL databases,
> I usually keep textual representation in a bunch of *.sql files. That
> is, I move from image-based to source-based scenario.

You don't need to KEEP sources, if the system is able to GENERATE
sources from the internal datastructures.

Cf. for example com.informatimago.ibcl which keeps the last definition
in a hash-table to be able to produce a source.

Granted the source generated could be different from the actual image,
but it's actually a feature, since it's assumed that any missing
redefinitions in the image are unwanted:

C/IBCL-USER[1]> (defun f () 'first)
F
C/IBCL-USER[2]> (defparameter *p* #'f)
*P*
C/IBCL-USER[3]> (defun f () 'second)
F
C/IBCL-USER[4]> (save-sources *standard-output* :package *package*)

(IN-PACKAGE "IMAGE-BASED-COMMON-LISP-USER")
(DEFUN F NIL 'SECOND)
(DEFPARAMETER *P* #'F)

C/IBCL-USER[5]>

Loading the source will produce a different binding to *P* than what
exists in the image.

If you want the image, use the image…

> My opinion (based on rather extensive SQL and Lisp experience) -
> modern way suggested by SLIME, when you have both sources and image in
> coordinated evolution is much better than pure image-based
> development.

You can also keep the source INSIDE the image!

[Rob Warnock]

Blake McBride <blak...@gmail.com> wrote:
+---------------

| I been interested in image based development for some time.

+---------------

Besides what others have said, for me to tolerate an
"image-based" system or even a "structure editor",
I'd want the ability to save comments *AS I WROTE THEM*
and export them as well when the image is exported
[upon request] to a plain-text source file.


-Rob

-----
Rob Warnock <rp...@rpw3.org>
627 26th Avenue <http://rpw3.org/>
San Mateo, CA 94403

[n2kr...@gmail.com]

Any suggestions for UNDO, a versioning / cvs filesystem?
Which Linux FS or ISO 9660 r/w?
hFS+ (Time Machine) or NTFS could have in theory.
Maybe a lisp VFS layer even if it just shadows the outer OS VFS.

[CAI GENGYANG]

Blake McBride ,

Do you have a visual diagram of what you want to build ? A flow chart of how it works ... easier to give constructive comments when I can see what it looks like ..

On Saturday, August 29, 2015 at 1:03:07 AM UTC+8, Blake McBride wrote:
> A popular development scenario that I term "disk based" looks like the
> following:
>
> A. A program is created / edited with a text editor (like emacs)
>
> B. The code is saved to a disk file (like myprogram.lisp )
>
> C. A language interpreter / compiler loads (from disk or a socket)
> and runs the code where the programmer evaluates its performance
>
> D. Go back to A.
>

> ---
>
> There is another development scenario that is old, but may be worth
> looking into. I term this other method "image based" and it looks like
> the following:
>
> 1. Code is added and edited right in the environment where it is being
> used (like CCL, SBCL, CLISP, etc.). Editing the code could be done
> with an editor like a structure editor written in Lisp rather than a
> text editor.
>
> 2. Run the code right in the environment it was created in.
>
> 3. Go to step 1 in order to continue development.
>
> Also:
>
> 4. The state of the project can be saved as an image file
> representing the entire project when necessary (i.e. CLISP .mem file,
> SBCL and CMUCL .core file, CCL image file, etc.).
>
> 5. A disk / text form of the application can also be created for
> reference or porting whenever needed (like mypackage.lisp ).
>
> Languages that use this latter model include Smalltalk and InterLisp.
>
> ---
>
> In order to do image based development in Common Lisp, certain tools
> are needed:
>
> a. A way of editing a program, like a structure editor
>
> b. A way of writing a text form of the application when desired (item 5
> above)
>
> c. The interpreter / compiler should be able to save / load an image
>
> I have completed item "a" (to be released shortly). I will write item
> "b" shortly. Item "c" is already included in many Lisp interpreters /
> compilers. Those without it can save/load a text form that will be
> provided by item "b", although this is somewhat clunky.
>
> I been interested in image based development for some time. I thought
> this would be a good road to investigate. Your input is appreciated.
>
> Thanks.
>
> Blake McBride