Question from a non-user: Primitives
Bennu Strelitzia
will make separate postings about. Sorry if they are obvious or
otherwise-unsuitable questions, but I have spent some time reading
trying to answer them and have not been able to do so.
Introduction of new primitives/atoms in Lisp
Is it common to introduce new primitives/atoms into a Lisp environment?
What if I want to introduce lots of new data primitives.
What if I define my own value space of foreign atoms that represents
every conceivable type of simple or structured value with functions to
manipulate them. I only really want to use the Lisp model for coding to
operate on those values. I will not feel tempted to cons/car/cdr down a
traditional Lisp data list or collection or do much of anything that
relies on the traditional Lisp way of modelling data, but instead want
to use my own universal data model implemented as atoms underneath the
covers.
Does this become too impure to be contemplated? Is there enough of Lisp
left that makes it still a valuable choice for defining functional
programming on this foreign value space?
Are there implicit conversions or other things that these foreign atoms
have to subscribe to?
What is the best document to read on this sort of thing?
Bennu
Tamas K Papp
> I would appreciate any pointers you may have on a few issues which I
> will make separate postings about. Sorry if they are obvious or
> otherwise-unsuitable questions, but I have spent some time reading
> trying to answer them and have not been able to do so.
>
> Introduction of new primitives/atoms in Lisp
>
> Is it common to introduce new primitives/atoms into a Lisp environment?
>
> What if I want to introduce lots of new data primitives.
I don't really see what you are trying to do. Perhaps some examples
would help.
> What if I define my own value space of foreign atoms that represents
> every conceivable type of simple or structured value with functions to
> manipulate them. I only really want to use the Lisp model for coding to
> operate on those values. I will not feel tempted to cons/car/cdr down a
> traditional Lisp data list or collection or do much of anything that
> relies on the traditional Lisp way of modelling data, but instead want
> to use my own universal data model implemented as atoms underneath the
> covers.
Presumably you want to do something with those atoms, so you will have
operations that handle them. There is no "traditional Lisp way of
modeling data", you can pretty much do what you want. Most people
find existing data types useful, but there is nothing to prevent you
from representing stuff as bits or bytes, eg
(deftype insanely-general-atomic-type ()
`(simple-array bit (*)))
and then keep manipulating these objects.
> Does this become too impure to be contemplated? Is there enough of Lisp
"too impure to be contemplated"? Gosh. Do you actually have a
problem you are trying to solve, or is this one of those grand thought
experiments about nothing?
> left that makes it still a valuable choice for defining functional
> programming on this foreign value space?
Lisp is not functional. You can make up a dialect of Lisp that is
functional. The big advantage of Lisp is macros: you can manipulate
your code as data. Since you don't want to handle standard Lisp
constructs, you will have to make up an alternative representation of
code that corresponds well to your idea of data. Pretty pointless,
IMO.
> What is the best document to read on this sort of thing?
Perhaps a text on Zen Buddhism. But you need to find someone that
beats you with a stick to get the full benefit.
Tamas
fortunatus
wrote:
> Introduction of new primitives/atoms in Lisp
> Is it common to introduce new primitives/atoms into a Lisp environment?
Not typically - usually define structures or classes to do regular
programming...
> What if I want to introduce lots of new data primitives.
> What if I define my own value space of foreign atoms ...
> Does this become too impure to be contemplated?
Do whatever you want. I think folks do this kind of experimentation
all the time.
> Is there enough of Lisp
> left that makes it still a valuable choice for defining functional
> programming on this foreign value space?
As already mentioned by Tamas, if you plan to stick with Lisp syntax
for your source code, then you can use macros to build control
constructs. That would be a primary reason to use Lisp for your
experiment.
For example if you were to use C or Pascal for your experiments, you
would be able to define functions for your operations, but you would
not be able to build new control constructs. In that environment you
would need to fully build your own whole language including parser,
interpreter or code generator, etc. Unless you are happy with the
basic control constructs, and you just want to make a function
library.
If you are targeting functional programming style, then some of Lisp's
function handling might be interesting to you - for instance being
able to write a (LAMBDA ...) in-line and pass it or return it. It
goes along with its "closed" environment (closure). In C you can
handle pointers to functions, but it's a bit easier in Lisp.
> Are there implicit conversions or other things that these foreign atoms
> have to subscribe to?
Up to you. If you are really defining a new universe of atoms there
may be little need to consider any conversions to existing data
types. Certainly there is no regular expectation of doing so in Lisp
style.
Bennu Strelitzia
> I don't really see what you are trying to do. Perhaps some examples
> would help.
I have defined my own integrated generic value space and representation
for data that I like for its universal nature. In my applications, I use
it for virtually everything. You would have to take it for granted that
I find this representation of great value for the purposes of this
discussion until I am able at some future point to release the
documentation (and I am sure it would continue to be refined by others).
Think of it as a virtual-machine-like generalization, but one that only
applies to the data.
It is fairly easy to use if I construct my own language, which I have
done with a relatively high degree of success, but I am trying to
explore the possible use in adopting this value space / data model into
existing languages, because there are lots of languages out there that
have significant success over many years representing programming, not
only well developed, but also more familiar to masses of people than the
language I somewhat-arbitrarily constructed to manipulate the data model.
Even for a simple operation like (+ 10 20), I really need my own data
model so that the primitives can support the model in the appropriate
fashion including such things as memberships of primitives in multiple
overlapping type sets, i.e. is there any difference between the value
that represents 30 as an integer, a rational, and an algebraic number,
caching known info about the numbers introduced into the system such as
factorization, etc. As I write this I see that it could possibly be done
with existing primitives in some cases.
My data model goes significant beyond numbers to cannonical non-string
representations of many values of various domains that today are loosely
thrown around inside of strings, containers/maps/functions as values,
and much more.
[...]
> Presumably you want to do something with those atoms, so you will have
> operations that handle them. There is no "traditional Lisp way of
> modeling data", you can pretty much do what you want. Most people
> find existing data types useful, but there is nothing to prevent you
> from representing stuff as bits or bytes, eg
>
> (deftype insanely-general-atomic-type ()
> `(simple-array bit (*)))
>
> and then keep manipulating these objects.
That is helpful. I am just trying to determine the residual value in the
language if I replace the value space of the data it manipulates,
assuming I supply corresponding functions to manipulate the values.
>> Does this become too impure to be contemplated? Is there enough of Lisp
>
> "too impure to be contemplated"? Gosh. Do you actually have a
> problem you are trying to solve, or is this one of those grand thought
> experiments about nothing?
I have read significant threads between users of Lisp complaining that
one form or another of some Lisp or another isn't Lisp because they have
"broken" some apect the language's structure.
While I realize someone contemplating using the new hybrid will
ultimately have to be the judge of the value of Lisp with a new set of
atoms (and it remains to be seen to what degree existing atoms can be
made compatible), I am just seeking some discussion before I head down
that road. Perhaps the discussion is not as practical as I had imagined.
We are currently talking at an absract-enough level that I appreciate
the difficulty of assessing to what degree the Lisp language is still a
useful way of programming it. It is by no means an acedemic question,
but one that I have not supplied enough details on. I was hoping by the
question to possibly gain some better insight about what that might to
the language if the value space it operates on changes significantly.
Ultimately I will find out when I get there, of course.
> Lisp is not functional. You can make up a dialect of Lisp that is
> functional. The big advantage of Lisp is macros: you can manipulate
> your code as data. Since you don't want to handle standard Lisp
> constructs, you will have to make up an alternative representation of
> code that corresponds well to your idea of data. Pretty pointless,
> IMO.
That is an important consideration. The language I created that I use
today has this capability, to manipulate code as data, and it is clearly
valuable.
Bennu Strelitzia
> I don't really see what you are trying to do. Perhaps some examples
> would help.
I have defined my own integrated generic value space and representation
[...]
> Presumably you want to do something with those atoms, so you will have
> operations that handle them. There is no "traditional Lisp way of
> modeling data", you can pretty much do what you want. Most people
> find existing data types useful, but there is nothing to prevent you
> from representing stuff as bits or bytes, eg
>
> (deftype insanely-general-atomic-type ()
> `(simple-array bit (*)))
>
> and then keep manipulating these objects.
That is helpful. I am just trying to determine the residual value in the
language if I replace the value space of the data it manipulates,
assuming I supply corresponding functions to manipulate the values.
>> Does this become too impure to be contemplated? Is there enough of Lisp
>
> "too impure to be contemplated"? Gosh. Do you actually have a
> problem you are trying to solve, or is this one of those grand thought
> experiments about nothing?
I have read significant threads between users of Lisp complaining that
one form or another of some Lisp or another isn't Lisp because they have
"broken" some apect the language's structure.
While I realize someone contemplating using the new hybrid will
ultimately have to be the judge of the value of Lisp with a new set of
atoms (and it remains to be seen to what degree existing atoms can be
made compatible), I am just seeking some discussion before I head down
that road. Perhaps the discussion is not as practical as I had imagined.
We are currently talking at an absract-enough level that I appreciate
the difficulty of assessing to what degree the Lisp language is still a
useful way of programming it. It is by no means an acedemic question,
but one that I have not supplied enough details on. I was hoping by the
question to possibly gain some better insight about what that might to
the language if the value space it operates on changes significantly.
Ultimately I will find out when I get there, of course.
> Lisp is not functional. You can make up a dialect of Lisp that is
> functional. The big advantage of Lisp is macros: you can manipulate
> your code as data. Since you don't want to handle standard Lisp
> constructs, you will have to make up an alternative representation of
> code that corresponds well to your idea of data. Pretty pointless,
> IMO.
That is an important consideration. The language I created that I use
Pillsy
wrote:
[...]
> > I don't really see what you are trying to do. Perhaps some examples
> > would help.
> Think of it as a virtual-machine-like generalization, but one that only
> applies to the data.
OK.
[...]
> Even for a simple operation like (+ 10 20), I really need my own data
> model so that the primitives can support the model in the appropriate
> fashion including such things as memberships of primitives in multiple
> overlapping type sets, i.e. is there any difference between the value
> that represents 30 as an integer, a rational, and an algebraic number,
> caching known info about the numbers introduced into the system such as
> factorization, etc. As I write this I see that it could possibly be done
> with existing primitives in some cases.
Well, + isn't really a primitive in (Common) Lisp, it's a function.
There's no reason for it not to be, since it's not like it's anything
special syntactically. If you want to define your own function named
+, there's really nothing stopping you, provided you do it in your own
"package". You can then have it do whatever sort of dispatch you want.
You can also have it make use of the built-in + function from the
standard "CL" package. For instance:
(defpackage "MY-PACKAGE"
(:using "CL")
(:shadow "+"))
(in-package "MY-PACKAGE")
(defun + (x y)
(if (and (my-fancy-type-p x)
(my-fancy-type-p y))
(my-fancy-type-+ x y)
(cl:+ x y)))
Without knowing more about what your "data model" looks like, it's
difficult to provide more information.
Cheers,
Pillsy
Duane Rettig
wrote:
> I would appreciate any pointers you may have on a few issues which I
> will make separate postings about. Sorry if they are obvious or
> otherwise-unsuitable questions, but I have spent some time reading
> trying to answer them and have not been able to do so.
>
> Introduction of new primitives/atoms in Lisp
>
> Is it common to introduce new primitives/atoms into a Lisp environment?
Since Common Lisp is an extensible language, it is certainly common to
introduce new elements into the Lisp environment. However, most Lisp
users who do so do so by creating Domain Specific Languages; little
languages built on top of the base lisp which define the specific
problem to be solved.
> What if I want to introduce lots of new data primitives.
Create a DSL for your problem domain, and then have at it. Lisp is
perfect for doing that.
> What if I define my own value space of foreign atoms that represents
> every conceivable type of simple or structured value with functions to
> manipulate them. I only really want to use the Lisp model for coding to
> operate on those values. I will not feel tempted to cons/car/cdr down a
> traditional Lisp data list or collection or do much of anything that
> relies on the traditional Lisp way of modelling data, but instead want
> to use my own universal data model implemented as atoms underneath the
> covers.
Other Lisps likely have similar concepts, but Allegro CL specifically
has a Foreign Type interface, in which you can model any structural
datatype as if it were a first-class datatype. See
http://www.franz.com/support/documentation/8.1/doc/ftype.htm
> Does this become too impure to be contemplated? Is there enough of Lisp
> left that makes it still a valuable choice for defining functional
> programming on this foreign value space?
"impure"? At least Common Lisp has _never_ put itself forward as
anything "pure". Almost everything you do in CL hides some of the
CL. It's a feature...
> Are there implicit conversions or other things that these foreign atoms
> have to subscribe to?
Again, in the case of Allegro CL, foreign types are documented as to
how they act, and in addition if you manipulate them in the Lisp heap
and call foreign functions with them, the
transformation rules are fairly intuitive. See
http://www.franz.com/support/documentation/8.1/doc/foreign-functions.htm
> What is the best document to read on this sort of thing?
Don't know how to answer this one. You seem semi-versed in Lisp
concepts, but the questions you're asking don't speak to much
experience programming in Lisp (especially Common Lisp). My
recommendation is to download a CL (most can be had for free, even
from the commercial vendors), go through any tutorials available, and
then ask specific questions like you're doing earlier in this post.
Duane
Kaz Kylheku
> I would appreciate any pointers you may have on a few issues which I
> will make separate postings about. Sorry if they are obvious or
> otherwise-unsuitable questions, but I have spent some time reading
> trying to answer them and have not been able to do so.
>
> Introduction of new primitives/atoms in Lisp
Hacking the implementation at this level is not a feature of ANSI Common
Lisp. Where foreign objects are supported, by implementations, they
usually look like some ``foreign object'' type or types. But that
doesn't imply they are second class in any way.
> Are there implicit conversions or other things that these foreign atoms
> have to subscribe to?
No; Lisp doesn't do conversions like C or C++ on assignment or
argument passing. A value keeps its representation (including type) when
it is passed as an argument to a function, assigned, et cetera.
So for instance, there is no such thing as a function which accepts a
floating point argument, such that when an integer value is passed, it
is automatically converted to floating point in the function call
itself.
> What is the best document to read on this sort of thing?
An actual reference manual! Where you can find out that Lisp has types
other than lists, such as classes and structures, so that you
don't have hack the implementation on a low level to add new types.
Generally if you want to know how to do something in language X,
it behooves you to study the reference manual for language X,
together with some good introduction (since reference manuals
can be heavy to digest).
The Lisp spec is online in HTML form (search for ``Common Lisp
HyperSpec''). There are numerous good books out there, too.
Kaz Kylheku
>> I don't really see what you are trying to do. Perhaps some examples
>> would help.
>
> I have defined my own integrated generic value space and representation
> for data that I like for its universal nature. In my applications, I use
> it for virtually everything.
Oh boy. ``I wrote a great library once for representing data which
neatly solved various problems that I was coping with. And now I want to
drag it with me into every project, every programming language, ...''
> Even for a simple operation like (+ 10 20), I really need my own data
> model so that the primitives can support the model in the appropriate
> fashion including such things as memberships of primitives in multiple
> overlapping type sets, i.e. is there any difference between the value
> that represents 30 as an integer, a rational, and an algebraic number,
Welcome to Lisp. 30 is simultaneously of type INTEGER and of type
RATIONAL.
(rationalp 30) -> T
(integerp 30) -> T
Is your system of numbers any better than the numeric tower that is
language built into Common Lisp? You should evaluate it and find out.
You can easily create your own.
The standard Lisp + function (and others) aren't polymorphic (cannot
be extended by the user into new domains), but that's just the symbol
COMMON-LISP::+. You can define your own package in which lives your own
symbol called +, and you can bind that to a polymorphic function which
works with whatever kinds of numbers you like (including
interoperability with Lisp numbers).
Lisp is pretty good for experimenting with new programming language
features. Probably your best bet compared to anything else.
> My data model goes significant beyond numbers to cannonical non-string
> representations of many values of various domains that today are loosely
> thrown around inside of strings, containers/maps/functions as values,
> and much more.
You will discover that these things are also not loosely thrown around
inside strings in Lisp, that containers are values, etc.
It may be worth it to take a fresh look at everything; learn the new
language, and then see how much of the data model is still valuable
in it.
> I have read significant threads between users of Lisp complaining that
> one form or another of some Lisp or another isn't Lisp because they have
> "broken" some apect the language's structure.
If you do something that works, others don't have to like it.
Sometimes others have poor quality ideas, and dislike something
for poor reasons.