[racket] Racket in the large

[gonzalo diethelm]

Hello everyone; this is my first post to this list. I present my apologies beforehand, because I know these kinds of topics can be easily misconstrued as trolling attempts or “mental masturbation”; all I can say in my favor is that these are real, honest questions on my part. If any part of this has been discussed before (and I cannot imagine it has not), please point me to the relevant places and I shall leave in peace; I must say I did go over the full Google Groups archive for this mailing list and could not find anything to answer my questions.

 

I would like to start by saying that I have always had a strong attraction to Lisp-like languages. In the past years I have been reading on functional programming topics, specially about Scheme, the Y combinator, macros, Paul Graham’s ramblings, etc., and I can say that I have finally come to understand the origins of my attraction: the purity, simplicity (of the basic concepts), elegance and power in functional programming (in general) and Scheme (in particular) are, in my view, unbeatable. So I declare myself an outright Scheme lover.

 

In addition to that, my professional career has always been tied to programming and software engineering, because that is what I enjoy doing the most (as opposed to managing). I have had long stints (several years) of developing in C, C++ and Java. Incidentally, when I am programming in language X in that list, I usually find myself pining for language X-1 (from Java to C++, from C++ to C); but I digress…

 

The thing is, I must admit I have a secret desire and it is time to come out in the open: I want to leave those imperative languages behind and finally use Scheme for a real, “in the large” project… There, I feel better now that I said that!

 

So let me get to my point before I bore everyone here to death: can my desire be realistically fulfilled? Can Scheme (Racket) be used for a more “enterprise-y” project (console app, GUI app, web app, whatever, accessing data from any RDBMS in transactional ways) where I will have a team of developers working on separate parts of the system at the same time? How can Scheme (Racket) help me with the software engineering aspects of such a project? Has anybody here had any experience in projects similar to this?

 

Which brings me to my second question: can Scheme (Racket) be used to develop all these different kinds of applications (console, GUI, etc.)? Can anybody point me to real life examples of each type of application developed with Scheme (Racket)?

 

[In fact, I specifically came to Racket after having had a grand vision of being able to develop all the components for a web app (SQL to access the DB, business logic, HTML for presentation, CSS for styling, JS for interactivity, even XML for configuration) using a single unified language (or many small separate DSLs based on a single language) and I have been kind of surprised at not finding this idea fleshed out anywhere as a framework, library, module, etc.]

 

I guess at the bottom of my questions lurks the following (and this is the non-trolling I was referring to before): if Scheme is the be-all and end-all of programming languages, how come I don’t hear of more success stories using it, other than Paul Graham’s (which, important as they are, in my view are not recent enough to provoke and inspire anymore)? Where are the flag web applications, GUI enterprise systems, even console applications, that have become watershed proofs of Scheme (Racket) as THE SINGLE BEST programming language there is (something of which I am 100% convinced)?

 

I really would appreciate (and am thankful for) any insight coming from the illustrious people on this list. Best regards.

 

--

Gonzalo Diethelm

 

---

Declaración de confidencialidad: Este Mensaje esta destinado para el uso de la o las personas o entidades a quien ha sido dirigido y puede contener información reservada y confidencial que no puede ser divulgada, difundida, ni aprovechada en forma alguna. El uso no autorizado de la información contenida en este correo podrá ser sancionado de conformidad con la ley chilena. Si usted ha recibido este correo electrónico por error, le pedimos eliminarlo junto con los archivos adjuntos y avisar inmediatamente al remitente, respondiendo este mensaje. Disclosure: This Message is to be used by the individual, individuals or entities that it is addressed to and may include private and confidential information that may not be disclosed, made public nor used in any way at all. Unauthorized use of the information in this electronic mail message may be subject to the penalties set forth by Chilean law. If you have received this electronic mail message in error, we ask you to destroy the message and its attached file(s) and to immediately notify the sender by answering this message.

[Matthias Felleisen]

On Aug 19, 2011, at 6:07 PM, gonzalo diethelm wrote:

Can Scheme (Racket) be used for a more “enterprise-y” project (console app, GUI app, web app, whatever, accessing data from any RDBMS in transactional ways) where I will have a team of developers working on separate parts of the system at the same time? How can Scheme (Racket) help me with the software engineering aspects of such a project? Has anybody here had any experience in projects similar to this?

Yes for Scheme and yes for Racket. Example: Disney World has some virtual rides running on Chez Scheme. Example: Boeing/USAF use Racket to control large, expensive telescope arrays. 

Which brings me to my second question: can Scheme (Racket) be used to develop all these different kinds of applications (console, GUI, etc.)? Can anybody point me to real life examples of each type of application developed with Scheme (Racket)?

Yes. See above. 

If Scheme is the be-all and end-all of programming languages, how come I don’t hear of more success stories using it

Correction: Scheme (current standard) is the third-best language. Racket is the second-best language. 

Compared to C++, Racket is an infant. This is especially true when you think in terms of programming language evolution. 

So the first answer is come back when Racket is as old as C++. 

The second answer is a simple statement about history and business. When tool X enables the constructions of systems S and T and is older than tool Y, then people will build systems S and T in X. Because people wish to get things done, not wait for some mythical perfect thing. Once a system S is built in X, there is little incentive to re-build the system in Y. Why? It costs money to do so. A good businessman must wonder whether he can recoup this money.  Based on historical experience we also know that the reconstruction of a system makes it buggier.  Some bugs will be avoided, new ones will be invented. Why introduce buggy behavior when the system is reliable enough to make money in the market?  Equally important, the world of software gets built in layers. When layer Bot is built in X, traditionalists believe that it is best to build the layer on top of Bot, call it Mid, in X, too. Guess what they use to build layer Top? Even when a new language finally injects radically new ideas into the commercial world of software, as Java did with sound/safe types and safe memory/gc, programmers tend to deal with it as if it were the previous language. Indeed, you will find Java code that looks like Fortran code. I am sure when Racket's popularity explodes, you will find Racket code that looks like Fortran code, too. 

-- Matthias

[Mark Engelberg]

On Fri, Aug 19, 2011 at 3:26 PM, Matthias Felleisen
<matt...@ccs.neu.edu> wrote:
> Correction: Scheme (current standard) is the third-best language. Racket is
> the second-best language.

OK, this is clearly a setup for someone to ask, so I'll ask :)
What's the first-best language?

_________________________________________________
For list-related administrative tasks:
http://lists.racket-lang.org/listinfo/users

[Neil Van Dyke]

First, due to what journalists call "full disclosure", I should mention that I make a living as a consultant who uses Racket almost exclusively now, and will probably write a Racket software engineering book soon, and so I have a conflict of interest in encouraging new projects to use Racket. :)

gonzalo diethelm wrote at 08/19/2011 06:07 PM:

So let me get to my point before I bore everyone here to death: can my desire be realistically fulfilled? Can Scheme (Racket) be used for a more “enterprise-y” project (console app, GUI app, web app, whatever, accessing data from any RDBMS in transactional ways) where I will have a team of developers working on separate parts of the system at the same time?

Yes.  One of my consulting clients has a large code base (order of magnitude larger than the Racket/DrRacket distribution and its standard libraries), in production for years, making heavy use of PostgreSQL.

Regarding project management, most of the usual practices apply, and there are only a few differences with Racket that I see, and you can predict what I'll say, but I will spell out for the larger audience here:

* The first biggest difference with Racket that you *must* consider is what, at least in the US, industry used to call "nobody ever got fired for buying IBM": any time you use something that not everyone considers a "safe" choice, that is a political vulnerability, so you need to have the political power to choose.  (This is not exclusive to Racket.  Separate from any technical considerations, choosing Python is politically risky in some environments, and even Java can be politically risky unless you're doing a routine intranet Java enterprise app and using a "safe" framework.  I think the difference here is "Why are you using Java/Python when I think it is unsuitable for this purpose," vs. "Why are you using something on which I do not already have an opinion?" :)

* The second biggest difference with Racket that you *must* consider is simply that usually project team members will not come to the project knowing Racket like they "know" Java.  This can be mitigated, and the learning cure might well be "in the noise" if you need and have smart developers on a substantial project.

* Then, as you know, there are advantages of Racket that you *may* consider, which are linguistic, like the productivity and maintainability leaps you might get when a team member makes DSLs that are used by multiple other team members.

* You might also find sometimes that there's a library for Java that does something you need, but you can't find a similar library for Racket.  This situation has improved a lot in the last few years, plus there is now an FFI for easily calling C libraries.  But eventually you will encounter this situation, so be prepared to bang out that bit of functionality (perhaps much better than the Java library does) after first asking on the Racket email list to see if anyone else has such code or knows an easy and good way to do it.  This downside to this situation might be more political than practical, if it's easy for you to get the same functionality, but you'll have to explain to management that the project is still winning with Racket, even if in this case you did not have a library that a Java programmer would've.

Which brings me to my second question: can Scheme (Racket) be used to develop all these different kinds of applications (console, GUI, etc.)? Can anybody point me to real life examples of each type of application developed with Scheme (Racket)?

DrRacket is a good example of a complicated GUI app that is used in production.

Racket ships with a few console apps that are implemented in Racket.  Close to 10 years ago, I had a long-running console app in Racket that was a specialized Web spider, and I also occasionally write utility scripts as console apps.  I think people write console apps frequently, without considering them noteworthy.

The server apps in Racket tend to be non-public, because they are for internal use of an enterprise, or for competitive or other reasons.  I believe that the PLaneT server is implemented in Racket, and it is public.

 

[In fact, I specifically came to Racket after having had a grand vision of being able to develop all the components for a web app (SQL to access the DB, business logic, HTML for presentation, CSS for styling, JS for interactivity, even XML for configuration) using a single unified language (or many small separate DSLs based on a single language) and I have been kind of surprised at not finding this idea fleshed out anywhere as a framework, library, module, etc.]

Yes, I have a client who has credited Racket's utility for SXML processing and dynamic programming with being a strategic advantage.

There is another company down the street from  me here that Google bought, after they used another Lisp dialect to implement what in the past had only been done with mainframes (and to interoperate with the very complicated legacy mainframe network).  They credit the DSL capabilities of Lisps with being crucial to being able to do this at all in a timely manner.  (They also credit using a Lisp with being able to attract a much better talent pool than they would've otherwise.)

 

if Scheme is the be-all and end-all of programming languages,

Before we go further, Racket is Racket, not Scheme.  As soon as we start saying Scheme, people start getting confused. :)

When people start talking about why does industry currently use the technologies it does, people identify and speculate on many reasons.  Some of the reasons concern pragmatism and accidents of history, and sometimes people are very critical about the state of current practice.

I think you'll find with Racket is that it's been a well-kept secret for a decade, up until this last year, around the time when the name was changed and it started to get noticed by more people.  Many of the people who stumbled upon Racket earlier ended up doing industrial-strength app development in Racket, but on apps for use by university students and schoolchildren, which is why I suggest looking at DrRacket as a good example of an app developed in Racket.  I think we'll start seeing more publicly-accessible projects using Racket now.

FWIW, I have a background in mission-critical software engineering and methodology, and, like you, did real development in C, then C++, then Java, and various other languages in there (Perl, Smalltalk, Python, Common Lisp, 4GLs, more obscure ones...), until I gravitated towards Racket.  I decided on Racket for the greater agility and sophistication of the core language, and for the smart developer community.

Since you asked, I don't think anyone has yet invented the be-all and end-all of programming language platforms, but Racket is the current all-around best I can think of for most modern app development purposes that are not externally constrained to use Java.  (There is even a project to adapt it for embedded systems.)  The other platform that came close for me, linguistically, is Haskell, but I think Racket wins for DSLs and a few other reasons.

Another, smaller, advantage over Haskell: I can take an industry programmer who used Pascal once in school, sit them in front of Racket and have them ready to start being productive with other people's libraries by the end of the day, treating Racket as only "a better Pascal, with parentheses, plus some code patterns I've I'm using without fully understanding".  They can then learn idiomatic Racket, how to make their own DSLs, and functional programming precepts incrementally as they work, over the course of months.  But sitting that same programmer in front of Haskell... would be much more difficult.

--
http://www.neilvandyke.org/

[Neil Van Dyke]

Matthias Felleisen wrote at 08/19/2011 06:26 PM:
> Indeed, you will find Java code that looks like Fortran code. I am
> sure when Racket's popularity explodes, you will find Racket code that
> looks like Fortran code, too.

As the old saying goes, "You can write Fortran in any language."

Today, thanks to the Racket platform's meta-language facilities, if we
are feeling cocky, we can suffix that old saying with, "... and you can
write any language in Racket." :)

--
http://www.neilvandyke.org/

[Marco Morazan]

On Fri, Aug 19, 2011 at 6:26 PM, Matthias Felleisen
<matt...@ccs.neu.edu> wrote:
> Indeed, you will find Java code that looks like Fortran code. I am sure when
> Racket's popularity explodes, you will find Racket code that looks like
> Fortran code, too.
> -- Matthias

I trust there is a bug in your crystal ball! What an awful prediction!
So, PLs advance but programmers do not? Not good!

--

Cheers,

Marco

[Ivanyi Peter]

Which brings me to my second question: can Scheme (Racket) be used to develop all these different kinds of applications (console, GUI, etc.)? Can anybody point me to real life examples of each type of application developed>
with Scheme (Racket)?>

Hi,

Just to be proud a little bit. :-)
Here is my project in Scheme which is still going strong after 4-5 years.
(This is basically a console program.)
http://www.hexahedron.hu/personal/peteri/autolisp/simulator/index.html
The company has used it in Europe several thousand times, and the construction of
several hundred halls have been executed using the results of the software.

Best regards,

Peter Ivanyi

[Jos Koot]

This reminds me of a project I once, some 20 or more years ago, did for converting SPSS system files from CDC6400 and CDC205 to SPSS system files to IBM9000. Unfortunately CDC has gone down shortly after. Different word size, different character encoding etc. Typically these system files were stored on magnetic tape, with or without labels. I designed a system for the conversion in natural language both for interpreting the labels and the data (may be interspersed with some pseudo code). The task to implement my design was given to a colleague who was confined to Cobol. The consequence was that I myself had to provide some code for binary reading of magnetic tapes and their labels (6 bit encoding to 8 bit encoding) and for the conversion of alphanumerical and numerical data from CDC to IBM EBCDIC. In the end it appeared to work very well. Many data that otherwise would have been lost, were recovered.

 

This I mention in order to show that good understanding of the required processes is more important than the choice of the language, although cooperation between two or more languages may require additional interfaces.

 

It remains a fact that using a LISP like language, in particular Racket, can reduce the time for development by at least a factor of 10. Four factors are important here:

 

1: The foreign interface of Racket gives full access to all alien formats of data.

 

2: Scheme, especially Racket, provides a very reliable representation of data. Also it provides very high hygiene.

 

3: Most important: Racket allows a very quick implementation of algorithms based on data structure (see HtDP)

 

4: Although PLT Racket does (not yet) claim to include a highly optimizing compiler, nowadays hardware allows me to use DrRacket to make computations that (for reason of time and/or memory) could not be made on supercomputers dating from 20 years ago. Even on the simple laptop I am using to send this email, I can do much more complicated computations than on a machine that 20 years ago was called a supercomputer.

 

Nowadays supercomputers may include thousands of processors, usually each processor having its own memory and interconnected by fast interconnection network. I have hardly seen any machines with many processors sharing one single space of memory. Probably because of racing conditions.

 

I sometimes have dreamt of a multiprocessor machine with one unified memory, possibly with addition of private memories for mutable data. When disallowing mutation on memory shared by multiple processors, there could be a good chance to develop multiprocessor machines running parts of a Racket program in parallel without requiring the user to specify which can and which cannot be done concurrently. For example you can imagine a Racket that computes the arguments of an application in parallel. For a single invocation of a procedure this does do not much, but think of arguments that themselves may call functions of arguments that can be computed concurrently.

 

Kind regards, Jos

 

 

---

From: users-...@racket-lang.org [mailto:users-...@racket-lang.org] On Behalf Of gonzalo diethelm
Sent: sábado, 20 de agosto de 2011 0:08
To: us...@racket-lang.org
Subject: [racket] Racket in the large

[pablo]

On Sat, Aug 20, 2011 at 2:31 AM, Mark Engelberg
<mark.en...@gmail.com> wrote:
> On Fri, Aug 19, 2011 at 3:26 PM, Matthias Felleisen
> <matt...@ccs.neu.edu> wrote:
>> Correction: Scheme (current standard) is the third-best language. Racket is
>> the second-best language.
>
> OK, this is clearly a setup for someone to ask, so I'll ask :)
> What's the first-best language?

I'd say the one you use Racket to create for the task at hand ;-).

--
Pablo.
http://pablo.rauzy.name/

[Gregory Woodhouse]

It seems to me that Racket would be more widely used if there were database drivers for major DBMSs (e.g., PostgreSQL, SQLite, MySQL. maybe even Oracle) as part of the standard distribution. I know there are PLaneT packages available, but it would sure be easier for potential adopters if they came bundled with Racket. Barring that, a consistent API or that could be implemented by drivers for specific systems would be very helpful, and would make Racket an easier "sell", as it were.

On Aug 20, 2011, at 6:49 AM, pablo wrote:

> I'd say the one you use Racket to create for the task at hand ;-).

_________________________________________________

[gonzalo diethelm]

I would like to thank everyone for their thoughtful answers. I will try to summarize a bit and then present my conclusions at the end. One minor note: I did try to clearly differentiate between Scheme and Racket in my original post; I am fully aware they are separate and different languages.

1. There definitely are enterprise-class applications in the wild which were built with Scheme and/or Racket. The following were mentioned: some of Disney World's virtual rides, the control system for some Boeing / USAF telescope arrays, a large enterprise system interacting with PostgreSQL, a simulator that helps the process of pricing hall structures, and a system used to convert data files between two very different hardware architectures.

2. It is possible to build console, GUI and web apps with Racket. There are countless console apps everywhere; DrRacket is a good example of a complicated GUI app; it is likely that the PLaneT server is implemented in Racket (could anyone please confirm this)?

3. Scheme (current standard) is the third-best language there is. Racket is the second-best language. It is not clear which would be the overall best language (this was tongue in cheek, I assume, but I would love to hear the answer).

4. Racket is a rather recent language that is just beginning to take off, and things are looking very bright.

5. There might be a Racket software engineering book soon, planned by Neil Van Dyke. I declare myself as a sure customer.

6. I think there is one thing missing in Racket, and this was also pointed out during the discussion: database drivers for major DBMSs (I would say at least Oracle, DB2 and SQL Server on the commercial front, and PostgreSQL, SQLite and MySQL on the open source front). In addition to that, I think Racket would be greatly enhanced by a single relational data access layer that would hide the differences between specific RDBMSs and facilitate switching from one to another. (Note: I am not trying to dictate anything about Racket, just voicing my opinion).

My conclusion is that I will keep learning and using Racket for my future projects; I fully expect that learning experience to be a thoroughly entertaining personal journey. Heartfelt thanks to everyone who contributed answers to my questions, and to those who have brought Racket to where it stands today. You guys rock.

[Robby Findler]

On Saturday, August 20, 2011, gonzalo diethelm <gdie...@dcv.cl> wrote:
> I would like to thank everyone for their thoughtful answers. I will try to summarize a bit and then present my conclusions at the end. One minor note: I did try to clearly differentiate between Scheme and Racket in my original post; I am fully aware they are separate and different languages.
>
> 1. There definitely are enterprise-class applications in the wild which were built with Scheme and/or Racket. The following were mentioned: some of Disney World's virtual rides, the control system for some Boeing / USAF telescope arrays, a large enterprise system interacting with PostgreSQL, a simulator that helps the process of pricing hall structures, and a system used to convert data files between two very different hardware architectures.
>
> 2. It is possible to build console, GUI and web apps with Racket. There are countless console apps everywhere; DrRacket is a good example of a complicated GUI app; it is likely that the PLaneT server is implemented in Racket (could anyone please confirm this)?

It is.

Robby

[Neil Van Dyke]

Thanks for the summary, this helps us fill in some of the gaps of what
was said before.

gonzalo diethelm wrote at 08/20/2011 11:15 PM:
> 5. There might be a Racket software engineering book soon, planned by Neil Van Dyke. I declare myself as a sure customer.
>

Besides my vaporware book, there are several already existing books for
Racket, included in electronic form for free with Racket. Plus there
are the related books HtDP and PLAI by the same people. And I have
heard of two other tentative new Racket books in the works other than my
own.

> 6. I think there is one thing missing in Racket, and this was also pointed out during the discussion: database drivers for major DBMSs (I would say at least Oracle, DB2 and SQL Server on the commercial front, and PostgreSQL, SQLite and MySQL on the open source front). In addition to that, I think Racket would be greatly enhanced by a single relational data access layer that would hide the differences between specific RDBMSs and facilitate switching from one to another. (Note: I am not trying to dictate anything about Racket, just voicing my opinion).
>

Looks like this question didn't get answered before.

Three generalized interfaces that support multiple RDBMS backends:

http://planet.racket-lang.org/display.ss?package=db.plt&owner=ryanc
http://planet.racket-lang.org/display.ss?package=dbi.plt&owner=bzlib
http://planet.racket-lang.org/display.ss?package=sqlid.plt&owner=oesterholt

There's also these (plus some unreleased ones):

http://planet.racket-lang.org/display.ss?package=snooze.plt&owner=untyped
http://planet.racket-lang.org/display.ss?package=spgsql.plt&owner=schematics
http://planet.racket-lang.org/display.ss?package=mysql.plt&owner=jaz
http://planet.racket-lang.org/display.ss?package=mongodb.plt&owner=jaymccarthy
http://planet.racket-lang.org/display.ss?package=mongodb-native.plt&owner=dabag
http://planet.racket-lang.org/display.ss?package=sqlid-helper.plt&owner=sweeney
http://planet.racket-lang.org/display.ss?package=sql-table.plt&owner=dfisher
http://planet.racket-lang.org/display.ss?package=libpq.plt&owner=synx
http://planet.racket-lang.org/display.ss?package=sqlite.plt&owner=jaymccarthy
http://planet.racket-lang.org/display.ss?package=sql-oo.plt&owner=jaymccarthy

Regarding Oracle and DB2, I suspect that anyone invested in one of those
could use some of the open source Racket interfaces for other RDBMSs as
examples for supporting another RDBMS. The cost might be relatively
minor (considering the large existing investment in the RDBMS).

> My conclusion is that I will keep learning and using Racket for my future projects;

Glad to hear it.

There is a lot of smart activity to going on with Racket, which is not
always well-advertised, so asking questions on the email list is often
helpful.

--
http://www.neilvandyke.org/

[key...@gmx.de]

Hi Neil, hi all,

>>
>
> Regarding Oracle and DB2, I suspect that anyone invested in one of those
> could use some of the open source Racket interfaces for other RDBMSs as
> examples for supporting another RDBMS. The cost might be relatively
> minor (considering the large existing investment in the RDBMS).

For my private use (and with the intention to put it on Planet some time if I find the courage :-;), I've started working on bindings to the open source OCILIB library, a widely used, very convenient and time-saving wrapper around Oracle's own OCI.
Although at the moment it would be version 0.0 ... 01 or such, it will be "work in progress" and evolve not for the next months, but for the next years for sure, as for me it will be the enduring link between job (having turned Oracle DBA since spring last year) and spare time interest (Racket).

>
>
>> 6. I think there is one thing missing in Racket, and this was also pointed out during the discussion: database drivers for major DBMSs (I would say at least Oracle, DB2 and SQL Server on the commercial front, and PostgreSQL, SQLite and MySQL on the open source front). In addition to that, I think Racket would be greatly enhanced by a single relational data access layer that would hide the differences between specific RDBMSs and facilitate switching from one to another. (Note: I am not trying to dictate anything about Racket, just voicing my opinion).
>

Perhaps I can add a remark to this "from the other" (the database admins') side :-; ? From the point of view of an Oracle DBA (I'm emphasizing "Oracle" here as I don't want to generalize, having no comparable experience with other rdbms's), applications written with a goal of "database independence" can be a big problem and lead to grave performance (or even, data integrity) issues. This is because even though the sql may be the same, important concepts like read consistency etc. are implemented in a very different ways in different rdbms'.
So anyway, when an application changes to use another rdbms, major work might be required ...

Ciao,
Sigrid

[Ryan Culpepper]

On 08/20/2011 09:15 PM, gonzalo diethelm wrote:
> [...]

>
> 6. I think there is one thing missing in Racket, and this was also
> pointed out during the discussion: database drivers for major DBMSs
> (I would say at least Oracle, DB2 and SQL Server on the commercial
> front, and PostgreSQL, SQLite and MySQL on the open source front).

My database package (ryanc/db) works for PostgreSQL, MySQL, SQLite
directly and Oracle and DB2 via ODBC. It should work with SQL Server
also, but I haven't tested that configuration.

Docs are here:
http://planet.racket-lang.org/package-source/ryanc/db.plt/1/4/planet-docs/db/index.html

> In addition to that, I think Racket would be greatly enhanced by a
> single relational data access layer that would hide the differences
> between specific RDBMSs and facilitate switching from one to another.
> (Note: I am not trying to dictate anything about Racket, just voicing
> my opinion).

There are many things you might mean by that (ORMs,
relational-algebra-to-SQL compilers, etc). The concerns that Sigrid
raised make me think that hiding *all* the differences is a hopeless
task, but maybe something with a narrower focus could be useful. Are
there any existing frameworks that accomplish what you have in mind?

Ryan

[gonzalo diethelm]

> > 6. I think there is one thing missing in Racket, and this was also
> > pointed out during the discussion: database drivers for major DBMSs (I
> > would say at least Oracle, DB2 and SQL Server on the commercial front,
> > and PostgreSQL, SQLite and MySQL on the open source front).
>
> My database package (ryanc/db) works for PostgreSQL, MySQL, SQLite
> directly and Oracle and DB2 via ODBC. It should work with SQL Server also,
> but I haven't tested that configuration.
>
> Docs are here:
> http://planet.racket-lang.org/package-source/ryanc/db.plt/1/4/planet-
> docs/db/index.html

Great information, thanks. I take it this package works natively with PostgreSQL, SQLite and MySQL. Is it possible and "cheap" to make it work natively with other RDBMSs, such as Oracle?

> > In addition to that, I think Racket would be greatly enhanced by a
> > single relational data access layer that would hide the differences
> > between specific RDBMSs and facilitate switching from one to another.
> > (Note: I am not trying to dictate anything about Racket, just voicing
> > my opinion).
>
> There are many things you might mean by that (ORMs, relational-algebra-to-
> SQL compilers, etc). The concerns that Sigrid raised make me think that hiding
> *all* the differences is a hopeless task, but maybe something with a
> narrower focus could be useful. Are there any existing frameworks that
> accomplish what you have in mind?

You are right, my requirement is ambiguous. This is what I have in mind:

1. A set of abstractions (probably written in Racket) that allow you to interact with any RDBMS in a uniform way: select records, join tables, insert / update / delete data, etc. Hopefully you would have one or more DSLs to specify what you want to do, and not have to use strings (as in JDBC) for your commands. This should apply to all RDBMSs.
2. A generic library (probably written in C / C++) that provides all the low-level trappings for #1.
3. A set of drivers (written in C / C++), one for each RDBMS, that #2 uses when configured to talk to that particular RDBMS.

So I would love to be able to write something like this in a Racket-y style:

(select-from '((tab-A A) (tab-B B)) '(A.col-1 A.col-2 B.col-3) '(some magical way to specify conditions and joins...))

And this would work for, say, Oracle, if somebody takes the time to develop #3 above for the Oracle case. Bonus points if the solution uses prepared statements (for reasons of security and efficiency).

> Ryan

--
Gonzalo Diethelm

[Ryan Culpepper]

On 08/24/2011 12:18 PM, gonzalo diethelm wrote:
>>> 6. I think there is one thing missing in Racket, and this was also
>>> pointed out during the discussion: database drivers for major DBMSs (I
>>> would say at least Oracle, DB2 and SQL Server on the commercial front,
>>> and PostgreSQL, SQLite and MySQL on the open source front).
>>
>> My database package (ryanc/db) works for PostgreSQL, MySQL, SQLite
>> directly and Oracle and DB2 via ODBC. It should work with SQL Server also,
>> but I haven't tested that configuration.
>>
>> Docs are here:
>> http://planet.racket-lang.org/package-source/ryanc/db.plt/1/4/planet-
>> docs/db/index.html
>
> Great information, thanks. I take it this package works natively with
> PostgreSQL, SQLite and MySQL. Is it possible and "cheap" to make it
> work natively with other RDBMSs, such as Oracle?

It's certainly possible. I haven't looked at the Oracle C API(s), but I
would estimate it would take between one and two thousand lines of
Racket code. ODBC support, for comparison, currently takes about 1500
lines. More features, like interval types and asynchronous execution,
will add to that.

What benefits would "native" Oracle support provide (compared to ODBC)
to justify the cost?

>> > In addition to that, I think Racket would be greatly enhanced by a
>>> single relational data access layer that would hide the differences
>>> between specific RDBMSs and facilitate switching from one to another.
>>> (Note: I am not trying to dictate anything about Racket, just voicing
>>> my opinion).
>>
>> There are many things you might mean by that (ORMs, relational-algebra-to-
>> SQL compilers, etc). The concerns that Sigrid raised make me think that hiding
>> *all* the differences is a hopeless task, but maybe something with a
>> narrower focus could be useful. Are there any existing frameworks that
>> accomplish what you have in mind?
>
> You are right, my requirement is ambiguous. This is what I have in mind:
>
> 1. A set of abstractions (probably written in Racket) that allow you
> to interact with any RDBMS in a uniform way: select records, join
> tables, insert / update / delete data, etc. Hopefully you would have
> one or more DSLs to specify what you want to do, and not have to use
> strings (as in JDBC) for your commands. This should apply to all
> RDBMSs.

It sounds like you want some kind of abstract SQL syntax. I'll probably
try to do something like that.

> 2. A generic library (probably written in C / C++) that provides all
> the low-level trappings for #1.
> 3. A set of drivers (written in C / C++), one for each RDBMS, that #2
> uses when configured to talk to that particular RDBMS.

Why would I write C code when I could write Racket code? These are
currently done using Racket and, when necessary, FFI bindings.

Ryan

[Neil Van Dyke]

Ryan Culpepper wrote at 08/24/2011 03:44 PM:
> It sounds like you want some kind of abstract SQL syntax. I'll
> probably try to do something like that.

Would be good if someone does the end-all-be-all of sexp SQL syntax. If
someone else doesn't do it, eventually I will try.

Sexp SQL done at least 3 times for Racket that I know of. Two of those
are open source. One is in PLaneT, but I haven't looked at it closely.
http://planet.plt-scheme.org/package-source/soegaard/sqlite.plt/1/2/doc.txt

--
http://www.neilvandyke.org/

[Grant Rettke]

Sounds like you are talking about the Racket version of Hibernate?

--
http://www.wisdomandwonder.com/
ACM, AMA, COG, IEEE

[gonzalo diethelm]

> It's certainly possible. I haven't looked at the Oracle C API(s), but I would
> estimate it would take between one and two thousand lines of Racket code.
> ODBC support, for comparison, currently takes about 1500 lines. More
> features, like interval types and asynchronous execution, will add to that.
>
> What benefits would "native" Oracle support provide (compared to ODBC) to
> justify the cost?

Better speed (at least in theory; benchmarks would be required).

> It sounds like you want some kind of abstract SQL syntax. I'll probably try to
> do something like that.

Exactly.

> > 2. A generic library (probably written in C / C++) that provides all
> > the low-level trappings for #1.
> > 3. A set of drivers (written in C / C++), one for each RDBMS, that #2
> > uses when configured to talk to that particular RDBMS.
>
> Why would I write C code when I could write Racket code? These are
> currently done using Racket and, when necessary, FFI bindings.

I am a noob regarding FFI, so I don't know what must be done in C and what can be done in Racket. C is the usual way to go for this kind of thing in other languages.

> Ryan

--
Gonzalo Diethelm

[gonzalo diethelm]

> > It sounds like you want some kind of abstract SQL syntax. I'll
> > probably try to do something like that.
>
> Would be good if someone does the end-all-be-all of sexp SQL syntax. If
> someone else doesn't do it, eventually I will try.

Yes, that would be great!

> Sexp SQL done at least 3 times for Racket that I know of. Two of those are
> open source. One is in PLaneT, but I haven't looked at it closely.
> http://planet.plt-scheme.org/package-
> source/soegaard/sqlite.plt/1/2/doc.txt

Maybe this could be a starting point?

--
Gonzalo Diethelm

[Robby Findler]

On Wed, Aug 24, 2011 at 7:26 PM, gonzalo diethelm <gdie...@dcv.cl> wrote:
>> Why would I write C code when I could write Racket code? These are
>> currently done using Racket and, when necessary, FFI bindings.
>
> I am a noob regarding FFI, so I don't know what must be done in C and what can be done in Racket. C is the usual way to go for this kind of thing in other languages.
>

Racket's FFI is very cool. You can really do a lot in Racket proper
because of the way it works.

One big, fairly recent example:

http://blog.racket-lang.org/2010/12/racket-version-5.html

Robby

[gonzalo diethelm]

> Sounds like you are talking about the Racket version of Hibernate?

Not really (at least not I). What I want is pure direct access to data in tables (no ORM), but using a natural DSL that applies to all RDBMSs.

I am thinking about something more in the lines of what is shown here: http://java.dzone.com/announcements/simple-and-intuitive-approach

I pine for a DSL that applies to all RDBMSs (even if I have to modify my queries when switching back-ends), that does not force me to write SQL in a string and that gives me access to the DATA in my TABLES (that is, I don't care about hiding the fact that there ARE tables and columns underneath).

Best regards.

--
Gonzalo Diethelm
DCV Chile