Database access and File I/O in Elm

[Shashi Gowda]

Hello list!

I am trying to make it possible to run compiled Elm code on top of nodejs as the server for my application. I am going to hack my own fork of Elm to do this. I want to implement database access and file I/O on Elm so that there can be any real use for Elm on the server.

What are some good ways you can think of to do this? What should code doing database / file I/O look like? Has there been any previous work similar to this in Elm? Any recommendation as to what database backend would play well here? Is it reasonable to make something similar to Haskell's IO monad? (I am not a Haskell jedi or anything, I only even have a simple idea of what a monad is. Links where technicalities are deep would be really nice :) )

The direction I want to go with this is make it possible for database records / query results to be used in the code as signals. An update to a certain record in the database should reflect as a change in the value of the signal. I feel this needs some careful thought. I am looking to do some high-level brainstorming about this here.

Thanks a million,

Shashi

[Alex Shroyer]

Database signals would be very cool.  The closest thing I've seen to this so far is in the Elm and Firebase thread.

[Jeff Smits]

Hi Shashi,

That's quite an ambitious project! Are you sure you realise just how much work this might be?
Of course it completely depends on how nice and reusable you want to make your database access and file I/O.

First of all, the IO monad is really a Haskell thing. In Elm all side-effects should be expressible using Signals. However, Elm is young and we don't have that much experience with building nice, Signal-based APIs yet. The best API (IMHO) is the output-only part of the Graphics in Elm.
Now what you want to do is a little more complex, because it involves both input and output. There are currently two ways being tried in Elm, both of them not ideal (again, my opinion):
One can be seen in the HTTP library. It's a simple transformation function from a signal with requests to a signal with responses. This is a simple enough way to interact with any kind of system outside Elm, perhaps there are ways to extend it.
The other can be fond in the Graphics.Input library. Notice that the problem with this part of Elm is that some functions are not completely pure (there are side-effects). The reason we still use it like this is that it's nicer in use and allows for a dynamic amount of inputs.

Have a look at those APIs and read up on the discussions about it on the mailing-list, perhaps it will inspire. You can also look into how other FRP frameworks use file I/O and the like. Though some of them can be tough to read if you're not used to it...

Anyway, ask questions whenever you like, and please let us know how you fare. I think if your project is successful there is a lot we can learn about FRP APIs for imperatively designed systems.

[John Mayer (gmail)]

Right, so two things:

First - elm records and database records. There is a project called MetaHDBC which allows you to do compile-time typed queries; in this way one can have some guarantee that DB results will un-marshall into native types. It uses template Haskell.

Second - finding the right general pattern for request-response promises and bidirectional async streams. Currently, Elm Http leaves something to be desired. However I recently learned of research out of Yale on FRP resource "wormholes" which are similar to an old idea on these lists:

openDB : DSN -> { results : Signal [[DBType]], queries : Signal String -> () }

I'm less averse to trying this than I used to be ;-)

Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "Elm Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Max Goldstein]

As has been alluded to, no one really likes the HTTP or input libraries. Part of what makes HTTP so weird (I have little experience with the input API) is that you have the Waiting value constructor. You need to thread Maybes, Eithers, or potentially empty lists through your program to handle the lack of data, but this only happens for a fraction of a second before the data loads. Then the signal is no longer time varying. That said, a (currently hypothetical) API for handling PUT requests over time fits the signal model nicely. At the moment, I'm pretty sure HTTP input is responses only, not requests.

[Shashi Gowda]

Hello again! :)

@Jeff: Yeah, I do realize this project can be quite hard. I have a
full semester to finish it. I am doing it as a course project. Looking
at Http API helped a little bit! But not much! Please read on :)

@John Thanks! I have tried to finish reading the Wormholes paper, I
keep having to stop reading to understand things having to do with
type classes, resource types, lack of any experience with Haskell is
not helping at all. From what I could understand, I really like the
idea of Wormholes and would like to implement an analogue if possible,
see my API and let me know your thoughts!

@Max: see my question about mutate function below!

I have been working on compiling Elm to Meteor, right now I have a
crude hack that works for toy projects. Using Meteor, I have created a
simplistic Database API, check it out here:
https://github.com/shashi/Elm/blob/elm-on-meteor/libraries/Database.elm

I made a simple example (Stamps, but multi-user)
http://stamptogether.meteor.com/
The code for the app:
https://github.com/shashi/Elm/blob/elm-on-meteor/stamp/main.elm (It's
29 lines!!!)

I would really love comments on the API!

Consider this alternative implementation of mutate:

mutate : Collection a -> Signal (Mutation a) -> Signal (Response a)

I really liked this at first. But there are issues with this. If
mutate were a function which takes a signal of mutations and returns a
signal of asynchronous responses (responses may not be in the same
order as the mutations), it would be pretty useless (because how are
you going to use it to do error handling?)! This is similar to
Http.send, but not quite. In Http.send you only care about the most
latest request (for example, as in Flickr example). But in my case
what I want is an acknowledgement for every mutation. But these may
not be received in the same order as the Mutation. The way JavaScript
in Database.Native deals with this right now is it just returns what
the non-blocking call to Meteor.Collection.update, but the right way
to do this is to pass an callback argument. See
http://docs.meteor.com/#update

I think I like imagining my data sources as a source from which
Database-dependent signals can be set up (using select) and a sink to
which mutations can be thrown (using mutate).

Evan likes DataLog type API, which makes a lot of sense in a
functional programming context. I have a few ideas on how to do this,
but meteor for now provides only MongoDB interaction, which should in
theory allow me to do something like DataLog, but I don't want to do
it if it's going to be tied down to MongoDB. Some sort of a thin layer
which can use any database as a backend would be cool. Will write my
ideas to the group soon.

Lastly, I am looking to allow programs to separate out client code and
server code. In Meteor, anything within an if (Meteor.isServer) {} is
private to the server. Meteor.isServer is the substrate with which
this will need to be implemented, but what is the best way to expose
the means of this separation in the language? You cannot do the same
in Elm since this is quintessentially stateful. You can probably allow
some way to annotate functions and modules as server-bound. What about
having a serverMain variable that is similar to main but runs inside a
Meteor.isServer? Although I have used Meteor on a few toy projects, I
haven't had to write a lot of server-bound code in them.

I have been wanting to write this for a long time, but kept wanting to
improve the API, but since recently there has been some interest in
doing this sort of thing, I decided to just put out what I have.

Thanks a lot!
Regards
Shashi