Haskell as a туурa language

[Rotsor]

This post is a branch of "Which language?" discussion, intended to
discuss the practical issues of using Haskell for development of
Tuura.

I've implemented a sample project containing a single dynamic page and
a form modifying a persistent state using Happstack Server, Happstack
State, Blaze Html and formlets libraries:
http://code.google.com/r/rotsor-tuura-hs/

Ulan and Andrey agreed that
> At first glance the code looks terrifying. :)
Do you mean conceptually, or visually?
Conceptually, I can't think of a simpler way of describing the state
schema than this, and formlets too seem a great simplifying
abstraction to me. Do you disagree, or is it just a visual aspect of
the code you don't like (naming / coding style / lack of comments)?

> MVC separation
Oh no, not this word again... I've never understood what MVC really
means. We could probably call the TuuraDb together with operations on
it a model, but I just fail to see a separable view/controller pair
here.

> How difficult is it to migrate from one schema to another?
It seems to be quite straightforward.
I followed this tutorial: http://www.kuliniewicz.org/blog/archives/2009/04/18/data-migration-in-happstackstate/
And the steps for my database conversion (appending the reference
solution language name to the problem name :)) were the following:
1. Rename ProblemInfo to ProblemInfo_0 representing a deprecated
version (Ideally it should be moved into a different module instead of
renaming)
2. Define a new ProblemInfo type
3. Write the following Version and Migrate instances:

instance Version ProblemInfo where
mode = extension 1 (Proxy :: Proxy ProblemInfo_0)
instance Migrate ProblemInfo_0 ProblemInfo where
migrate (ProblemInfo_0 name statement testValidator
referenceSolution)
= ProblemInfo newName statement testValidator
referenceSolution
where newName = name ++ " (solution in " ++ show
(programLanguage referenceSolution) ++ ")"

That's it.

> We should make all examples with some kind of session (login/logout).
Yes, session handling and authentication are instrumental, and are the
next thing I'll try to look into.

[Ivan]

I still don't understand what are the advantages of using Haskell for
mundane things such as database interaction and html generation.

I don't mean to be a Haskell hater or anything, I just don't see the
point. The capabilities of Happstack are very likely similar to any
other web framework, and the Haskell code that is algorithmically
simple but deals almost exclusively with the I/O just looks messy.

Like this:

summaryPage :: ServerPart Response
summaryPage =
do methodM GET
db <- query ReadTuuraDb
return $ toResponse (tuuraHtml db)

tuuraHtml :: TuuraDb -> Html
tuuraHtml db = html $ do
head $ title "Tuura page!"
body $ do
a ! A.href "/tuura/addProblem" $ "Add a new problem"
p "Available problems:"
mapM_ problemHtml (Map.elems (problems db))



All you do is use the "do" notation to write an imperative program.
Why do that in Haskell if you could just use an imperative language?

[Ulan]

> Do you mean conceptually, or visually?

> Oh no, not this word again... I've never understood what MVC really
> means.

The database is a model.

We can say that "Pages" contains views. I think if we minimize
dependency of a view on the rest of the system the code would be
easier to understand. Ideally, a view would be a function that takes
simple arguments (like list of problems, not the whole db) and
constructs an html page. Now a person who wants to edit the html don't
have to understand the details of the db.

The function "summaryPage" is a controller, it interacts with db and
prepares input for the view. We can extract all this gory details from
the "Pages" and put in into "Routes" or "Controllers".

On Mar 19, 5:18 pm, Rotsor <rot...@gmail.com> wrote:
> This post is a branch of "Which language?" discussion, intended to
> discuss the practical issues of using Haskell for development of
> Tuura.
>
> I've implemented a sample project containing a single dynamic page and
> a form modifying a persistent state using Happstack Server, Happstack
> State, Blaze Html and formlets libraries:http://code.google.com/r/rotsor-tuura-hs/
>
> Ulan and Andrey agreed that> At first glance the code looks terrifying. :)
>
> Do you mean conceptually, or visually?
> Conceptually, I can't think of a simpler way of describing the state
> schema than this, and formlets too seem a great simplifying
> abstraction to me. Do you disagree, or is it just a visual aspect of
> the code you don't like (naming / coding style / lack of comments)?
>
> > MVC separation
>
> Oh no, not this word again... I've never understood what MVC really
> means. We could probably call the TuuraDb together with operations on
> it a model, but I just fail to see a separable view/controller pair
> here.
>
> > How difficult is it to migrate from one schema to  another?
>
> It seems to be quite straightforward.

> I followed this tutorial:http://www.kuliniewicz.org/blog/archives/2009/04/18/data-migration-in...

[Arseniy Alekseyev]

> I still don't understand what are the advantages of using Haskell for
> mundane things such as database interaction and html generation.

The advantages of Haskell over C#/Java are the same whatever task you
are trying to solve: type-safety, type inference, lazy evaluation,
easier code abstraction and re-use.

From the example you provided, it can quite easily be shown that the
tuuraHtml function is free from side effects (it takes a pure value of
the database state and produces a pure Html). This is one advantage of
this Haskell code over equivalent Java/C#.

Another advantage is that the Html is produced lazily, i.e. the
functions are only evaluated when needed. This makes it very different
from an imperative implementation. It may allow to reduce the
roundtrip latency by starting the transmission earlier or even to
produce an otherwise impossible response of unbounded size.

Third advantage is the Haskell's terse notation. A semantically close
(mostly functional except for "for" loop) function in Java could look
like:

Html tuuraHtml(TuuraDb db) {
ArrayList<Html> problemTableLines = new ArrayList<Html>();
for(ProblemInfo problem : db.problems().values())
problemTableLines.add(problemHtml(problem));
return html(sequence(
head(title("Tuura page!")),
body(sequence(
withAttribute(a(),
A.href("/tuura/addProblem")).enclose("Add a new problem"),
p("Available problems:"),
sequenceList(problemTableLines)
)
));
}

Which is much uglier.

I doubt it will look much nicer if you fully convert it to imperative
style. And even if it will, such a conversion will probably reduce
(de-)composability of your code.

And, if you mention the "do" notation, it does not do anything
"monady" in this case because it is only used for sequencing the HTML
elements, not some clever computations. Monoid instance for Html would
be enough, but the code laid out with "do" notation just looks nicer.

Oh, no, I am helping to start a holywar... :(

[Ulan Degenbaev]

> Why do that in Haskell if you could just use an imperative language?

"having computations as first-class values makes Haskell the ultimate imperative language :-)" (c) haskell.org

[Arseniy]

I have discussed some code comprehension issues with Andrey and he said that the good code should be like a good talk or a good paper -- that is easy to follow.

That reminded me of a Haskell feature called literate programming, inspired by the Knuth's WEB system. 

So, I've taken a single function -- the HTML table maker, and turned it into a paper. :)

Check it out.

Source code (a valid haskell module): http://code.google.com/r/rotsor-tuura-hs/source/browse/src/Pages/Tables.lhs

The same, rendered into HTML: http://www.honey-net.eu:8000/Pages.Tables.html

The same, TeX-processed into PDF: http://www.honey-net.eu:8000/Pages.Tables.pdf

Not sure if using this is really practical, but it was definitely interesting thing to try. :)

[Andrey Mokhov]

Wow, for me that's miles better for understanding than previous
versions %) Perhaps I would not be able to rewrite the program from
scratch, but at least I seem to get what each code line means.

Surely this may not be practical, but as a marketing it worked: I see
that Haskell is not that ugly by itself, but it can be made ugly when
writing everything in a single sentence. If we agree to follow some
reasonable rules on readability I should be able to do something
useful in Haskell too.

As of now, there are no examples of other implementations. It seems
that Ulan dropped python/ruby approaches in favour of HappStack %)
Examples from Ivan and Splash are not here yet.

By the way, what do you think of the following C++ equivalent? What
are its disadvantages to the Haskell code?

Html renderTable(const vector<Html> &headers, const
vector<vector<Data>> &data, const vector<cellRenderers> &renderers)
{
Html result;
int n = headers.size();
int m = data.size();

for(int i = 0; i < n; i++) result += th(header[i]);

for(int i = 0; i < m; i++)
{
Html row;
for(int j = 0; j < n; j++) row += td(renderer[i](data[i][j]));
result += tr(row);
}
return result;
}

[Arseniy]

> By the way, what do you think of the following C++ equivalent? What
> are its disadvantages to the Haskell code?

Well, this one is easy!

First of all, separating semantically-linked headers and renderers
into separate parameters made this code unsafe! In case of the length
mismatch it just crashes. The same story goes with
vector<vector<...>>.

Secondly, it's unclear why you do have renderers at all if you already
demand a 2d-matrix on input. Why not require a matrix of Html then?

Next, you omit the template-syntax for "Data" type parameter. Is your
function intentionally non-polymorphic? If so, then there are further
problems, which I will only be able to address after the preceding
problem has been taken care of.

Those were the concrete correctness/interface issues. Here goes the
abstract "good code" bullshit:

First, the language ones:
* No laziness
* No purity
* No GC

Next, the implementation ones:
* Error-prone with all those explicit loop counters:
- loop counters are accessible for modification throughout the body
- no obvious, not to say compiler-checked, correspondence between
the iteration range and the vector index range, making invalid index
errors easy to make
* I would say it has bad modularity, but the example is so small that
it does not really matter.
* Arguably more difficult to read than the Haskell solution.
* The important missing definition of a "cellRenderers" (probably
meant as CellRenderer?) abstract class would probably take much more
comprehensive resources of the reader than the Haskell's (a -> Html)

[Andrey Mokhov]

> First of all, separating semantically-linked headers and renderers
> into separate parameters made this code unsafe! In case of the length
> mismatch it just crashes. The same story goes with
> vector<vector<...>>.

True. However, I think that obsessing with code safety at the expense
of readability and simplicity is not always justified. I mean there is
no need for perfectionism in such small project as this one. I've seen
many projects die because their authors put too much effort on perfect
design rather than on implementing simple but *working* prototypes.

> First, the language ones:
> * No laziness
> * No purity
> * No GC

Again, why would any of these be important for rendering HTML? Just
for the sake of being perfect?

> Next, the implementation ones:
> * Error-prone with all those explicit loop counters:
>   - loop counters are accessible for modification throughout the body
>   - no obvious, not to say compiler-checked, correspondence between
> the iteration range and the vector index range, making invalid index
> errors easy to make

Same goes here. Somehow programmers got used to deal with loops and
iterations. And created things like StarCraft II %) I'm not sure they
would have created it had they started obsessing with their code being
perfect and safest. And do you have an example of a real-life
application with a perfect implementation in Haskell? Please, don't
say Darcs :)

I agree that my interface is simplistic and may not be scalable enough
for all our rendering purposes. I didn't think it through as my point
was to compare Haskell's list-oriented approach with the standard
array-iteration one. And for me the latter is simpler. Perhaps it is
not simpler for everyone, but simpler for most of us. And as long as a
program works, who cares how perfectly it is implemented?

[Arseniy]

May I summarise the discussion so far?
- How is A worse than B?
- A is worse than B in (x,y,z,v,w) and is simply broken!
- x,y,z,v,w are not really important so long as A is not broken and
the SC2 space ships travel through the universe, ergo A is better than
B

I find it difficult to continue discussion after such an argument. :D
But I will.
However "unimportant" all those issues are, I will elaborate on the
points you questioned.

> why laziness
As stated earlier, for example to avoid keeping the resulting page in
memory and to be able to start HTML transmission immediately.

> why purity
To be able to understand the nature of function behaviour looking only
at its type signature.

> why GC
To be able to re-use different parts of HTML in different places and
not care about memory management. Smart pointers usually help here all
right, but there are corner cases when they don't.

> haskell has bad programs
Of course it does! Because using Haskell you can

* translate a bad imperative program line-by-line without changing
anything
* perform I/O and foreign function calls
* have a non-terminating recursion
* have a memory overflow
* have a pattern-match failure

However, most of those (except maybe a memory overflow) are much
easier to detect in design-time in Haskell than in any mainstream
imperative programming language.

> as long as a program works, who cares how perfectly it is implemented

Well, the Haskell program does, the C++ one does not. Which is better
now?

[Andrey Mokhov]

Well, I never said your Haskell implementation of renderTable is wrong
and my implementation in C++ is right, did I? :)

The point is that for this project to be successful we do not need a
perfect design. Really, we don't! But we need the code to be simple
enough so that many people can contribute and maintain the code, not
just you and Ulan.

Your final implementation of renderTable is close to what I'd like to
see in the end but will you be able to write everything in this style?
Isn't it similar to "translation of a bad imperative program line-by-
line without changing anything"?

I myself find Haskell fun. And I'd like to learn it. I just don't want
to scare away any potential contributors by perfectionism which is
difficult to comprehend.

[Arseniy Alekseyev]

> Well, I never said your Haskell implementation of renderTable is wrong
> and my implementation in C++ is right, did I? :)

Okay, you did not, however you asked for comparison. :)

> Your final implementation of renderTable is close to what I'd like to
> see in the end but will you be able to write everything in this style?

Well, there is always a trade-off between the effort and the quality
of product (code clarity in this case). Who knows where the optimum
is. The renderTable-style commenting is almost certainly too much
though. It's kind of similar to the comments like
for(int i=0;i<n;i++) // iterate over the [0..n) interval using i as
the loop counter

> Isn't it similar to "translation of a bad imperative program line-by-
> line without changing anything"?

I find no similarity at all. No assignments here, no implicit
state-carrying with the help of monads. (Okay, formally there is a
Html monad, but it is not really used as a monad here)

> I just don't want
> to scare away any potential contributors by perfectionism which is
> difficult to comprehend.

Haskell will scare some away, no matter if the code is comprehensible
or not, for Haskell has a smaller user base and no advanced IDE I
know of. However, I find programming in Haskell much more fun and
rewarding (e.g. "it compiles -> it probably works" relation is a very
pleasant one), so the remaining unscared-by-Haskell contributors
should be able to do more than they'd do in other language. :)

Maybe a difficult trade-off decision for project, but an easy decision for me.

[Andrey Mokhov]

> Maybe a difficult trade-off decision for project, but an easy decision for me.

Perhaps, in our case the decision is not that crucial. The front-end
is going to be quite simple: a couple of forms, several tables. I'm
not even sure we need Ajax... maybe just for showing status of
submissions. All this can be easily managed by several people as long
as they have fun doing that :-)

> I find no similarity at all. No assignments here, no implicit
> state-carrying with the help of monads.

By the way, I hope all intermediate variables that are created to
untangle a single Haskell statement are optimised away by the
compiler? So there is no runtime overhead due to this step-by-step
style?

[Arseniy]

> By the way, I hope all intermediate variables that are created to
> untangle a single Haskell statement are optimised away by the
> compiler?

They are.
AFAIK GHC inlines every expression that is used no more than once.

[Арсений Алексеев]

For some reason people have requested some HTML-templating for Haskell.

I know of three HTML templating options for Haskell: Heist, Hamlet and HSP.

Heist seems to be a port of Scala Lift's template engine, inheriting
both its strengths and weaknesses.

Hamlet does a compile-time code generation with Template Haskell from
the HTML-like templates with full type-checking. It can even
type-check CSS classes and URLs, which is impressive! However, the
language it uses is not exactly HTML (see examples in
http://docs.yesodweb.com/book/templates), which Ivan found terrible,
but I find rather OK.

HSP does a template pre-processing with a separate tool before feeding
it to the Haskell compiler, in a way similar to ASP.NET. You can see
examples in a paper at
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.21.9537&rep=rep1&type=pdf
The HTML used is more well-formed than the Hamlet's one, but still is
not a valid XML in constrast to Heist approach.

[Andrey Mokhov]

I've looked through Hamlet and HSP examples and didn't like them.
Hamlet is a new language altogether. HSP, while being closer to HTML,
requires some sort of explicit coding. Perhaps, Heist is the way to
go, although I'd prefer to look at an example before jumping to
conclusions.

[Arseniy]

I have been implementing a "Submit solution" form and ran into the
following problem.

As has been said, I don't use an external database server for the sake
of simplicity. Happstack State provides its in-memory database with
transaction logging and claims high robustness. However, as it turns
out, robustness is not the only good thing about relational databases.
The other important features are table keys, constraints and indexes.
The specific feature I need now is foreign key constraint. Indexes
will certainly be needed later. So, the natural question arises -- are
there good equivalents for the said relational DB features applicable
to the data in memory?

For now, I want the existence of a submission for non-existing problem
to be impossible. The straightforward way to do it now is to declare a
database access function, like "insertSubmission :: SubmissionInfo ->
Update TuuraDb", and check the condition there. However, this has
several disadvantages:
* Does not help writing the future "updateSubmission" without the fear
of database corruption
* Does not help to avoid broken links in case of a problem deletion
* Does not help to find a list of submissions for the given problem
(no usual index associated with the foreign key)
Furthermore, as the foreign key constraint is a crucial database
integrity property, it is only natural to put this information into
the database schema definition rather than into the accessor
functions.

So, the question is -- how to define and enforce foreign keys for the
in-memory data? Are there any libraries for that?

Or should we store the data differently, in a way that the key
violation would be structurally impossible? Like, store a list of
submissions inside the ProblemInfo itself. This may help sometimes,
but I don't think this will work in general because tables with
multiple foreign key columns usually do exist (imagine an additional
"AuthorId" field for SubmissionInfo).

I'm afraid that If we don't find any convenient way to create foreign
keys and indexes in memory, the external database solution (with some
kind of ORM) may even become the simpler one. :)