What do you hate about Yesod?
Michael Snoyman
If anyone's been following Github activity, you might be aware that
we're well on our way towards the 0.10 release. At this point, I'd
like to ask everyone: what drives you crazy about Yesod? Now is the
absolute best chance to address problems, and we have plenty of time
to do it. So let's get it all out there. I'll start with some things
that are already being fixed:
* GGHandler. I don't like it, and I think with the move to conduits,
it will go away.
* Awkward structuring of streaming responses. Also addressed by conduits.
* The current routing code is a mess. I have much cleaner, and *far*
more efficient code waiting for the finishing touches, and I think
I'll push to get that into 0.10.
* Likewise, the Persistent entity code is currently a mess, but the
beta cleans it up a lot.
* Error messages. I think we can make these much shorter if we get rid
of GGHandler and introduce a few newtypes.
Now some things I'd like some input on:
* I don't really like the name GHandler (or GWidget for that matter).
Any ideas on a better name?
* I'm guessing there are a fair number of inconsistent APIs. Can
anyone point out things that are confusing. (Example: the previously
discussed cookie/text/bytestring issue.)
Be brutal, please :-).
One other thing: I'm going to ask for a *lot* of user testing on this
next release. There aren't actually that many changes to the code,
which is surprising given the fact that we're completely excising
enumerator. But let's make 0.10 solid enough that it just rolls into
1.0.
Michael
Justin Greene
The biggest pain points for me have always been dealing with persistent. Here are my complaints about it:
1. Lack of projections makes complex queries almost impossible. Along with this, persistent isn't very composable. This really adds friction to any project that needs these types of queries (I've never run into a project that doesn't need these types of queries). It is also impossible to add parameters to queries that do not return in the result set of a query.
2. Persistent has memory leak issues (or yesod does). I haven't had time to write up a small example on this. Attempting to do a long action (reading a large csv file and inputting it into the database for example) in a yesod handler with persistent will continuously take more memory. This issue went away when I moved the same code into it's own executable. This isn't really a use case I'd normally attempt to support, but it may point to other problems.
3. Migrations fail to create entities with recursive foreign keys.
4. Last I checked the "text" column type in postgres is not recognized by persistent. This only comes up when dealing with heroku's postgres instances (at least for me).
3 of these issues are really fairly minor and I've been able to work around them. Fixing the first one would basically require rewriting persistent from scratch, which I'm doubting people are willing to do. I've considered moving my code to haskellDB, but haven't found the time (I'm looking at releasing my first yesod app in the next couple of weeks), and it's serialization isn't very good at the moment.
Just my two cents.
Thanks,
Felipe Almeida Lessa
<justin....@gmail.com> wrote:
> 1. Lack of projections makes complex queries almost impossible. Along with
> this, persistent isn't very composable. This really adds friction to any
> project that needs these types of queries (I've never run into a project
> that doesn't need these types of queries). It is also impossible to add
> parameters to queries that do not return in the result set of a query.
What I'd like to see (but don't know yet "how") is a better way of
writing raw SQL queries. Right now it's difficult to return back the
entities from SQL code. If we had a way of doing raw SQL queries
without having to manually get the entities back, it would be good
enough for me. It's extremely difficult to have something simple as
persistent but powerful as haskellDB, so having a nice fallback to raw
SQL is nice.
> 2. Persistent has memory leak issues (or yesod does). I haven't had time
> to write up a small example on this. Attempting to do a long action
> (reading a large csv file and inputting it into the database for example)
> in a yesod handler with persistent will continuously take more memory. This
> issue went away when I moved the same code into it's own executable. This
> isn't really a use case I'd normally attempt to support, but it may point to
> other problems.
Maybe your entire POST was being saved into memory?
Which reminds me that, although wai supports saving file uploads
directly to a file, AFAIK yesod-form doesn't.
Cheers,
PS: I'll do later some brainstorming about this question.
--
Felipe.
Greg Weber
Justin Greene
Glad to hear Persistent is the main complaint. After the move to Conduits and some further enhancements we now think WAI will be a solid piece of the Yesod ecosystem like Hamlet, and that Persistent is the main weak link.There is an issue open for foreign key issues in migrationsThis is related:https://github.com/yesodweb/persistent/issues/32As mentioned previously, the new version of Persistent (on master for real now) separates serialization from querying. However, we still need the better query interfaces.It should be possible to hook HaskellDB up to Persistent's serialization interface - one of the biggest problems with HaskellDB is that it does not give you a nice Haskell record. Although we still don't know how exactly to have projections.
Without projections this seems like a waste of time in my opinion. You'd still be limited to the forms of queries that persistent can already write.
Have you tried persistent-hssqlppp ? This project is young, but may hold the answer to raw SQL and projections. The upside is plain old SQL, but the downside is a lack of composition.
I have tried persistent-hssqlppp and it suffers from the same limitations normal persistent does. You can write sql as you say, but you can't insert parameters that are not contained in the result set. This means you may as well just write a view and map it to a normal persistent entity.
Greg Weber
On Tue, Dec 27, 2011 at 5:17 PM, Greg Weber <gr...@gregweber.info> wrote:Glad to hear Persistent is the main complaint. After the move to Conduits and some further enhancements we now think WAI will be a solid piece of the Yesod ecosystem like Hamlet, and that Persistent is the main weak link.There is an issue open for foreign key issues in migrationsThis is related:https://github.com/yesodweb/persistent/issues/32As mentioned previously, the new version of Persistent (on master for real now) separates serialization from querying. However, we still need the better query interfaces.It should be possible to hook HaskellDB up to Persistent's serialization interface - one of the biggest problems with HaskellDB is that it does not give you a nice Haskell record. Although we still don't know how exactly to have projections.
Without projections this seems like a waste of time in my opinion. You'd still be limited to the forms of queries that persistent can already write.
Have you tried persistent-hssqlppp ? This project is young, but may hold the answer to raw SQL and projections. The upside is plain old SQL, but the downside is a lack of composition.
I have tried persistent-hssqlppp and it suffers from the same limitations normal persistent does. You can write sql as you say, but you can't insert parameters that are not contained in the result set. This means you may as well just write a view and map it to a normal persistent entity.
Arash Rouhani
to your cabal file, then yesod-devel will reply with a totally
unreadable error message, even tough I encounter this all the time, I
sometimes forget that this is caused by a simple mistake, and I waste
time debugging for quite some time. Indeed this is just a minor issue,
but an overall "always working" development toolset would make Yesod
more loveable.
I would like to point out a success story, where Cameron noticed that in
hamlet the $tags should clearly state that an $unrecognized tag is not a
defined tag, instead of giving some confusing output. See this resolved
issue: https://github.com/yesodweb/hamlet/issues/28
So better error messages in our development tools is important, and we
are definitely already improving on this but still we have some issues,
I just gave one example but there are others too of course.
Best wishes and keep on hacking,
Arash Rouhani
David McBride
but it would be nice if the templates directory could be organized
into sub directories. If you try to do that now, it will work but
yesod devel will not pick up changes until you touch the handler that
calls that template.
I heard that ghc was getting a feature to detect files that changed
via a template haskell, which would solve this issue, so if that has
actually already been fixed since I last upgraded, then please
disregard and thanks a lot.
Greg Weber
Justin Greene
On Tue, Dec 27, 2011 at 8:27 PM, Justin Greene <justin....@gmail.com> wrote:
On Tue, Dec 27, 2011 at 5:17 PM, Greg Weber <gr...@gregweber.info> wrote:Glad to hear Persistent is the main complaint. After the move to Conduits and some further enhancements we now think WAI will be a solid piece of the Yesod ecosystem like Hamlet, and that Persistent is the main weak link.There is an issue open for foreign key issues in migrationsThis is related:https://github.com/yesodweb/persistent/issues/32As mentioned previously, the new version of Persistent (on master for real now) separates serialization from querying. However, we still need the better query interfaces.It should be possible to hook HaskellDB up to Persistent's serialization interface - one of the biggest problems with HaskellDB is that it does not give you a nice Haskell record. Although we still don't know how exactly to have projections.
Without projections this seems like a waste of time in my opinion. You'd still be limited to the forms of queries that persistent can already write.Yeah, the idea is to figure out a way to do projections - perhaps trying to copy the persistent-hssqlppp approach to projections.One of the things that makes all this harder is Haskell's weak records. I am going to try to push for a decision on a record solution for GHC next month. Obviously even if this moves quickly it will be a long time before we can use it.Have you tried persistent-hssqlppp ? This project is young, but may hold the answer to raw SQL and projections. The upside is plain old SQL, but the downside is a lack of composition.
I have tried persistent-hssqlppp and it suffers from the same limitations normal persistent does. You can write sql as you say, but you can't insert parameters that are not contained in the result set. This means you may as well just write a view and map it to a normal persistent entity.
Can you very briefly explain "you can't insert parameters that are not contained in the result set" so that everyone on the list understands?
Here is a quick made up example:
select count(*) as count from "User" where id = @userId
-----------------------------------------------------------------------------
count Int
In the above hssqlppp query you would need to have id be returned by the query so that you could filter on it. It's a general problem that persistent doesn't really handle at all, and is strongly related to projections. Now obviously we could pull off the above query with persistent just fine, but there are plenty of queries that you couldn't.
Michael Snoyman
1. For bugs, the best way to fix them is to submit a failing test case
and file a Github issue. If you can't get a failing test case, then
bring the issue up on the list and we can try and put one together.
2. We're not going to solve the large-scale Persistent issues
overnight. It could be that we'll introduce some radically different
query interface on top of what we have, or we might even move away
from Persistent entirely ultimately. But anything we do here deserves
a lot of thought and a dedicated discussion. FWIW, the cleanup Greg
and I have done so far on Persistent should make it much more
hackable, and let us solve problems like the recursive references
issue.
3. There's an entire issue devoted to yesod devel improvements[1].
What we really need is another person to step up and tackle this, like
Luite did previously. The nice thing about working on yesod devel is
that it's almost completely isolated from the rest of Yesod, so it
should be approachable by even a Yesod novice. I will eventually get
to it myself, but there are many other more pressing matters. If
you're interesting in being a "hero" here, please speak up.
4. The templates folder issue is funny, because it *used* to be in
separate folders, and has since moved into a single, shared folder.
While I don't think we're going to move back in the opposite
direction, modifying yesod devel to support either the templates/
folder or the hamlet/ folder should be possible (and easy, just a
one-line change to the determineHamletDeps function in the Build.hs
module in the yesod package). If anyone wants to experiment with this
and submit a pull request, I'll merge it.
Michael
Piyush P Kurur
> Hi all,
>
> If anyone's been following Github activity, you might be aware that
> we're well on our way towards the 0.10 release. At this point, I'd
> like to ask everyone: what drives you crazy about Yesod? Now is the
> absolute best chance to address problems, and we have plenty of time
> to do it. So let's get it all out there.
[snip]
>
> Be brutal, please :-).
It is difficult to be brutal when yesod gets so many things right but
here is my take
1. My biggest complaint is documentation. Yes the yesod book is good
but the haddock documentation is not up to the mark for an
otherwise great piece of software. It hurts especially when you
want to do non-standard things and you want to really know how
stuff works inside.
I had some difficulty figuring out the internals while hacking the
admin site. It would be good to have a documentation drive just
before the release of 1.0 when all features are frozen and the
remaining bugs are features. It is, I think very important to get a
stable well documented 1.0 which can compete with the likes of
rails and django.
2. Some more internal developer docs. This will not be user visible
but will aid development. Question to ponder would be whether it
make sense to use literate haskell.
3. Routing as you said is messy. In particular, I would like the old
approach: convert the uri to the route data type and dispatch on
the route data type. While this might be a bit slow I think in the
long run when developing nonstandard routing writing the
YesodDispatch class by hand will be easier.
Regards
ppk
Michael Snoyman
For the docs, I *really* need to ask for help from others. I agree
that we should make good Haddocks a priority for 1.0 (let's hold off
on 0.10, unless someone wants to go in and do it now). Having others
write those docs is probably a good idea, since I don't always know
what people find confusing.
For internal developer docs: I'm happy to write content, but I don't
know *what* to write or where to do it. I'm thinking we should keep a
bunch of files- as markdown- in the yesod repo itself. I think a good
starting point would be a FAQ. Of course, the first part of a FAQ is
the questions: can you compile together a list of questions you have?
It doesn't have to be long, just a few to get us started. Better yet:
submit a pull request to the yesod repo with the questions inside a
doc folder.
(Speak up now if you have a better idea of what to put the developer docs.)
As for the routing: I don't think going to two steps will make things
any simpler. You can always do that yourself internally (write a parse
function, then use it in your render function), and it will definitely
complicate the TH code.
Michael
Piyush P Kurur
[snip]
> For the docs, I *really* need to ask for help from others. I agree
> that we should make good Haddocks a priority for 1.0 (let's hold off
> on 0.10, unless someone wants to go in and do it now). Having others
> write those docs is probably a good idea, since I don't always know
> what people find confusing.
Yes I think the best time to go for the documentation drive is when
there is a feature freeze before the release of 1.0. I will try to
contribute to it as the internals become clearer to me.
> For internal developer docs: I'm happy to write content, but I don't
> know *what* to write or where to do it.
Can we have some kind of overview of the class structure of Yesod?
What each class is supposed to do and may be also a brief rational for
why things are the way it is. Some parts are already there in the book
but may be some where accessible inside the code.
Another question is should we go for literate haskell for parts of the
code and auto generate the developer notes ?
> I'm thinking we should keep a bunch of files- as markdown- in the
> yesod repo itself. I think a good starting point would be a FAQ. Of
> course, the first part of a FAQ is the questions: can you compile
> together a list of questions you have? It doesn't have to be long,
> just a few to get us started. Better yet: submit a pull request to
> the yesod repo with the questions inside a doc folder.
>
> (Speak up now if you have a better idea of what to put the developer docs.)
I will try to collect my thoughts and send a detailed response/pull
request. As I see it there should be enough docs for the developer to
define the necessary class instances for a yesod site by hand when the
default stuff no longer works or is less convenient.
> As for the routing: I don't think going to two steps will make things
> any simpler. You can always do that yourself internally (write a parse
> function, then use it in your render function), and it will definitely
> complicate the TH code.
It is quite likey that I am missing a trick here but if you can
clarify this will be great.
The current definition of YesodDispatch is something like this
(yesod-0.9.3.1 sorry did not try the latest)
class YesodDispatch a master where
yesodDispatch ::
Yesod master =>
a
-> Maybe Web.ClientSession.Key
-> [Data.Text.Internal.Text]
-> master
-> (Route a -> Route master)
-> Maybe Application
...
And here the [Data.Text.Internal.Text] is supposed to be the textual
representation of Route a (if I remember right). If we were to do
things by hand (which the current version of admin site does) there is
a possibility of making a typo in the strings used in rendering and
the dispatch part. In other words the developer does not get the
advantage of type safe route. Of course this will not occur in the
normal usage where routes and dispatch are auto generated but will
occur when defining by hand (more a developer issue than a user
issue).
Regards
ppk
Michael Snoyman
But that's the whole point of the system: doing these things by hand
is *always* error prone and can cause bugs, which is why we
auto-generate them. There's absolutely nothing we can do at the type
level to prevent this from happening.
Michael
Drew Haven
Drew Haven
drew....@gmail.com
Bryan Richter
Have you seen http://www.yesodweb.com/wiki yet?
I can't blame you if you haven't.
While I don't necessarily "hate" it, I think the website could use some love!
A. Even just copying more of the front page's style throughout the
site might add a more professional feel.
B. The wiki pages have no dates or history, so you can't tell if
they're part of some old, dead project or being actively edited.
C. The top-level nav links are paltry and well-hidden.
Compare these two pages:
1. http://www.yesodweb.com/wiki/faq
2. http://snapframework.com/faq
I wish I was a designer so I could offer more constructive feedback.
Maybe someone else has ideas?
Drew Haven
Max Cantor
Compilation time in yesod devel. I know with all the TH there isn't much that can easily be done unless there is another difficult change to the dynamic loading system, but I thought I'd throw it out there.
No Pure Handlers:
IIRC, happstack had a ServerPartT and a ServerPartTIO monad so pure handlers can be statically typed to be such.
No CRUD/Admin system:
I know that this is being worked on, and i've been thinking about it for a while, but it would be nice to be have some very simple way to go from PersistEntity <-> Form <-> Handler. I haven't yet come up with a solution which I liked but I'm sure this will be resolved soon with all the work going into it.
Persistent won't try to restore DB connections
If I restart postgres, I need to restart Yesod. We know that we can probably fix this by catching an error in the scaffolded Main.hs, but it would be nice if there was a standard way to do this.
Thats all I can think of for now that hasn't already been mentioned. In general, I love yesod and the more i learn about rails, the firmer that love becomes.
max
Greg Weber
Max Cantor
max
Michael Snoyman
something we should be able to solve with an improved pool package.
Rune Harder Bak
I meant to write a blog-post after splitting out my my yesod-utility
functions from the app and putting it on github.
But I've never really gone to that, and seeing this post I wanted to
add my 2. cents.
* Persistent
The simplicity of get/insert is very nice, and gets you started quickly.
But almost as quickly you find yourself needing more advanced queries,
and this is quite awkward with persistent.
I made my own automatic type-marshaling similar to the idea mention in
another thread and used by mysql-simple which help, but it is still
very unsafe and non-composeable, and feels not in the spirit of yesod
nor Haskell write your database-queries as strings.
Unfortunately not only do you need to resort to this for more advanced
queries, but if you need more advanced database
schemas, the persist DSL is not powerful enough either (Primary key as
combination of fields,
other data types as columns, foreign keys for other columns as id
etc).. This means you have to ditch it completely.
Doing that you might run into trouble as other part of Yesod
(yesod-form/yesod-auth) depend on it.
I end up using persistent for some tables, but sql only for the rest,
and doing most of the work in sql.
* yesod-form.
Again, it's very simple to have a simple working form with the
applicative interface, and that's great!
But then your designer comes and needs and extra class on the input fields, and
some special label-layout, and a separator here and there,
and you end up writing new fields for every field in your forms, and
using the clumsy and very verbose monadic syntax.
I find applicative syntax much clearer, and I really don't need the
power of monads.
I haven't tried coding it with digestive-functors, but it seems possible.
Also have some small points, like two forms on the same page doesn't
work doesn't.
I've address these myself although the api is not perfect.
* Long Types
The types when using both persistent and forms get's so long that I
usually omit them.
That is too bad, and I've added some type-synonyms to get around this.
* long compilation time.
This is especially a problem with yesod-devel
The compilation time is simply to high.
I use wai-handler-devel instead, though this isn't perfect.
I guess this is because TH is slow and working with Strings.
For production, perhaps the templates could be compiled by a program
instead of Template Haskell?
Or Template Haskell could simply call this program?
There is also some higher level features missing, not really things
wrong with Yesod,
but some things that would be nice to build on top.
* Support for dynamically updating your page with javascript.
I really think it is crucial, as more and more website is moving
towards dynamically updating one page via javascript,
instead of reloading the web page all the time.
When I first heard about widgets I thought this was what is was all about,
but they actually turn out to be more in the way than helpful.
I managed to have a separate default-layout for when it is requested
from javascript,
to not load the menus etc, but if the widget for-instance uses a
yesod-form jquery element, the whole jqeury library is requested on
load, And to properly load the widget css/javascript, I tried with the
yepnope loader, but then yepnope itself is loaded on every request.
* CRUD/Admin site generation
Already been mentioned.
* Job Scheduling
Some standard way of running scheduled background jobs, and scheduling jobs.
This could be done entirely outside Yesod, and is not I suppose
already quite possible without much trouble,
as we are programming in a high-level language, so there just need
some documentation on this.
Sorry for the long post!
-Rune
Max Cantor
Rune Harder Bak
> Your issue with multiple forms in a single page, is that when you have
> multiple AForms or MForms in a single html form element or when you have
> multiple html form elements?
The problem is that when two different forms submit to the same page,
yesod doesn't know which one was submitted.
I made a function that attached an id to the submit buttons to get
around this issue.
But maybe I should just submit to a different page and then redirect back.
Max Cantor
Rune Harder Bak
> in that case, I'm kind of surprised that you can just run both forms on the
> handler and see which one has FormSuccess instead of FormMissing or
> FormFailure
will look as if
both forms a submitted, and both forms will have the data autofilled
on the next reload.
Quite anti-intuitive. But the auto-generated name have to be the same
if you submit both
to different URLS, where you might only have one form.
So I actually not only needed to add an identifier to the submit button,
I also needed to be sure it generated the form that was not submitted
without looking at the submitted parameters.
(I.e. via the generateForm variant, instead of runForm)
