SnappyVote: A Weblocks-based voting application

[Anthony Fairchild]

Greetings Weblocks users,

I have been quite successful using Weblocks to build a web application and I thought I would introduce it on this list and provide some additional information about how I use Weblocks.  Maybe this information will be useful to newcomers and I'd love to get some feedback :-)

My application is called SnappyVote, http://snappyvote.com, and is meant to be a general purpose voting application.  The basic idea is you build a ballot with choices and then invite your friends to vote on it.  This can be useful for a variety of applications, like voting on where to go for lunch, or what is the greatest movie of all time or 'Employee of the Month'.  Right now it is alpha quality but it is being used by several people already.   I plan on extending it to allow additional voting types like ranked list, yes/no, to do surveys and provide web widgets that you can embed into blogs or facebook or whatever.

Now a little bit about how I use Weblocks.   I started using Weblocks when it was relatively new and used it for several small projects.  I also tried other lisp web frameworks (uncommon web, plain hunchentoot) and found that Weblocks made the most sense to me.   It was quite a learning curve and I did a lot of things wrong in the beginning.  At times I got incredibly frustrated with it but somewhere in the process everything just clicked and I have been productive ever sense.  One thing I learned fairly early on is the bundled widgets were great for quickly prototyping UI but when it comes to customizing the behavior they are fairly limited.  This is *not* really a problem because building custom widgets is not that hard.   So every widget in SnappyVote, except maybe forms, are all custom.

I do not use any of the continuation-based flow stuff like do-widget except for maybe one or two places.  While they do make some things cleaner, IMHO I dont think they are necessary and, at least for me, they caused more headache than what they were worth.  I recommend that newcomers stay away from these things unless there is a good reason to use them. YMMV.

I use Elephant with the BDB backend and I like it a lot.  It was very easy for me to understand the basics and I like defining my data in terms of CLOS and not tables.  The only thing I'd recommend is staying away from is associations as they seem to break under some conditions.  If the site outgrows Elephant I will probably refactor my db code to use postmodern.

For unit testing I use stefil, mostly for its simplicity and my familiarity working with it on other projects.  I have had great success using cl-selenium with stefil for UI testing.   cl-selenium is quite stable and very easy to set up.  I cannot recommend this enough and I was surprised that there is  no mention of it on the weblocks site or on this mailing list.  In addition to unit testing, I use SBCL's sb-cover library to get code coverage for my unit tests.  sb-cover has a nice form-by-form color-coded report that shows me exactly which code paths are missed in my unit tests.   Again, highly recommended!

Well, that's it.  Again, feedback is welcome and I hope this is helpful to some of you.

Thanks!

Anthony

[Ian Tegebo]

Thanks Anthony,

I'm still considering weblocks. Your description was informative.
I've been reading the docs and trying to understand the demos. I'd
love to see your code and any kind of documentation material you'd be
kind enough to supply would be a bonus.

By the way, the site looks good with one possible bug; after clicking
the "Send signup link", I was shown a message box directing me to my
email but after clicking okay I got the following message above the
email input field:

"The email address is not valid. Use password recovery if you already
have an account."

> --
> You received this message because you are subscribed to the Google Groups
> "weblocks" group.
> To post to this group, send email to webl...@googlegroups.com.
> To unsubscribe from this group, send email to
> weblocks+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/weblocks?hl=en.
>

--
Ian Tegebo

[Anthony Fairchild]

Ian,

Thanks for the feedback and the bug report.  I will fix that one soon.   

As for sharing the source I would be ok with sharing widget snippets or examples but unfortunately I cannot share the full source at this time.  Sorry that is not very helpful.

Anthony

[Ian Tegebo]

Anthony,

I think it was mentioned before in the "Warned off Weblocks on
StackOverflow?" thread, but weblocks would benefit from documentation
work. I'd enjoy a new tutorial in a style that doesn't use
continuations. No worries about snappyvote source, I just thought
that'd be a low-energy way to contribute. I'm glad you pointed to
elephant, cl-selenium, postmodern, and stefil - it's instructive to
know what the community is using.

Cheers,
On Sat, Jul 2, 2011 at 4:57 PM, Anthony Fairchild

[Nandan Bagchee]

work.  I'd enjoy a new tutorial in a style that doesn't use
continuations.

http://github.com/nunb/weblocks-tutorial ... suggestions?

 

 No worries about snappyvote source, I just thought
that'd be a low-energy way to contribute.  I'm glad you pointed to
elephant, cl-selenium, postmodern, and stefil - it's instructive to
know what the community is using.

Well, especially on the elephant side I think some more detail would be useful to the community.

cl-prevalence is in use by default, and elephant/bdb seems almost deprecated (future elephant releases will use cl-prevalence) and hard to get working on some platforms (OSX 10.5 comes to mind).

cheers, 

 Nandan

 

[Anthony Fairchild]

I actually develop on a mac with OSX 10.5 and then deploy to a debian server.   Configuring elephant was not exactly strait forward and took a little bit of trial and error.   I use the darcs elephant-1.0 branch instead of pulling from quicklisp.  I had some trouble with the quicklisp version but it could be user error on my part.

The critical piece I think was to get the proper version of bdb install, which is bdb4.7.  I installed this with macports "sudo port install db47".   Next I set up the my-config.sexp file under elephant-1.0/ with the following:

#+darwin ((:COMPILER . :GCC) (:BERKELEY-DB-VERSION . "4.7")

          (:BERKELEY-DB-INCLUDE-DIR . "/opt/local/include/db47/")

          (:BERKELEY-DB-LIB-DIR . "/opt/local/lib/db47/")

          (:BERKELEY-DB-LIB . "/opt/local/lib/db47/libdb-4.7.dylib")

          (:BERKELEY-DB-DEADLOCK . "/opt/local/bin/db47_deadlock")

          (:BERKELEY-DB-CACHESIZE . 20971520) (:BERKELEY-DB-MAX-LOCKS . 2000)

          (:BERKELEY-DB-MAX-OBJECTS . 2000) (:BERKELEY-DB-MAP-DEGREE2 . T)

          (:BERKELEY-DB-MVCC) (:CLSQL-LIB-PATHS) (:PREBUILT-LIBRARIES)

          (:WARN-WHEN-DROPPING-PERSISTENT-SLOTS . T)

          (:RETURN-NULL-ON-MISSING-INSTANCE . T)

          (:NO-DESERIALIZATION-PACKAGE-FOUND-ACTION . :WARN))

#+linux ((:COMPILER . :GCC) (:BERKELEY-DB-VERSION . "4.7")

         (:BERKELEY-DB-INCLUDE-DIR . "/usr/include/")

         (:BERKELEY-DB-LIB-DIR . "/usr/lib/")

         (:BERKELEY-DB-LIB . "/usr/lib/libdb-4.7.so")

         (:BERKELEY-DB-DEADLOCK . "/usr/bin/db4.7_deadlock")

         (:BERKELEY-DB-CACHESIZE . 20971520) (:BERKELEY-DB-MAX-LOCKS . 2000)

         (:BERKELEY-DB-MAX-OBJECTS . 2000) (:BERKELEY-DB-MAP-DEGREE2 . T)

         (:BERKELEY-DB-MVCC) (:CLSQL-LIB-PATHS) (:PREBUILT-LIBRARIES)

         (:WARN-WHEN-DROPPING-PERSISTENT-SLOTS . T)

         (:RETURN-NULL-ON-MISSING-INSTANCE . T)

         (:NO-DESERIALIZATION-PACKAGE-FOUND-ACTION . :WARN))

I don't have much experience with cl-prevalance but Elephant works well for me for now.  I may try cl-prevalence in a future project.   

Hope this helps,

Anthony

[Scott Helvick]

>> cl-prevalence is in use by default, and elephant/bdb seems almost
>> deprecated (future elephant releases will use cl-prevalence)

Not to go too off-topic, but I thought the postmodern backend was the
"way of the future" for elephant. Is this not recommended or is it
just a matter of options?

[Ian Eslick]

For the past 5 years or so I've been the lead developer on Elephant and the BDB backend.  I have been and will be inactive for some time yet.  I may make some updates when I do a bug / upgrade pass on the two legacy commercial websites I have that are running on weblocks + Elephant.

The BDB backend hasn't been updated to the latest release, nor well-tested in probably 1.5 years.  However, with a few quick tweaks, it shouldn't be hard to keep it current and fix any simple problems (e.g. the one mentioned for Mac 10.5).  The downside of BDB generally is multiple-machine scalability as we don't currently have anything in place that would build on BDB's replication infrastructure to scale out.  That's a fairly hairy project and the right person hasn't come along to pull it off yet.

To deal with scaling, some smart folks wrote the Elephant-postmodern backend that simulates the primitive BTrees on which the elephant object API is based so it's a bit slower than BDB depending on use case.  Because it is based on postmodern / postgres, you can use postgres replication to scale the Elephant API over more machines which should be enough for most applications.  Direct use of the postmodern API is of course also an option and will have somewhat better performance characteristics.

cl-prevalence is really nice and highly performant, but I don't like the fact that I have to encode the transaction information by hand - Elephant makes persistence really easy.

Oh yes, associations are still a beta feature and along with sets, set-valued slots and cached slots have not been fully vetted yet.  That's one of the big reasons we've been in a perpetual holding pattern waiting for a 1.0 release.  Anyone who wants to write some tests for these features might motivate me or others to invest a day or two to fix bugs.  I know one or two other people have expressed interest in ironing down the last few issues and issuing a 1.0 release of Elephant.

I probably won't invest any more development time in Elephant beyond 1.0.  For awhile I'd envisioned a pure lisp backend but I don't have the flex time for a project of that magnitude anymore.  Moreover, with the proliferation of NoSQL technologies since Elephant was first developed, I'd be tempted to migrate to something like CouchBase, Mongo or another document DB solution that scales horizontally (usually by ignoring database-level, cross-object transactions) and has a simple API.  

Both of these databases are fast, scalable and would be easy to implement underneath an Elephant-like persistent-object API.  They would work well behind the weblocks API also.  Mongo, with explicitly managed indices, could probably support a huge chunk of the Elephant API.  Associations and similar referential integrity problems might be tricky though.

Cheers,

Ian

[Nicholas Leippe]

On Monday, July 4, 2011 10:51:38 PM UTC-6, Ian Eslick wrote:

For the past 5 years or so I've been the lead developer on Elephant and the BDB backend.  I have been and will be inactive for some time yet.  I may make some updates when I do a bug / upgrade pass on the two legacy commercial websites I have that are running on weblocks + Elephant.

The BDB backend hasn't been updated to the latest release, nor well-tested in probably 1.5 years.  However, with a few quick tweaks, it shouldn't be hard to keep it current and fix any simple problems (e.g. the one mentioned for Mac 10.5).  The downside of BDB generally is multiple-machine scalability as we don't currently have anything in place that would build on BDB's replication infrastructure to scale out.  That's a fairly hairy project and the right person hasn't come along to pull it off yet.

To deal with scaling, some smart folks wrote the Elephant-postmodern backend that simulates the primitive BTrees on which the elephant object API is based so it's a bit slower than BDB depending on use case.  Because it is based on postmodern / postgres, you can use postgres replication to scale the Elephant API over more machines which should be enough for most applications.  Direct use of the postmodern API is of course also an option and will have somewhat better performance characteristics.

cl-prevalence is really nice and highly performant, but I don't like the fact that I have to encode the transaction information by hand - Elephant makes persistence really easy.

Oh yes, associations are still a beta feature and along with sets, set-valued slots and cached slots have not been fully vetted yet.  That's one of the big reasons we've been in a perpetual holding pattern waiting for a 1.0 release.  Anyone who wants to write some tests for these features might motivate me or others to invest a day or two to fix bugs.  I know one or two other people have expressed interest in ironing down the last few issues and issuing a 1.0 release of Elephant.

I probably won't invest any more development time in Elephant beyond 1.0.  For awhile I'd envisioned a pure lisp backend but I don't have the flex time for a project of that magnitude anymore.  Moreover, with the proliferation of NoSQL technologies since Elephant was first developed, I'd be tempted to migrate to something like CouchBase, Mongo or another document DB solution that scales horizontally (usually by ignoring database-level, cross-object transactions) and has a simple API.  

Both of these databases are fast, scalable and would be easy to implement underneath an Elephant-like persistent-object API.  They would work well behind the weblocks API also.  Mongo, with explicitly managed indices, could probably support a huge chunk of the Elephant API.  Associations and similar referential integrity problems might be tricky though.

What about rucksack?