The 80/20 rule

67 views
Skip to first unread message

mdipierro

unread,
Jul 16, 2009, 10:49:14 PM7/16/09
to web2py Web Framework
Pareto (http://en.wikipedia.org/wiki/Pareto_principle) once said that
80% of effects come from 20% of causes. For software we can re-phrase
as 80% of users use only 20% of features.

These are not at all a scientific statements but you understand what
they mean.

We can add more and more features to web2py with the effect that:
- web2py grows in complexity to the point that it becomes harder to
maintain
- web2py gets slower
- I spend all my time reviewing patches instead of improving
documentation
- You spend all your time thinking how to improve web2py instead of
building applications with it
- Many of these improvement will not buy us a significant number of
users.

I am not trying to discourage people from submitting patches and I
very very much appreciate people who have done so. Most if not all the
patches that I have received were good and needed. I am sure I will
continue to receive excellent patches.

Yet, when discussing the roadmap it has to be clear that the point
should not be adding new features. The goal should be locking existing
features: make sure they are well documented, make sure there are no
problems, make sure everything has tests.

We also need a plugin system but that is not a new web2py feature.
That is a set of specifications and naming conventions for writing
models/views/controllers. It is possible that we may have to modify
admin as result of these specs (in order to apply/remove plugins).

Finally, this community has to do a better job at outreach. You should
talk to your friends and blog about web2py. You must talk about the
applications you build with it. We do not lack developers. We lack
salesmen. This is not because we want to have more user but because
more users means web2py is tested more, there will be more web2py
jobs, and you (as early adopters) will have a better chance to sell
your web2py based solutions.

Massimo

Vidul Petrov

unread,
Jul 16, 2009, 11:18:18 PM7/16/09
to web2py Web Framework
Hi Massimo,

I fully agree, everything is very well said IMHO.

Only one small addition - most of the MVC users have never even heard
about most of the features (not to mention the rich set that WEB2PY
offers).

Do you think that a website for WEB2PY like jobs.perl.org and
jobs.rubynow.com would help (probably some sample works should be
shown there also)?

Regards,
Vidul

mdipierro

unread,
Jul 16, 2009, 11:24:52 PM7/16/09
to web2py Web Framework
I am not familiar with those web sites. What do you suggest?

Vidul Petrov

unread,
Jul 16, 2009, 11:42:49 PM7/16/09
to web2py Web Framework
I suggest a similar website (actually the latter is a total copy of
the former), the work will be a motivation for many people to start
learning WEB2PY.
Also short intros to WEB2PY - probably what / how / when can be done
(more or less a cookbook).
And last but not least - how can anyone find a work with WEB2PY -
there are no projects, no search and demand announced, at least not
officially?

Bottiger

unread,
Jul 17, 2009, 3:34:11 AM7/17/09
to web2py Web Framework
> Finally, this community has to do a better job at outreach. You should
talk to your friends and blog about web2py. You must talk about the
applications you build with it. We do not lack developers. We lack
salesmen.

I apologize in advance for being blunt.

I thought about writing about Web2Py on my blog, but what can I say?
That everything is a little different but slightly better than Django?
They aren't going to believe me, and it will get ignored and downvoted
just like the recent Reddit stories. A simple search for web2py on
Reddit shows many stories with heavy downvotes and people simply not
interested that there is a Django or Pylons clone, when Django and
Pylons do just fine.

The problem with Web2Py is that it does not have a niche. Right now,
Django is occupying the same niche. As the first-comer, it has the
advantage that people already know how to use it. Until Web2Py manages
to have a benefit greater than its own learning curve, Web2Py will
continue to sit in second place.

If it has something that no one else does (a feature), then you will
be justified in bragging about it, and the detractors will not be
justified in denouncing it. This is not to say that we should have
every feature possible; only that we need ones that set Web2Py apart.

As for which feature we should include, I currently have 1 in mind
that will set it apart from the others:

First class support for a free distributed database.

The reason is simple. There is a need for a scalable database that
does not rely on Google or Amazon. By focusing on the needs of
ambitious projects, you will increase the likelihood of getting free
press that matters: from big flashy projects. Instead, right now the
ecosystem consists mostly of hobby projects on the GAE and internal
websites. This is probably not the only killer improvement that Web2Py
can make, but its the one the personally interests me the most.

(P.S. Massimo, I believe you've received my synopsis on supporting
MongoDB before CouchDB).

Joe Barnhart

unread,
Jul 17, 2009, 5:29:52 AM7/17/09
to web2py Web Framework
Let me also be blunt.

Your message started with a serious but constructive criticism of
web2py and then degenerated into a whining post for including your own
favorite database. If that's the only feature you seek in web2py,
download the source code and have at it.

What a disappointing finish.

JohnMc

unread,
Jul 17, 2009, 7:34:08 AM7/17/09
to web2py Web Framework
The problem with Web2Py is that it does not have a niche. Right now,
Django is occupying the same niche. As the first-comer, it has the
advantage that people already know how to use it. Until Web2Py manages
to have a benefit greater than its own learning curve, Web2Py will
continue to sit in second place.

I hereby invoke the Jack Welch rule - Be first or second in any
endeavor, otherwise get out.

Were Web2Py to be number two behind Django after a concerted effort,
that quite frankly would be an outcome worth crowing about. Being
number two in the software game is not a bad place to be.

I have said in a previous post that Web2Py support of a schemeless DB
would put it in a position to have a niche. The 'cloud' might have a
lot of action but there is still going to be whole swaths of the
economy where the cloud will not be feasible (eg. NYSE). Which DB I
leave up to those more insightful than myself.

JohnMc

JohnMc

unread,
Jul 17, 2009, 7:38:45 AM7/17/09
to web2py Web Framework
One other point. Web2Py is lightweight enough using sqlite that it
fits quite well in areas where --

* portability is required.
* space is at a premium.
* The ability to have multitude of support applications are not
possible (eg java creep, jre)

Just a thought.

viniciusban

unread,
Jul 17, 2009, 9:57:14 AM7/17/09
to web2py Web Framework

On Jul 16, 11:49 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> We can add more and more features to web2py with the effect that:
> - web2py grows in complexity to the point that it becomes harder to
> maintain
> - web2py gets slower

I think these 2 are not good choices because a good framework must be
fast and simple. You achieved that already.

I'd choose to improve free documentation. I really find myself
confused with web2py's documentation.
I think the free available resources must be improved and we won't see
many developers until it gets stronger.


>
> Finally, this community has to do a better job at outreach. You should
> talk to your friends and blog about web2py. You must talk about the
> applications you build with it. We do not lack developers. We lack
> salesmen.

I just started it the day before yesterday, in Brazilian Portuguese:
http://aprenda-web2py.blogspot.com
In 2 days it has about 300 views with virtually no sharing.

We have a strong Django community, but we don't have statistics about
how many are really *working* with it.

I talked to Alvaro Justen about it and we'll write some posts from the
web2py user's point of view.

It won't be a manual, absolutelly! It'll be a blog with practical
web2py development emphasis. Maybe a lot of howtos and many principles
explained.

I started this blog with 2 posts "selling" some nice features about
web2py.
The first one, I got from web2py's homepage.

The other one talks about the "backwards compatibility" thread I
started this week, but translated.
As I said before, I think this is some of the most important web2py's
characteristics.

I can see many advantages in web2py and I will write about them in a
series of posts, emphasizing the time-to-market web2py allows us to
have.

--
Vinicius Assef.

Hans Donner

unread,
Jul 17, 2009, 11:32:24 AM7/17/09
to web...@googlegroups.com
i m seeing many posts about the doc needing improvement, and some discusions about it should be done in the wiki that is not there yet.
However, i see little that people offer documentation or patches that work on the documentation.

please, can we see more of the latter and less of the first? also, do no point only to massimo that he should provide it.
document what is not clear to you, and after help from this list, complete it and submit it so it can be added to the documentation or examples.

hans

mdipierro

unread,
Jul 17, 2009, 1:23:38 PM7/17/09
to web2py Web Framework
+1

Zoom.Quiet

unread,
Jul 17, 2009, 1:27:35 PM7/17/09
to web...@googlegroups.com
On Sat, Jul 18, 2009 at 01:23, mdipierro<mdip...@cs.depaul.edu> wrote:
>
> +1
>

+1
but that realy always matter breaking my plan,
sooooo chinese documanet,is always in my head...
--
http://zoomquiet.org 人生苦短,Pythonic!-)
流程是对先前蠢行的内在反应! ~ Clay Shirky

Yarko Tymciurak

unread,
Jul 17, 2009, 1:48:54 PM7/17/09
to web...@googlegroups.com
funny - sometimes no matter how much people want it,  "just do it" doesn't seem to work.... sometimes you have to figure out what is missing or what is wrong...

sometimes "just doing it" is part of the road to figuring that out; sometimes it doesn't help at all - we are, it seems to me, experiencing the frustration of the latter...

Bottiger

unread,
Jul 17, 2009, 3:17:49 PM7/17/09
to web2py Web Framework
I wanted to do so, but Massimo made it clear that he wanted to work on
the new DAL by himself. I already offered to help.

Bottiger

unread,
Jul 17, 2009, 3:43:18 PM7/17/09
to web2py Web Framework
Django also has the ability to use SQLite.

Web2Py is a somewhat better at everything Django tries to do, but the
differences are simply not enough for the average programmer to see
through in 1 sitting. Let's take a look at the commonly cited features
of Web2Py.

**Please note this is not a bashing of Web2Py.**

- Web Editor

This is pretty nice yes, but its no replacement for a regular text
editor. Often times when I am using it, there will be graphical
glitches where text gets smeared all over the place. Often times when
I am editing even a simple tutorial, the save stops working, and
doesn't even tell you that the session has timed out. For most people,
this is not that much of a difference.

- Compilation

Django does this automatically, so I don't know why this is considered
a feature.

- Secure

The average programmer does not have the ability to see any concrete
benefit this has over Django. From what I've seen learning Web2Py, it
doesn't do anything that Django doesn't do with filtering and escaping
text, html etc.

- Server-side form validation

This is nice, but it is the trademark of Django for having really nice
auto-generated forms.

- Internationalization

http://docs.djangoproject.com/en/dev/topics/i18n/

- SSL Streaming

Django does it with middleware.

- And everything else I can think of.

Hernan Olivera

unread,
Jul 17, 2009, 4:02:03 PM7/17/09
to web...@googlegroups.com
2009/7/17 Bottiger <bott...@gmail.com>:
Hi guys

Excuse me because my donkey-english-style :) I hope you will undestand.

I think there is a niche that web2py could fit. I really have not made
more than little tests with web2py. I'm less than a beginner. What's
my point? I think there is a need to migrate a lot of old
administrative systems to a good web solution. Everyone with a vb or
.net application in an organization is thinking about migrating it to
a web-app.

I've started to migrate such app, first evaluating web frameworks,
only Python ones, and choose Django.

BUT

- It must be more simple to build a usual dataentry form, preloading
values, filling associated values (ej: item number -> description,
price, etc). Yo need to work a lot with jquery, use some autocomplete,
but all is hand-made. Admin is ok, but nothing to do with this kind of
stuff. There are not so much use cases, it must be easy to automatize
this.

- It must be more simple to work with an existing DB and a new DB,
migrating data, synchronizing them, in a smooth way to replace the old
system, withouth stopping, aligning and testing both systems, still
everyone wants to use only the new system.

- It must be more simple to build reports, and printing formatted stuff.


I think that organizations tend to mix web sites and administrative
systems in one web system, with one front to internet and other (or
the same) for the intranet.

I believe that if web2py can do that, tons of systems will migrate.

And, everyone can build a web site with Rails, Django, Symfony etc.

My 2 cents




--
Hernan Olivera

mdipierro

unread,
Jul 17, 2009, 4:09:39 PM7/17/09
to web2py Web Framework
Not quite. I am justing waiting to cleanup the specs before I can ask
for help. I hope to be done soon so I can use your help.

Massimo

mdipierro

unread,
Jul 17, 2009, 4:11:11 PM7/17/09
to web2py Web Framework
Herman, you have good points

On Jul 17, 3:02 pm, Hernan Olivera <lholiv...@gmail.com> wrote:
> 2009/7/17 Bottiger <bottig...@gmail.com>:

Yarko Tymciurak

unread,
Jul 17, 2009, 4:12:45 PM7/17/09
to web...@googlegroups.com
On Fri, Jul 17, 2009 at 2:43 PM, Bottiger <bott...@gmail.com> wrote:

Django also has the ability to use SQLite.

Web2Py is a somewhat better at everything Django tries to do, but the
differences are simply not enough for the average programmer to see
through in 1 sitting. Let's take a look at the commonly cited features
of Web2Py.

**Please note this is not a bashing of Web2Py.**

- Web Editor

This is pretty nice yes, but its no replacement for a regular text
editor. Often times when I am using it, there will be graphical
glitches where text gets smeared all over the place. Often times when
I am editing even a simple tutorial, the save stops working, and
doesn't even tell you that the session has timed out. For most people,
this is not that much of a difference.

You have these kinds of lockups?!?

I've always found I prefer using MY editor - if I _really_ want to do it thru the web (even for small stuff) I use firefox, and get the "It's all Text" plugin - specify "my favorite editor" (vim, for me).  The only thing annoying is that "plain text" is not the default mode in web2py, so I always have to either twiddle this (which is what I do - because I usually develop from WingIDE, which I greatly prefer) or patch the defaul (don't know I've ever done that... I just stay out of any but the most minor of developing on the web).

https://addons.mozilla.org/en-US/firefox/addon/4125
 


- Compilation

Django does this automatically, so I don't know why this is considered
a feature.

Really?!?

Are we talking byte code compilation / precompilation (prior to deployment), or are we talking about compilation of all-in-one executables for Mac of PC?

"Automatic" compilation (bytecode) is what Python does, actually.... but I don't think that's what we mean here...
 


- Secure

The average programmer does not have the ability to see any concrete
benefit this has over Django. From what I've seen learning Web2Py, it
doesn't do anything that Django doesn't do with filtering and escaping
text, html etc.

Does django now do this by default?  (I think it didn't used to - it just provided a way for you to, as I recall...).
Anyway, that's a good thing.
 


all the rest of this is smack-dab peachy:  I'll remind you of one thing - there was a website, guy from NOAA I think it was - that showed all the frameworks that claimed to have something;  he tried building something simple with them and uncovered all the flaws and gotchas and try to say "here's what I would (wouldn't) want to build with".... most things just took a long time...

web2py wasn't in that, but we discussed this about a year ago - web2py would have done really well in that.   I have often pined that it would be good to re-do that video with current versions of "stuff", and include web2py...

For example:  1 year ago, there was a contest:  "come bring your favorite framework and deliver an app" - no pre-anouncement of what you would build, 24 hr (? or was it 12 hrs)  time limit, start to finish.

The ONLYONE that even finished:  you guessed it:  web2py; and that in LESS than the alloted time.  (See a running result on http://web2py.appspot.com/survey).   Since no one else completed, web2py didn't get recognized at the conference (I think they didn't want to alienate the other frameworks).

SO - this is what there is to remember about web2py.

There is more - at the PyCon DoJo, no one (not Bruce Eckel - who suggested and helped us outline and plan it - he thought we were too ambitious, that no one would get "all that done" in an hour; not people participating, including some knowledgable people - they were surprised... but it was a lot of "mind shift" to absorb in an hour regardless)...

SO when you spout features, spout accessibility of those features too....

The reason (my opinion) Massimo is not wanting "help" w/ next DAL is he's trying to make it easier for people to use, based on all the questions he's answered (no one else has that accumulation of experience w/ the questions).

Is there work to do?  Yes.

Is there anything to web2py ("that other's don't have") - You bet ... just be sure you're measuring the right things.
(I really, now, am going to have to dig up that URL to the video comparison...)


- Yarko

Bottiger

unread,
Jul 17, 2009, 5:15:54 PM7/17/09
to web2py Web Framework
> Since no one else completed, web2py didn't get recognized at the conference

You also forgot to mention that no one else *competed*.

Now it may very well be true that Web2Py is quicker to develop. When I
look at the code, it does seem shorter and cleaner, but I can still
program the same thing faster in Django than I can in Web2Py because:

1. I learned it a while ago first.
2. The free Django documentation is much better than the Web2Py one.

Again, this goes back to my point that Web2Py will remain in 2nd place
until its benefits are larger than its learning curve. Fortunately for
me, I have not sunk too much time into Django and I have had the time
to see where it is better than Django. However the majority of people
are not so fortunate.

Being easy to develop is good, but at the same time it can come with a
cost. It produces a bunch of unpolished apps that the public sees and
becomes immediately turned off. Here are a few examples that I found
on Reddit and Google:

http://www2.un.int/ - Broken pictures everywhere. Completely static
front page. Subpages appear completely static.
https://mdp.cti.depaul.edu/personal/default/register - Broken
http://yao.appspot.com/ - Featureless blog
http://mmlado.com/ - Yet another featureless blog
http://morrisdecode.com/ - Looks like the same theme as Web2Py
default.
http://www.professionalit.com.br/ - Completely static page that could
have been done in HTML.

This is not to say that being easy to develop is bad. But it does say
that Web2Py is not so easy compared to Django that we can rely on it
to set us apart.

On Jul 17, 1:12 pm, Yarko Tymciurak <yark...@gmail.com> wrote:
> than the alloted time.  (See a running result onhttp://web2py.appspot.com/survey).   Since no one else completed, web2py

mdipierro

unread,
Jul 17, 2009, 5:37:14 PM7/17/09
to web2py Web Framework


On Jul 17, 2:43 pm, Bottiger <bottig...@gmail.com> wrote:
>
> - Compilation
>
> Django does this automatically, so I don't know why this is considered
> a feature.

Not the same thing. Python automatically bytecode copiles modules tha
first time they are imported. that is the same for all python
frameworks, including web2py and Django.

web2py does two more things for you:
1) It parses all views and collapses the three structure associated to
each action into one byte-code compiled view. that means there is no
parsing and minimal file IO when executing a view in a bytecode
compiled app. In Django the parsing/incude/extend happen a runtime
instead, no matter what.
2) web2py includes a simple mechanism to package the bytecode compiled
app and distribute it without source code. In Django you have to do
this manually by locating the files you need and packaging them. As
discussed in the previous point, in Django templates are not parsed
before bytecode compilation therefore, no matter what, you have to
distribute them in source form even if the rest of the modules are
distributed bytecode compiled.

Also do not forget the licensing issues. Web2py executes applications
and applications (generally) do not import web2py modules (with some
exceptions). Instead Django applications import Django modules. I may
be wrong but I believe this is not a minor issue. I believe Django app
are derivatives of Django. web2py apps are not derivatives of web2py.
In fact I do allow you to distribute your apps under any license you
like.

Massimo

mdipierro

unread,
Jul 17, 2009, 5:46:25 PM7/17/09
to web2py Web Framework
On Jul 17, 2:43 pm, Bottiger <bottig...@gmail.com> wrote:
> - Secure
>
> The average programmer does not have the ability to see any concrete
> benefit this has over Django. From what I've seen learning Web2Py, it
> doesn't do anything that Django doesn't do with filtering and escaping
> text, html etc.

Django does a good job at security with a few caveats. For the Django
admin does not force secure cookies and you can have a Django admin
over an unsecure channel. The web2py admin does not allow this unless
you trick it by using a proxy. Django does escape by default all text
coming from the users (XSS vulnerability). It leaves to the developer
the task to do it. We take the opposite approach. We do not trust the
developer.

> - Server-side form validation
>
> This is nice, but it is the trademark of Django for having really nice
> auto-generated forms.

Yes

> - Internationalization
>
> http://docs.djangoproject.com/en/dev/topics/i18n/

Yes. But as an exercise try explain us how to translate a Django app
vs how to translate a web2py app.

> - SSL Streaming
>
> Django does it with middleware.

But that is the point. In Django, the developer has to setup the
middleware to make it work. In Django all files are automatically
streamed in upload and download. Downloads also provide out of the box
the if modified since protocol and range requests.

Note to self.... remember how much more stuff web2py does under the
hood when commenting on benchmarks.

Massimo

Bottiger

unread,
Jul 17, 2009, 5:54:48 PM7/17/09
to web2py Web Framework
> It parses all views and collapses the three structure associated to each action into one byte-code compiled view. that means there is no
parsing and minimal file IO when executing a view in a bytecode
compiled app.

I don't think this makes a large difference in practice from the
benchmarks I have seen. In Django the code is automatically compiled
to a pyc upon running it. These pycs are likely to be cached in memory
on the OS level because they are not that big. So the difference
really is between 1 big disk cache lookup and 3 smaller ones. Even if
this does make a difference, it is arguably not big enough for the
common programmer to warrant switching. This is why web programming
has shifted to dynamic languages like Python and Ruby even though
there are people still doing C++, Java, and C#.

> Also do not forget the licensing issues. Web2py executes applications and applications (generally) do not import web2py modules (with some exceptions). Instead Django applications import Django modules...

Actually this is not an issue at all. All the 3 major frameworks:
Pylons, Django, and Rails, do not use GPL (unlike Web2Py). They
instead use a freer BSD-style license that does not come with the
definition of "derivatives" which is so often a problem in GPL.

mdipierro

unread,
Jul 17, 2009, 6:06:32 PM7/17/09
to web2py Web Framework
On Jul 17, 4:15 pm, Bottiger <bottig...@gmail.com> wrote:
> > Since no one else completed, web2py didn't get recognized at the conference
>
> You also forgot to mention that no one else *competed*.

This is an old wound and we should let it alone. But to get the fact
straight. At Flourish 2008 a number of people were invited to compete
in building app with their framework. There was no sign in process.
You only had to pick up the rules and had 24hrs to deliver. Lots of
people were on the mailing list. I have seen 4 people picking up the
rules representing different frameworks, but people could pick them up
anytime so there may have been more. We were not required to work
there (which I though was odd since I was expecting some supervision)
so everybody left. I decided to work there instead. I was the only one
to deliver something which is not quite the same as saying that I was
the only one to participate. It should be noted that in the days
before the event we (the 10 people people who said they were going to
participate) we were told this was going to last 4hrs and start on
Sunday. I received an email on Saurday morning (while I was at the
office in a meeting) saying the rules had changed and they were
starting immediately. I protested. Other people did. So they agreed to
count the 24hrs from the moment people would pick up the rules. It was
a mess. Ian Bicking was there. Ask him.

> Now it may very well be true that Web2Py is quicker to develop. When I
> look at the code, it does seem shorter and cleaner, but I can still
> program the same thing faster in Django than I can in Web2Py because:

I do not think Django is our competitor. Django is our friend. If
people have already chosen Django I do not see much of a reason to
move to web2py. I think we should go after PHP and .NET and J2EE
users. Show them how life could be better.

> 1. I learned it a while ago first.
> 2. The free Django documentation is much better than the Web2Py one.

Java has more documentation than Django because it needs it. The same
applies to the Django documentation vs web2py. I am in the process of
revising the book and with the exception of CRUD, AUTH and services,
It is amazing how everything else in the book is there and current.

> Again, this goes back to my point that Web2Py will remain in 2nd place
> until its benefits are larger than its learning curve. Fortunately for
> me, I have not sunk too much time into Django and I have had the time
> to see where it is better than Django. However the majority of people
> are not so fortunate.

Again, if you are a Django enthusiast and you spent time learning it I
see very little advantage in learning web2py. On the other side if you
have not learned Django yet, web2py will be easier to learn. In the
long run web2py does more stuff for you out of the box than Django
does. Except the Django admin looks much better than the web2py
appadmin.

> Being easy to develop is good, but at the same time it can come with a
> cost. It produces a bunch of unpolished apps that the public sees and
> becomes immediately turned off. Here are a few examples that I found
> on Reddit and Google:
>
> http://www2.un.int/- Broken pictures everywhere.

That is a Django app not a web2py app. It was working fine until 2yrs
ago when my team of students released any access to the application.
It is still working fine considering propably has had no maintenance
since. BTW all pages but the frontpage are dynamic.

> This is not to say that being easy to develop is bad. But it does say
> that Web2Py is not so easy compared to Django that we can rely on it
> to set us apart.

Once I again, 1% of the J2EE market share or 10% of the PHP share are
both better than 100% of the Django market share.

Massimo

waTR

unread,
Jul 17, 2009, 6:10:22 PM7/17/09
to web2py Web Framework
I would really love to see a web2py-developers google group form, and
a web2py-management group. At least then there can be a forum to
discuss the strategic plan, and general organization for the project,
and not pollute the web2py-users group.

One must not forget that we are comparing frameworks that are at
different points in their development. Django is established and
mature, Web2py is young and still in early stages. While the web2py
code is as solid and stable as (if not more than) Django, a project is
more than just code, it is the community of volunteers and the
consistent effort they are putting in.

The best example of the framework that COULD, but didn't due to
failure in creating an involved community was Perl's Gantry. The tech
was great, it was also far simpler to get up and running than any
other perl framework. It was really a "rails" for perl. HOWEVER, the
community never materialized, and the project is now dead in the
water. One of the biggest killers of open-source projects is the
failure to build a strong participatory community.

In fact, I learned web2py before any other framework, but not before
researching a lot of other frameworks (including Django, and those in
other languages other than Python). However, as it is, if the effort
behind web2py is built around convincing coders from Django and making
them see the light that web2py is better. I am not interested in
participating in that crusade. I would prefer if the framework
concentrates on just being another framework that is different in just
the right ways. It is still a goal to be better than the rest, but you
need to be a bit more specific than that to reach your goal.



On Jul 17, 2:15 pm, Bottiger <bottig...@gmail.com> wrote:
> > Since no one else completed, web2py didn't get recognized at the conference
>
> You also forgot to mention that no one else *competed*.
>
> Now it may very well be true that Web2Py is quicker to develop. When I
> look at the code, it does seem shorter and cleaner, but I can still
> program the same thing faster in Django than I can in Web2Py because:
>
> 1. I learned it a while ago first.
> 2. The free Django documentation is much better than the Web2Py one.
>
> Again, this goes back to my point that Web2Py will remain in 2nd place
> until its benefits are larger than its learning curve. Fortunately for
> me, I have not sunk too much time into Django and I have had the time
> to see where it is better than Django. However the majority of people
> are not so fortunate.
>
> Being easy to develop is good, but at the same time it can come with a
> cost. It produces a bunch of unpolished apps that the public sees and
> becomes immediately turned off. Here are a few examples that I found
> on Reddit and Google:
>
> http://www2.un.int/- Broken pictures everywhere. Completely static
> front page. Subpages appear completely static.https://mdp.cti.depaul.edu/personal/default/register- Brokenhttp://yao.appspot.com/- Featureless bloghttp://mmlado.com/- Yet another featureless bloghttp://morrisdecode.com/- Looks like the same theme as Web2Py
> default.http://www.professionalit.com.br/- Completely static page that could

mdipierro

unread,
Jul 17, 2009, 6:13:55 PM7/17/09
to web2py Web Framework


On Jul 17, 4:54 pm, Bottiger <bottig...@gmail.com> wrote:
> > It parses all views and collapses the three structure associated to each action into one byte-code compiled view. that means there is no
>
> parsing and minimal file IO when executing a view in a bytecode
> compiled app.
>
> I don't think this makes a large difference in practice from the
> benchmarks I have seen. In Django the code is automatically compiled
> to a pyc upon running it. These pycs are likely to be cached in memory
> on the OS level because they are not that big. So the difference
> really is between 1 big disk cache lookup and 3 smaller ones. Even if
> this does make a difference, it is arguably not big enough for the
> common programmer to warrant switching. This is why web programming
> has shifted to dynamic languages like Python and Ruby even though
> there are people still doing C++, Java, and C#.

I am not talking about parsing Python files. I am talking about
parsing {{extend 'layout.html'}}{{for i in range(3)}}<h1>{{=i}}</h1>
{{pass}} and turning this into a python file. My benchmarks show a big
difference on complex pages. If you dig on the list they were
published once.

> > Also do not forget the licensing issues. Web2py executes applications and applications (generally) do not import web2py modules (with some exceptions). Instead Django applications import Django modules...
>
> Actually this is not an issue at all. All the 3 major frameworks:
> Pylons, Django, and Rails, do not use GPL (unlike Web2Py). They
> instead use a freer BSD-style license that does not come with the
> definition of "derivatives" which is so often a problem in GPL.

The issue is freer for who. BSD is designed so that a business can
take over a project and close source its derivative work. The "freer"
part of BSD protects these kind of companies, not the actual project
or its original developers.

waTR

unread,
Jul 17, 2009, 6:17:26 PM7/17/09
to web2py Web Framework
I agree with Massimo on the bellow points about framework competition.
> >http://www2.un.int/-Broken pictures everywhere.

mdipierro

unread,
Jul 17, 2009, 6:28:29 PM7/17/09
to web2py-users
done, excellent suggestion
> >http://www2.un.int/-Broken pictures everywhere. Completely static
> > front page. Subpages appear completely static.https://mdp.cti.depaul.edu/personal/default/register-Brokenhttp://yao.appspot.com/-Featureless bloghttp://mmlado.com/-Yet another featureless bloghttp://morrisdecode.com/-Looks like the same theme as Web2Py
> > default.http://www.professionalit.com.br/-Completely static page that could

JohnMc

unread,
Jul 17, 2009, 7:51:34 PM7/17/09
to web2py-users
Hans,

I have already offered to do the screen captures as my bit to the doc
effort. As soon as the marching orders are released I'll work it.

I might suggest one addendum sometime in the future. The book itself
covers what I call the application developers view. That's appropriate
considering the target is envisioned as a teaching tool. But a simple
1-2 pages that would suggest a breadcrumb of the gluon code to review
for deeper insights might be a worthy addition someday.

(Somebody will probably point me to an AlterEgo article that does
this.....)

JohnMc

On Jul 17, 10:32 am, "Hans Donner" <hans.don...@pobox.com> wrote:

viniciusban

unread,
Jul 17, 2009, 7:55:27 PM7/17/09
to web2py-users

On Jul 17, 12:32 pm, "Hans Donner" <hans.don...@pobox.com> wrote:
> i m seeing many posts about the doc needing improvement, and some discusions about it should be done in the wiki that is not there yet.

Which wiki? I could not find a link to it in web2py site.

> However, i see little that people offer documentation or patches that work on the documentation.

Are we supposed to do that?
I didn't know we could do it. Really.


>
> please, can we see more of the latter and less of the first? also, do no point only to massimo that he should provide it.

OK, but reading the contributors page (http://web2py.com/examples/
default/who) I couldn't see any invite to enjoy the team.

Sorry, I'm new to web2py and I don't know what can be done here.


>
> document what is not clear to  you, and after help from this list, complete it and submit it so it can be added to the documentation or examples.

Right, I'll try to do that in English. :-)

To whom must I submit this kind of stuff? What is the formal way?


By the way, Hans, you have the same exact name of one of the most
brilliant adman in Brazil. :-)
He works to Globo TV, the biggest one here.

--
Vinicius Assef.

Bottiger

unread,
Jul 17, 2009, 9:42:07 PM7/17/09
to web2py-users
> Once I again, 1% of the J2EE market share or 10% of the PHP share are both better than 100% of the Django market share.

Well I would say that the people migrating from J2EE and PHP look at
Pylons first. Pylons is more of a bare bones minimalist framework in
the same vein.

Those who wanted a Full-Package framework look towards Django first
and Web2Py second.


On Jul 17, 3:06 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
> >http://www2.un.int/-Broken pictures everywhere.

Bottiger

unread,
Jul 17, 2009, 9:51:32 PM7/17/09
to web2py-users
> The issue is freer for who. BSD is designed so that a business can take over a project and close source its derivative work. The "freer" part of BSD protects these kind of companies, not the actual project or its original developers.

I don't think Web2Py is in danger of that happening. That has happened
to neither Django, Pylons, Rails, or any other web framework that I
know of. Being a web framework means that 99% of the usage is behind a
server and no code is being sold, and thus beyond the GPL's
jursidiction.

However, it does make it difficult for a company to do something like:
build a packaged and compressed one click executable web2py that runs
a single application. These little things lessens the usage you can do
with a framework.

Bottiger

unread,
Jul 17, 2009, 10:03:27 PM7/17/09
to web2py-users
Sorry about making so many replies, I'm still getting used to the
Google Groups forums, but here is another response:

> I am not talking about parsing Python files. I am talking about parsing {{extend 'layout.html'}}{{for i in range(3)}}<h1>{{=i}}</h1> {{pass}} and turning this into a python file. My benchmarks show a big difference on complex pages. If you dig on the list they were published once.

I don't know if Django still does not do that, but I know for sure
that Mako, which is used in Pylons, does. It automatically produces
PYCs in that manner. From the direct comparisons from Django's
templating language and a compiled template engine like Mako (I
couldn't find a benchmark for the Web2Py templating engine anywhere),
the differences were 2:1 at most, and usually closer.

http://markmail.org/message/46opofod6ixwgill

On Jul 17, 3:13 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:

mdipierro

unread,
Jul 17, 2009, 10:10:16 PM7/17/09
to web2py-users
Thanks Vinicius,

we do not have a formal way. Usually people offer to do something on
this list and if considered valuable you have a go. Lots of people
email me patches personally. If the patch is substantial, I include
the name in the who.html page. This is far from a perfect system and
we need to improve it but it has worked so far.

You are welcome to help us anyway you want. Probably nobody will tell
you what to do but there are things we really need:
- improve docstrings
- finish port the epydoc docs to sphinx
- improve the pages layout css
- if somebody reports a bug, try reproduce it to confirm it. perhaps
fix the problem and email me a patch.

Massimo

mdipierro

unread,
Jul 17, 2009, 10:11:10 PM7/17/09
to web2py-users
I know that mako does. Mako is the closest thing to web2py templates

weheh

unread,
Jul 18, 2009, 8:42:54 AM7/18/09
to web2py-users
-----
all the rest of this is smack-dab peachy: I'll remind you of one
thing -
there was a website, guy from NOAA I think it was - that showed all
the
frameworks that claimed to have something; he tried building
something
simple with them and uncovered all the flaws and gotchas and try to
say
"here's what I would (wouldn't) want to build with".... most things
just
took a long time...
-----

Actually, it was a guy from JPL as I recall. In fact, watching his
screencast is exactly what got me started looking at frameworks and
CMSs. One thing led to the next and I found Django. And then I watched
a video of one of Django's developers and he said something like this:
"... Django's templataing language is different from python because
it's made for page designers. Page designers don't write programs and
programmers don't design pages."

That is exactly what lost me for Django. Then I found web2py and the
rest is history.

I agree wholeheartedly with MDP's observation of the 80:20 rule.
However, I find that web2py is an exception. On my first web2py app I
probably used 90-95% of the features of web2py. On the next app, it
will be 100%. Interestingly, my plate will be clean AND my appetite
sated. There is nothing extraneous in web2py that I can discern.

Web2py's niche is that one person of reasonable skill can develop a
sophisticated enterprise web application in minimal time with minimal
effort. This is because of its 3Cs: consistency, completeness, and
conciseness.

Yarko Tymciurak

unread,
Jul 18, 2009, 12:21:03 PM7/18/09
to web...@googlegroups.com
Ach!  Yes that's it - JPL - It was from Sean Kelly who's video starts with him working at NOAA:

http://oodt.jpl.nasa.gov/better-web-app.mov

Yarko Tymciurak

unread,
Jul 18, 2009, 6:33:56 PM7/18/09
to web...@googlegroups.com
If you've never looked at this video, please do - it's about 1/2 hour but worth the time (I just took it in pieces over the afternoon).

From my notes:

In this test, web2py would have came out on top (and understandably so).

Opportunities for web2py (based on observing this video):
  • docs
    • (nothing you didn't already know, but note: django in this version rated fair on docs because there were no books yet, so this is a stage-of-development thing; AND there actually already is good documentation - the book; only it's not free, and you may not be thrilled about that - but it is good, works, and is there; and there's more in the works)
  • Legacy Database Interface
    • we get a lot of request for this, and this video just validates that those are reasonable and valid requests.
    • automatic database reflection is possible (to an extent; proved by and within the limits SQLAlchemy already does, and probably others;)
    • mapping existing db names to DAL naming conventions needed .... this may come up in new DAL; if not, we can probably do this (first) in a contrib package;  the most obvious use is mappind to ID, but any relevant db name needs to map to (e.g. names starting with "_").  This is all do-able, given time.
  • Full text search
    • I have no idea what the state of this is, but know that having it, and having it easy for an application to add / use will be great;
  • Skinning
    • This means not changing the template language elements of a particular site in order to get a different look
    • We can get to this, with some conventions.  Seems 2 rough places:  CSS standard names for basic layout elements will facilitate;   jPolite kind of layout, with look-and-feel for content frames seems to me the second part.
    • I have no idea what this means for Flash/Flex UI's, but suspect this is an entirely separate ballgame.
Everything else, I'm pleased to say, web2py already excels in quite well, thank you - and even more....

Not from this video, but my own observations:
  • C-DAL
    • We have some interplay with Google's big tables from DAL, but the "fit" is partial, less than ideal (though still workable).
    • My personal opinion has been (and is growing in conviction) that a Column oriented DAL, that is abstracting things specific to Big Tables, Cassandra (Facebook), and other prime users (I don't think couchDB falls into this bucket; I'm not sure what Amazon S3 is exactly - is it a column oriented thing too? - it's not advertised)  I see that Apache Hadoop can be hosted on S3, so my suggestion for an initial abstraction experiment is Big Tables, Cassandra, and Hadoop.
    • Over time (and with experience) it will be interesting to see what overlap / abstraction synergy between C-DAL and R-DAL (relational - the current DAL; I just want a way to distinquish them).  It would be nice to have a common DAL with abstractions that fit well in both (sort of what we have now, only less skewed towards relational, and perhaps better cenetered), and ability to move to either R-DAL or C-DAL to get more performance / feature control into either domain.
Lots of fun ahead, even without "many changes" - there are enough to keep things nicely interesting.

And notice:  none of what is layed out here breaks or affects backward compatibility - it's all forward motion, enhance / extend.

- Yarko

Yarko Tymciurak

unread,
Jul 18, 2009, 6:38:21 PM7/18/09
to web...@googlegroups.com
I would add one more thing to my list (below):

On Sat, Jul 18, 2009 at 5:33 PM, Yarko Tymciurak <yar...@gmail.com> wrote:
If you've never looked at this video, please do - it's about 1/2 hour but worth the time (I just took it in pieces over the afternoon).

From my notes:

In this test, web2py would have came out on top (and understandably so).

Opportunities for web2py (based on observing this video):
  • docs
    • (nothing you didn't already know, but note: django in this version rated fair on docs because there were no books yet, so this is a stage-of-development thing; AND there actually already is good documentation - the book; only it's not free, and you may not be thrilled about that - but it is good, works, and is there; and there's more in the works)
  • Legacy Database Interface
    • we get a lot of request for this, and this video just validates that those are reasonable and valid requests.
    • automatic database reflection is possible (to an extent; proved by and within the limits SQLAlchemy already does, and probably others;)
    • mapping existing db names to DAL naming conventions needed .... this may come up in new DAL; if not, we can probably do this (first) in a contrib package;  the most obvious use is mappind to ID, but any relevant db name needs to map to (e.g. names starting with "_").  This is all do-able, given time.
    • I would also like a manual version of migrations... for me the most appealing aspect of making the "automatic" atomic such that it can be manually controlled is the potential for inspecting prior to migration, and with that manual, human discernment, have the potential for rollback.  This seems appealing in the real world.

Hans Donner

unread,
Jul 18, 2009, 6:42:49 PM7/18/09
to web...@googlegroups.com
my wish is that Yarko would be able to express himself in one mail
instead of several (in a short timeframe) :-)

Yarko Tymciurak

unread,
Jul 18, 2009, 6:45:09 PM7/18/09
to web...@googlegroups.com
deal with it..

Hans Donner

unread,
Jul 18, 2009, 6:59:31 PM7/18/09
to web...@googlegroups.com
assuming you saw my [-2:] ?

waTR

unread,
Jul 18, 2009, 7:02:39 PM7/18/09
to web2py-users
Go Yarko!



On Jul 18, 3:33 pm, Yarko Tymciurak <yark...@gmail.com> wrote:
> If you've never looked at this video, please do - it's about 1/2 hour but
> worth the time (I just took it in pieces over the afternoon).
>
> From my notes:
>
> In this test, web2py would have came out on top (and understandably so).
>
> Opportunities for web2py (based on observing this video):
>
>    - docs
>    - (nothing you didn't already know, but note: django in this version
>       rated fair on docs because there were no books yet, so this is a
>       stage-of-development thing; AND there actually already is good
> documentation
>       - the book; only it's not free, and you may not be thrilled
> about that - but
>       it is good, works, and is there; and there's more in the works)
>    - Legacy Database Interface
>       - we get a lot of request for this, and this video just validates that
>       those are reasonable and valid requests.
>       - automatic database reflection is possible (to an extent; proved by
>       and within the limits SQLAlchemy already does, and probably others;)
>       - mapping existing db names to DAL naming conventions needed .... this
>       may come up in new DAL; if not, we can probably do this (first)
> in a contrib
>       package;  the most obvious use is mappind to ID, but any relevant db name
>       needs to map to (e.g. names starting with "_").  This is all
> do-able, given
>       time.
>    - Full text search
>       - I have no idea what the state of this is, but know that having it,
>       and having it easy for an application to add / use will be great;
>    - Skinning
>       - This means not changing the template language elements of a
>       particular site in order to get a different look
>       - We can get to this, with some conventions.  Seems 2 rough places:
>       CSS standard names for basic layout elements will facilitate;
> jPolite kind
>       of layout, with look-and-feel for content frames seems to me the second
>       part.
>       - I have no idea what this means for Flash/Flex UI's, but suspect this
>       is an entirely separate ballgame.
>
> Everything else, I'm pleased to say, web2py already excels in quite well,
> thank you - and even more....
>
> Not from this video, but my own observations:
>
>    - C-DAL
>       - We have some interplay with Google's big tables from DAL, but the
>       "fit" is partial, less than ideal (though still workable).
>       - My personal opinion has been (and is growing in conviction) that a
>       Column oriented DAL, that is abstracting things specific to Big Tables,
>       Cassandra (Facebook), and other prime users (I don't think couchDB falls
>       into this bucket; I'm not sure what Amazon S3 is exactly - is it a column
>       oriented thing too? - it's not advertised)  I see that Apache
> Hadoop can be
>       hosted on S3, so my suggestion for an initial abstraction
> experiment is Big
>       Tables, Cassandra, and Hadoop.
>       - Over time (and with experience) it will be interesting to see what
>       overlap / abstraction synergy between C-DAL and R-DAL (relational - the
>       current DAL; I just want a way to distinquish them).  It would be nice to
>       have a common DAL with abstractions that fit well in both (sort
> of what we
>       have now, only less skewed towards relational, and perhaps better
>       cenetered), and ability to move to either R-DAL or C-DAL to get more
>       performance / feature control into either domain.
>
> Lots of fun ahead, even without "many changes" - there are enough to keep
> things nicely interesting.
>
> And notice:  none of what is layed out here breaks or affects backward
> compatibility - it's all forward motion, enhance / extend.
>
> - Yarko
>
> On Sat, Jul 18, 2009 at 11:21 AM, Yarko Tymciurak <yark...@gmail.com> wrote:
> > Ach!  Yes that's it - JPL - It was from Sean Kelly who's video starts with
> > him working at NOAA:
>
> >http://oodt.jpl.nasa.gov/better-web-app.mov
>

mdipierro

unread,
Jul 18, 2009, 7:37:36 PM7/18/09
to web2py-users
LOL.

On Jul 18, 5:42 pm, Hans Donner <hans.don...@pobox.com> wrote:
> my wish is that Yarko would be able to express himself in one mail
> instead of several (in a short timeframe) :-)
>
> On Sun, Jul 19, 2009 at 00:38, Yarko Tymciurak<yark...@gmail.com> wrote:
> > I would add one more thing to my list (below):
>
> >> On Sat, Jul 18, 2009 at 11:21 AM, Yarko Tymciurak <yark...@gmail.com>
> >> wrote:
>
> >>> Ach!  Yes that's it - JPL - It was from Sean Kelly who's video starts
> >>> with him working at NOAA:
>
> >>>http://oodt.jpl.nasa.gov/better-web-app.mov
>
> >>> On Sat, Jul 18, 2009 at 7:42 AM, weheh <richard_gor...@verizon.net>

DenesL

unread,
Jul 19, 2009, 5:37:35 PM7/19/09
to web2py-users
Thanks for the link Yarko.
It feels like time traveling and it does show how we got to where we
are today.

Yarko Tymciurak

unread,
Jul 19, 2009, 6:07:54 PM7/19/09
to web...@googlegroups.com
Note:  Massimo was surprised by this presentation / video when someone (on list, I believe) pointed it out.

Massimo was already in this direction, so by-and-large, it was a pleasant surprise.   Independently arrived at same conclusion - that sort of thing.

The notes I made about "opportunities" are things we generally know about (so they are reminders), and - as Massimo has pointed out - do not involve (e.g. skins) Web2py core (but would be really useful to have defined, even distrib. w/ web2py).

JohnMc

unread,
Jul 20, 2009, 6:44:52 PM7/20/09
to web2py-users
Yarko,

Liked your list.

As to schemeless key:value databases, an interesting one is redis.
(http://code.google.com/p/redis/) Has built in persistence, is very
fast, extensions to support primitive lists. Have it installed on a
box as an experiment.



On Jul 19, 5:07 pm, Yarko Tymciurak <yark...@gmail.com> wrote:
> Note:  Massimo was surprised by this presentation / video when someone (on
> list, I believe) pointed it out.
>
> Massimo was already in this direction, so by-and-large, it was a pleasant
> surprise.   Independently arrived at same conclusion - that sort of thing.
>
> The notes I made about "opportunities" are things we generally know about
> (so they are reminders), and - as Massimo has pointed out - do not involve
> (e.g. skins) Web2py core (but would be really useful to have defined, even
> distrib. w/ web2py).
>

Bottiger

unread,
Jul 20, 2009, 9:16:35 PM7/20/09
to web2py-users
Redis requires the entire database to fit in memory.

I've done a comparison a couple weeks ago between all of the free
distributed databases and found that mongodb and couchdb are the most
advanced and with the least quirks and requirements.

Yarko Tymciurak

unread,
Jul 20, 2009, 11:35:49 PM7/20/09
to web...@googlegroups.com
from a recent couchDB presentation at ChiPy, mention was about stability and "non-existent security / access controls...."

Not sure the extent, but this put it off my radar for my list (but I'm happy for any evidence / additional info to change that)

mdipierro

unread,
Jul 21, 2009, 9:45:43 AM7/21/09
to web2py-users
I would not be concerned about security too much because it can be
achieved by blocking access. I am not convinced it will help the
majority of our users, perhaps 1% of them or less. Anyway, we should
support it in the new DAL.

Massimo

On Jul 20, 10:35 pm, Yarko Tymciurak <yark...@gmail.com> wrote:
> from a recent couchDB presentation at ChiPy, mention was about stability and
> "non-existent security / access controls...."
>
> Not sure the extent, but this put it off my radar for my list (but I'm happy
> for any evidence / additional info to change that)
>

Yarko Tymciurak

unread,
Jul 21, 2009, 1:40:46 PM7/21/09
to web...@googlegroups.com
my other (somewhat uninformed) concern about adding couchDB to a BigTable/Column-oriented DAL was this:
http://en.wikipedia.org/wiki/CouchDB

saying couch is not a column oriented DB.  If it maps resonably well into a C-DAL abstraction thought, it should be ok.  Just something to watch.

Bottiger

unread,
Jul 21, 2009, 2:51:19 PM7/21/09
to web2py-users
Do you mean that we will support couchdb or that the DAL will allow
support?

I've talked to the CouchDB developers and they have said that its
currently in Alpha and they will probably be making changes that break
backwards compatibility. MongoDB on the other hand, is nearing 1.0
release by the summer and is now being used by sourceforge.

Hans Donner

unread,
Jul 21, 2009, 3:05:21 PM7/21/09
to web...@googlegroups.com
The new DAL should be flexible enough to build the support.

JohnMc

unread,
Jul 21, 2009, 3:11:04 PM7/21/09
to web2py-users
Agreed that it does require it fit in memory just as memcached. But
being able to handle 150 page views per sec in a 1mb memory space I
don't know if for most that would be much of a concern, for most
websites web2py would be brought to bear on.

JohnMc

Yarko Tymciurak

unread,
Jul 21, 2009, 3:15:20 PM7/21/09
to web...@googlegroups.com
On Tue, Jul 21, 2009 at 1:51 PM, Bottiger <bott...@gmail.com> wrote:

Do you mean that we will support couchdb or that the DAL will allow
support?

This discussion should probably be moved to the web2py developers group:

I suggested that a clean, column-oriented DAL would be an interesting experiment / excercise.
This was mostly because of discussions around GAE, how big tables api offers things that didn't fit well into DAL abstraction, etc.

Rather than make something GAE specific, I suggested - in order to force discovering a "useful" abstraction - to select several column stores which were most likely to have use (e.g. testors, interested, knowledgable parties).

Support seems highly premature in the discussion; the experiment should show something first, and then - to me an interesting - a review of the overlap, lack or, possibility for better overlap between (what I dubbed, for 'column') C-DAL and current, or R-DAL would make for an interesting meeting / session.

From this motivation, I hesitated to throw couchDB in the mix, as I think its different enough that it would potentially muddy the discovery of a crisp abstraction (I would be more for investigating the abstraction by adding something like couchDB to a second stage of the experiment - extending C-DAL, and testing how the abstraction holds up).

Anyway, those are my thoughts and motivations.


I've talked to the CouchDB developers and they have said that its
currently in Alpha and they will probably be making changes that break
backwards compatibility.

the Alpha state (which I also heard @ ChiPy) was another reason my gut is not for throwing couchDB in the mix for first phase of the experiment... but those who do it should decide.  I think the more important thing is creating / finding / working out a "correct" abstraction for column-stores, and then comparing w/ DAL and seeing what new insights that may give us.


 
MongoDB on the other hand, is nearing 1.0
release by the summer and is now being used by sourceforge.

This, like couchDB, is classified as another "document oriented" db...  just on that, I would hold those off to see how they play w/ column-oriented abstraction;

Remember, the thing I would like to see is development of a clean starting abstraction.  Mixing storage types doesn't seem like a good idea (at the start) for that.
Maybe a separate dal for Mongo/couch ... and later merge w/ column... but hat feels too much like competition, and not enough like investigation.... I think investigation is what will yield interesting results.
 
- Yarko

Bottiger

unread,
Jul 21, 2009, 3:38:02 PM7/21/09
to web2py-users
"This discussion should probably be moved to the web2py developers
group:"

I don't have access to the web2py developers group. But maybe that is
the point. See why I don't like the separation?

I am no novice to open source. I know that not every patch will be
accepted, but every project has varying levels of acceptance. Adding
document-database support to the DAL is not a small change, and I
would not interested in sinking the required large amount of time into
unless it would be accepted.This is why I first contacted Massimo.

Recently, I had been researching distributed databases for quite some
time. While I am not a relational algebra expert, I believe that
document based databases such as Amazon SimpleDB/MongoDB/CouchDB can
be reasonably translated to and from a relational model. In fact, the
document based model is closer to an object oriented design since it
is schema free.

The map/reduce paradigm also allows you to scale complex queries to
multiple cores and machines. For everything except banking, document
based stores are the future.


On Jul 21, 12:15 pm, Yarko Tymciurak <yark...@gmail.com> wrote:

Yarko Tymciurak

unread,
Jul 21, 2009, 3:51:59 PM7/21/09
to web...@googlegroups.com
On Tue, Jul 21, 2009 at 2:38 PM, Bottiger <bott...@gmail.com> wrote:

"This discussion should probably be moved to the web2py developers
group:"

I don't have access to the web2py developers group. But maybe that is
the point. See why I don't like the separation?

Just ask for access to the developers group.
 

I am no novice to open source. I know that not every patch will be
accepted, but every project has varying levels of acceptance. Adding
document-database support to the DAL is not a small change, and I
would not interested in sinking the required large amount of time into
unless it would be accepted.This is why I first contacted Massimo.

Recently, I had been researching distributed databases for quite some
time. While I am not a relational algebra expert, I believe that
document based databases such as Amazon SimpleDB/MongoDB/CouchDB can
be reasonably translated to and from a relational model. In fact, the
document based model is closer to an object oriented design since it
is schema free.

Yes, I am just now playing w/ mongo (thanks to your post! ;-)

Since you've been researching this quite some time, could you give a really short (1 or 2 paragraph) summary comparing doc-oriented vs. column oriented, and any intersting implicatinos, performance, unknowns? 
 
Thanks!

- Yarko

Yarko Tymciurak

unread,
Jul 21, 2009, 4:37:59 PM7/21/09
to web...@googlegroups.com
more on playing w/ MongoDB -

looking at mongokit (which tries to make an orm-type adapter to mongo),  I think this might be a relatively easy basis for a (?) generic document-type-persistence-store...

I don't know how long /far I'll play with it (that it's been this easy so far is a good sign).

Bottiger

unread,
Jul 21, 2009, 5:59:56 PM7/21/09
to web2py-users
"Just ask for access to the developers group."

Massimo has said though that only people who have actually contributed
code to Web2Py and not people that are merely planning to may join.

As for doc-oriented vs column oriented, they are more similar than
they are different. Column oriented databases still have a relation-
like query system while CouchDB has moved to a map/reduce paradigm
which can allow for complex queries. MongoDB attempts to include some
of the more traditional relational queries in JSON format with a plan
for fully paralllel map/reduce in the future (they have single server
map/reduce atm).

The other distinction is that document based databases do not have a
schema so you can add and remove attributes at any time like a Python
object.

Performance benchmarks have not really been done between column vs
document databases. However, MongoDB benchmarks have shown it to be
comparable or faster to MySQL. CouchDB is not a performer because it
uses the heavy HTTP protocol to communicate. However the main feature
of these databases is not performance, but scalability.

The most important characteristic compared to a relational DB, complex
transactions and joins are unavailable due to the distributed nature.
This of course is unavoidable unless you have a global semaphore,
which then becomes equivalent to a single relational database with
multiple slaves. In fact, this is the same problem you get when you
start partitioning relational databases. There is no way around it
when you need to scale.

On Jul 21, 12:51 pm, Yarko Tymciurak <yark...@gmail.com> wrote:

Fran

unread,
Jul 21, 2009, 6:05:49 PM7/21/09
to web2py-users
On Jul 21, 10:59 pm, Bottiger <bottig...@gmail.com> wrote:
>> "Just ask for access to the developers group."
> Massimo has said though that only people who have actually contributed
> code to Web2Py and not people that are merely planning to may join.

He also said:
"I have no strong object to the idea of removing every prerequisite
from admission to web2py-developers."

F

Yarko Tymciurak

unread,
Jul 21, 2009, 6:16:40 PM7/21/09
to web...@googlegroups.com
I suggest you just ask Massimo;  I think you would have backing.

Anyway, mongo (I'm still playing, reading) is looking pretty interesting, and I've only been playing about 2 hours...

Thanks for the discussions on this...
Reply all
Reply to author
Forward
0 new messages