web2py EuroPython 2013 Talk

[Massimo DiPierro]

Thanks to the EuroPython organizers. It was an excellent conference. I attended at the last moment and I had the opportunity to give a talk in the open space. Here it is:

http://www.youtube.com/watch?v=LsLgZUGM3kg

[Ovidio Marinho]

 very good ,you have the slides?

      

         Ovidio Marinho Falcao Neto

                  ITJP.NET.BR
             ovid...@gmail.com 

               83   8826 9088 - Oi

               83   9336 3782 - Claro
                        Brasil

              

2013/7/6 Massimo DiPierro <massimo....@gmail.com>

Thanks to the EuroPython organizers. It was an excellent conference. I attended at the last moment and I had the opportunity to give a talk in the open space. Here it is:

http://www.youtube.com/watch?v=LsLgZUGM3kg

--
 
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Massimo Di Pierro]

This is the talk from PyCon Brazil:

https://dl.dropboxusercontent.com/u/18065445/Tmp/PyConBr-2012.pdf

At EuroPycon I only used a few slides which are a subset of that talk. The rest was an interactive demo.

Massimo

[Alan Etkin]

At EuroPycon I only used a few slides which are a subset of that talk. The rest was an interactive demo.

Thanks. Much shorter than PyConUS or PyConAr talks, but still worth seeing. I liked the interactive shell script (or wathever you've used to create the app).

[shapova...@gmail.com]

Nice talk! Worth watching even for those who are already familiar with Web2Py. 

It was interesting to hear your opinion about not so big developers community as a major problem. But we are here to solve it, after all :)

Thanks and regards from Moldova (which is in Europe, btw :)

[Massimo Di Pierro]

The interactive shell is this one: https://github.com/mdipierro/kryten

I posted it after PyCon Argentina.

Massimo

[Arnon Marcus]

Awesome lightning-talk...

They should have given you much more time...

I actually would have opted for a slightly different prioritization for what to show:

1. For me, as a developer, the ticketing system is one of the most useful and impressive piece of work in the whole of web2py - I think for a conference of developers, it would really spice things up for many people who don't know about it - having such well-designed introspection in production, has made our lives much easier - it's like a built-in auto-error-logging instrumentation that records the bugs in production-mode- and with a really well-done interface.

2. I really liked the all-in-browser show-off of the old web2py videos... The text in the software you where using for inputting the text, is was really hard to read (dark-blue text-color on a black-background?? and with zero-syntax-highlighting??) I mean, I get the incentive to show that you can use any text-editor so people won't get the wrong impression - but I think it would have been much better to show it using the web-based text editor, as it would have been more readable. If you are going to use an external text-editor, might as well use a "more" readable alternative than the web-based one, but surely not a less-readable one. The text is going  so fast, and shows up for such short a time, that nobody could possibly manage to read it unless it extremely readable. I get the incentive to use something to type for you, especially for live-demoing on such a tight schedule - but to me it overall seems like a bad choice, as the     bad readability and super-fast-paste  makes the whole thing deletes the purpose. If it was me, I would have kept "most" of the lecture in the web-browser (with copy-pasting of code to avoid live-coding errors and keep it fast-pace), and show a short-demonstration using an external text-editor, right before the end (preferably something slim and readable like sublime-text).

Alternatively, the beta-version of PyCharm 3.0 could have been tested-out to showcase it's support, it's also very readable...

3. The constant scrolling of the web-pages was kinda distracting - Either the web-browser was set to an annoying re-zooming-in thing, or the layout.html is in need of a little layout-tweak (condensing things a bit) to make things fit better into the page by default - even on pages with data in them.

Anyways, all-in-all a good talk - that's just my personal feedback.

[Arnon Marcus]

BTW, is it really true that web2py is twice as slower than django nowadays?

How can that be?

Didn't it used to be twice as fast?

When I first evaluated it 3 years ago, it was by-far the fastest - what changed?

You said that one of the core principles of accepting changes to web2py, is that they should always make it run faster - never slower. Has that principle been broken?

And what about the whole ORM-vs-DAL fiasco? Didn't you guys always say that a DAL is always faster than an ORM? Given that it makes sense, how come Django is faster? Shouldn't it's ORM make it slower?

And as for executing-vs-importing - didn't you say before that it should be a non-issue in terms of performance? What changed?

[Massimo Di Pierro]

It is true but not an issue. Django is faster only in hello world examples because does not perform as many header validation/conversions as web2py does and because you cannot turn off sessions in web2py. As soon as one uses templates, web2py is faster. If you use databases the speed is about the same because that becomes the bottle neck.

Massimo

[Arnon Marcus]

I see.
In that case, I think it would be advisable to note that in presentations, as peope might get the wrong impression...

> --
>  
> ---
> You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/kWwIuQjeatQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to web2py+un...@googlegroups.com.

[Massimo Di Pierro]

I agree. I will do.

[Vinicius Assef]

Massimo, how about you writing an article about this subject and share with us?

So, this could be spread.

On Thu, Jul 11, 2013 at 2:12 PM, Massimo Di Pierro

> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an

[Massimo Di Pierro]

Because I think it is pointless for various reasons:

1) I am biased and people outside the community would not trust it

2) Code changes so it would become obsolete quick

3) One can produce benchmarks to produce almost any result one wants

4) People who are concerned about 2x factors in speed are not web2py audience. The point of web2py is security and speed of development. For example we can make web2py more than 2x faster moving sessions management to the app level (like Flask) and not creating the request.env object. Yet we do not do it. What matters is not speed but scalability and web2py scales no better or worse than any other major Python framework.

5) Every time I published any comparison between web2py and other frameworks in the past, somebody got upset and attacked us. Not worth it.

[Ovidio Marinho]

Web2py can not be the fastest, but it is the most simple and functional for everyone.

      

         Ovidio Marinho Falcao Neto

                  ITJP.NET.BR
             ovid...@gmail.com 

               83   8826 9088 - Oi

               83   9336 3782 - Claro
                        Brasil

              

2013/7/12 Massimo Di Pierro <massimo....@gmail.com>

[Arnon Marcus]

I seem to remember vividly seeing a benchmark about 4 years ago that showed that web2py was indeed the fastest.

Either that benchmark was misleading, or things have changed.

In either case, it would still remain important to explain to the public the reasons for the trade-offs that were chosen. If "speed-of-development" or "security" had to come with a performance-penalty, and still it was a conscious-choice to for-go with performance, than this trade-off choice should be out there in the open and explicit. It's a form of expectation-management and quality public-relation.

If people out-there understand that "hello-world" benchmarks are meaningless for real-world use-cases, as the bottleneck always end-up being the database whenever one is used (which is practically 100% of real-world use-cases), they would be more apt to trying-out web2py, and have less irrelevant/erroneous-biases against it.

And if the DAL is really faster than an ORM, than the bottleneck should theoretically be less severe in web2py, since less queries would be executed to the database. Meaning, that any performance-loss of non-database-related executions, should be grossly overshadows and nullified by virtue of having fewer database accesses. Meaning, that for real-world production use - web2py ultimately should be faster for database-intensive applications. 

[Massimo Di Pierro]

It really depends on what one compares.

I do not think we ever published benchmarks claiming web2py is faster than other frameworks. If one want speed raw WSGI (perhaps with gunicorn or gevent) without templates and without databases beats every framework. The more a framework does, the slower it gets. I do claim web2py is better than other frameworks because it is easier to use and because it does more for you than other frameworks do.

According to my tests:

- the DAL is faster in generating SQL queries than ORM based on metaclasses but this is pointless since all the time is in DB IO, not in generating queries. That time is the same. Django in some cases (postgresql) has the ability of returning an iterators while web2py always returns the data. The Django approach is better from a memory management way and faster is only plan to loop over a subset of the selected records. If you plan to loop over all of the records selected the web2py approach is better.

- The web2py template language is faster than the Django templates because uses the python interpreter and not its own. Yet it depends on the details of the view page.

- Apart for templates and DAL, the execution of models and controllers is slower in web2py. In Django models are executed only once (at startup) while in web2py at every request. We do this for a reason. It is a small performance penalty to pay for a greater flexibility (multi-app support, hot plug and play, no library conflicts, etc.). This only hits in hello world kind of tests and in the case of very very long models (that is why some of use prefer to put code in modules and not in models) and then import the modules in models.

Massimo

[Michele Comitini]

I don't care about ORM performance which is worse both in term of bare performance and in development speed, but using large databases and doing profiling I find that most of the time that is "spent on DB IO" is ispent inside python code unless executesql is used.  There is still a lot of room for improvement even if a lot has been done already.

Our target should be that in the future:

rows = db(query).select(...)

vs.

sql=db(query)._select(...)

rows = db.exequtesql(sql)

have a not a difference of one order of magnitude anymore.

2013/7/13 Massimo Di Pierro <massimo....@gmail.com>

[Massimo Di Pierro]

The difference between the two is that in former case the output returned by the DB is normalized (each record is a Row object) while in the executesql case the output is not normalized (each record is a tuple containing data in the database specific representation). It does take time to loops over a list of tuples and convert it into into a list (Rows) or dictionaries (Row) of normalized objects. 

You can in fact do:

rows = db(....).select(cacheable=True) # some speedup

rows = db(....).select(processor=lambda rows,*a,**b: rows) # as fast as executesql but no parsing