price for assignRef

[ykorotia]

have somebody tested cpu/memory price for usage of function assignRef,
instead of direct assign?

I mean
$v->assignRef('blabla-bla-bla', $mega_object); <-- by the way, it
looks like we will have epic fail
$v->blabla = $mega_object;

if you didn't test it I could do it as far as I have performance
"question" to this function

public function assignRef($key, &$val)
{
if (is_string($key) && substr($key, 0, 1) != '_')
{
$this->$key = &$val;
return true;
}

return false;
}

p.s.
going to test it anyway, got half an hour of time

[ykorotia]

as I thought, here's the results

Time Analysis (sec)
$v->assignRef 100000 times: 3.79494882 (96.94%)
$v->blablaN 100000 times: 0.11972213 (3.06%)
Total time: 3.91482091

can you confirm same results?



test environment is IIS7, PHP 5.3.3
used http://simplyeng.com/en/simple-php-time-testing-unit/

test file below
<?php
error_reporting (E_ALL); require_once "time-testing.php";

class mega {
private $p; public $p2; public $p3;
function __construct($p = 1.0000002, $p2 = 'sssssssssssss<br/>', $p3
= null)
{ $this->p = $p; $this->p2 = $p2; $this->p3 = new stdClass(); }
}

class view {

public function assignRef($key, &$val)
{
if (is_string($key) && substr($key, 0, 1) != '_')
{
$this->$key = &$val;
return true;
}

return false;
}
}

$index = 100000; $key = 'blabla';
$megaobject = new mega();


ctk_start();
$v = new view();
for ($i = 0; $i < $index; $i++) {
$v->assignRef($key+$i, $megaobject);
}
unset ($v);
ctk_marktime('$v->assignRef '.$index.' times');

$v = new view();
for ($i = 0; $i < $index; $i++) {
$v->{$key+$i} = $megaobject;
}
unset ($v);
ctk_marktime('$v->blablaN '.$index.' times');


echo ctk_report();

[ykorotia]

oops... mistake +

[ykorotia]

but anyway it shows the same

$v->{$key.$i} = $megaobject;

[Peter van Westen]

Just a side note:
It is much faster (about 20% faster) to use:
$key['0'] != '_'
instead of:

substr($key, 0, 1) != '_'

[Yurii Korotia]

default
$v->assignRef 1000000 times: 39.24994683 (96.62%)
$v->blablaN 1000000 times: 1.37450004 (3.38%)
Total time: 40.62463403

with your changes
$v->assignRef 1000000 times: 39.27207518 (96.63%)
$v->blablaN 1000000 times: 1.36982083 (3.37%)
Total time: 40.64204812

with removed is_string($key)
$v->assignRef 1000000 times: 22.68345094 (94.41%)
$v->blablaN 1000000 times: 1.34228897 (5.59%)
Total time: 24.02593088

without if statement
$v->assignRef 1000000 times: 21.90689898 (94.40%)
$v->blablaN 1000000 times: 1.29837990 (5.60%)
Total time: 23.20543194

empty function, void on return
$v->assignRef 1000000 times: 19.57859182 (93.62%)
$v->blablaN 1000000 times: 1.33358312 (6.38%)
Total time: 20.91231489

how have you achieved -20% ? I'm using php 5.3.6+IIS7.5-fast-cgi

my point was to avoid this method at all as it is useless.

[Nicholas K. Dionysopoulos]

I watch this thread with dismay. You are wasting too much effort on something pointless. Let me be the devil's advocate.

How many times are you going to call this function? Even on a bastardly huge, inefficiently written component, you won't call that method over 100 times. Based on your benchmark, 100 executions take 0.00003924994683 seconds, or 0.039 msec, or 39 nanoseconds. The alternative would take about 2 nanoseconds. So, you would benefit a whooping... 37 nanoseconds?!

Note: If you want to pass a crapload of data from the model to the view you should pass one megaobject, which is a cheap operation (less than one nanosecod), instead of dozens of small bits of information. If you are not doing that, you are doing something utterly wrong, a textbook case of PEBKAC.

Get real, people. I don't think that ANYONE would notice an average of 37 nanoseconds improvement per page load. Optimisation is good, but please try to optimise the code that matters, i.e. the code which runs thousands of times in a tight loop or which takes a significant ( > 100 msec ) time to execute in a typical pageload.

Back in the day, I used to write software in Turbo Pascal for 80286 computers. Remember them? They had a clock at 12MHz (in "turbo mode") and were subscalar, i.e. they would execute on average about one instruction every 5 cycles IIRC. I would of course go to great lengths to optimise the execution time of tight loops, usually resorting to writing them in 80286 Assembly. Five CPU cycles in a loop called 200,000 times could make or break a programme. I would never do that kind of optimisation, however, for something as trivial as parsing input data which would be called once. It'd be overkill and a waste of my time (which I had a terrible lack of, as I was still a high-school student). Nowadays we have machines with CPUs clocked at 2.4GHz and which are superscalar, i.e. the execute more than one instruction per CPU cycle, ergo about a million times faster, or even more. And we are still talking about optimising trivialities which only get called a couple of times in the code? Meh...

-- 
Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: http://www.AkeebaBackup.com

Blog: http://www.dionysopoulos.me/blog

--
You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.
To post to this group, send an email to joomla-de...@googlegroups.com.
To unsubscribe from this group, send email to joomla-dev-gene...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/joomla-dev-general?hl=en-GB.

[Yurii Korotia]

php is pretty optimized in math but still you may see random
difference:

1000000 / 2 = 500000: 0.39716697 (50.41%)
1000000 >> 2 = 500000: 0.39053512 (49.57%)
Total time: 0.78782296

we just lost 0.007 sec in trivial operation. I guess website will do
this this kind of job more often. I don't talk to jump from objects to
procedures (which has near +40% performance against objects) and use
bit operations, but why to use function that is initially almost
useless and gives +96% overhead to addition?

p.s.
yes, you can bye new hardware all the time when you get +1000 users or
your can use optimized technique and bye new hardware after +3000
users.

p.p.s.
you have some websites on one host. let's say current second visits
are equal to 10.000 users. You have used views and assigned 3 objects
in each view. you have 30k execution of type $v->assignRef()
$v->assignRef 30000 times: 1.14100289 (96.98%)
$v->blablaN 30000 times: 0.03539205 (3.01%)
Total time: 1.17651892

1 second of processor time.

I don't talk about caching, not applicable

thanks for comment

On 18 July, 13:27, "Nicholas K. Dionysopoulos" <nikosd...@gmail.com>
wrote:

> > To post to this group, send an email to joomla-de...@googlegroups.com (mailto:joomla-de...@googlegroups.com).
> > To unsubscribe from this group, send email to joomla-dev-gene...@googlegroups.com (mailto:joomla-dev-gene...@googlegroups.com).

[Yurii Korotia]

oops again: 1000000 >> 1

[Nicholas K. Dionysopoulos]

Yuril,

Please provide me a real world example where assign() and assignRef() are used in excess of 10,000 times so that they have any significant impact on page load.

I did not say that there is no speed difference. I said that 37 nanoseconds (37 millionths of a second) make no difference. Assuming that your site traffic exploded tenfold, which would also introduce a higher system load and degrade the performance by an order of magnitude, the speed difference would still be 0.37 milliseconds. On a server which is melting down. On a site whose pages now take 20 seconds to load. There is not a cat's chance in hell of anyone noticing this delay.

So, by all means, please do find something more important to optimise. I would look at the database, table and model implementation for areas where PHP could be improved a little - they are called the most per page load. Moreover, if I had a component of my own I would do this:

- Optimise my SQL (biggest bottleneck)

- Run my code against XDebug and find the tight loops which consume most of the time (second biggest bottleneck)

- Rework some of my code to make fewer redundant calls and use static variables for immutable data which needs be initialised once (the third biggest bottleneck)

That's how you optimise code, not by chasing down 37 nanoeconds per page load.

Finally, your math is awfully wrong (not that it matters, but anyway). If a function takes 39 nanoseconds t ocomplete and a competitive function takes only 2 nanoseconds, the speed ratio is 39/2 = 19.5 times as much, or 18.5 times (19.5 - 1) slower or 1850% slower. It's not 96% slower. I would also argue that your analysis is completely wrong, as it doesn't try to take a statistically significant sample and extrapolate the mean value and standard deviation and repeat that test under different load conditions on different architectures (if the standard deviation is comparable to the mean value you have a statistical error or a measurement error). But given the stupefyingly minuscule speed difference in absolute numbers, that kind of analysis would also be an overkill.

In other words: solve a problem which exists, do not try to create a problem that's not there just so you can solve it. The former is called "writing software", the latter is called "academic research".

-- 
Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: http://www.AkeebaBackup.com

Blog: http://www.dionysopoulos.me/blog

On Monday, 18 July 2011 at 14:17, Yurii Korotia wrote:

oops again: 1000000 >> 1

--
You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.

To post to this group, send an email to joomla-de...@googlegroups.com.
To unsubscribe from this group, send email to joomla-dev-gene...@googlegroups.com.

[Yurii Korotia]

fine then. I didn't say 96% slower, but +96% overhead, where +4% gives
direct assign.
about tracing - it's pretty much the same, same with mysql logging. If
you make loop testing near 1kk it makes your result less floating, not
because it hasn't random, but because time difference becomes higher.

about xdebug, here we have sentence like "don't shoot fly with cannon"

and yes, you're right, I'm going to shut up, will return with
something other

have a nice day

[Andrew Eddie]

The suggested changes are small for low load sites, but could possibly
add up in very high traffic sites. Whatever the case, the assignref
method is a hangover of the php 4 days and we should be standardising
on not using it. Peter's suggestion about substr is also good. Each
change in isolation is small, maybe even insignificant, but if people
are willing to do the work to provide patches, I wouldn't stop them.
These are quick, good coding practice wins.

Regards
Andrew Eddie

[Nicholas Dionysopoulos]

With all due respect, Andrew, if anyone is running a high traffic website without APC and a transparent CDN he is either stupid or just plain dangerous. When Gazetta.gr ran on Joomla! it could server several millions of unique visitors with a single server. I have not heard of a more heavy traffic website.

Besides, if your are looking for performance bottlenecks, please consider how JModel and, as a result, com_content get to provide pagination. I have seen beefy servers being reduced to a pile of junk just because com_content was trying to paginate the front page of the site. The site only had a few dozen thousand articles.

Well, if you are still looking for a way to save 39 nanoseconds per page load, I would suggest removing comments and whitespace from all PHP files. You will save about 50 nanoseconds doing that. Just because you can doesn't mean you should ;)

Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: https://www.akeebabackup.com
Blog: http://www.dionysopoulos.me/blog

Sent from my iPad. Please excuse my brevity.

[Yurii Korotia]

well, Nicholas,
it was a question about rational usage and benefits. I didn't
understand sense of this method, but Andrew had clarified it somehow.
Your sarcasm is well understandable and is accepted.

pagination is provided by jpagination class. You may not use it at
all. This class provides only html and state logic, while sql requests
is on you. Can you explain what's the trouble? 'Cause I still didn't
touch everything.

p.s.
about removing comments and tabs etc. Zend optimizer is what it is.
And I'm sure, someone uses this technique and is happy about x2
increase in performance.


found one bug with compatibility in mootools, tooltips don't work..
got to go

[Nicholas K. Dionysopoulos]

Hi Yuril,

Yeah, I understand that you guys started this conversation with good intentions, but it strikes me as odd discussing a 40 nanosecond improvement when performance sucks big time in other areas of the CMS ;)

OK, regarding the pagination, let's see how it works.

The JPagination object is generated by the model. The actual source of the object is JModelList::getPagination(). This method calls JModelList::getTotal() to get the total number of pages. This method internally fetches the list query using JModelList::_getListQuery() and then calls JModel::_getListCount() using that query. The latter runs the SELECT query and merely calls JDatabase::getNumRows() to get the total rows.

If you have thousand of articles in a category (typical news site when nobody is archiving old content) you end up SELECTing thousands of rows without a query LIMIT. MySQL dutifully proceeds to load the whole freaking lot to memory which a. makes things dead slow and b. consumes all your RAM. You site is now self-p0wned.

Solution: (I won't claim to have invented this; K2 and Nooku are doing that for a very long time). Instead of running a SELECT * SQL statement, run a SELECT COUNT(*) query, or whatever is the proper count-only query with the current DBO. I'd assume that having JDatabaseQuery would make that trivial.

On a heavy traffic website with tons of contents (typical news site, the very site which begs for a CMS) the speed savings would be around 10 seconds and the memory savings around 2.4Gb. If you don't believe me, please ask my friend Tim. I advised him to move to K2 because com_content is just "not there" when it comes to large scale content management.

I hope that now everyone understands why I face the <40 nanoseconds speed improvement with a great dose of irony. It's like the ship is sinking and we're getting ourselves busy with cleaning the toilets and what is the best method to do so.

Best regards,

-- 

Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: http://www.AkeebaBackup.com

Blog: http://www.dionysopoulos.me/blog

[Denis Ryabov]

I wrote about this issue in 2007 (http://forum.joomla.org/viewtopic.php?p=1473825#p1473825), but we still there.

19.07.2011, 11:13, "Nicholas K. Dionysopoulos" <niko...@gmail.com>:

If you have thousand of articles in a category (typical news site when nobody is archiving old content) you end up SELECTing thousands of rows without a query LIMIT. MySQL dutifully proceeds to load the whole freaking lot to memory which a. makes things dead slow and b. consumes all your RAM. You site is now self-p0wned.

-------
Best regards,

Denis Ryabov 

[Yurii Korotia]

currently I stuck with 1.5.23. Digging if upgrade is possible, not
joking, pure truth.

In 1.5 we have only JModel. So I see things like
function some-function( $query ... )
so, we can set what-ever we want.

let's see in 1.7 (from trunk)
JModelList <-- JModel, where JModel is pretty much the same, just has
more proficient coding, like protected methods (stil cannot get why
$__state_set is protected not private, but anyway)

I see this :
you should use getItems() to get items with your limit. what's up?


** but I really lack this JModelList as in 1.5 you have to implement
it by your own, and it is going to have same look at JModelList.
REALLY. Just hope for successful upgrade to 1.7 someday

[Yurii Korotia]

JModelList --> JModel

gg, excuses

[Andrew Eddie]

Nic, I think the main point here is the guy is willing to make some
improvements which are logical, you and I don't have to do it so I
think the best response is not to berate, but to say go ahead, submit
a patch. I don't see why we have to put up a big red stop sign to
this kind of suggestion. Just maybe the guy will then feel good
enough about his experience to go analyse our slow queries and such as
a next step :)

The pagination query I've already tested in practice, only use
COUNT(<pk name>) not COUNT(*), and it works a treat (got a >10 fold
improvement for one case I was working on - crazy). Someone is
already discussing that somewhere IIRC. Replacing some joins with well
crafted sub-queries also helps in some cases. Whatever the case, big
or small improvements - everyone's a winner. But you know, I'd
actually start with getting rid of assign and assignRef myself because
we just don't need the stupid things anymore and it makes for much
cleaner code :P

Regards,
Andrew Eddie

On Jul 19, 7:39 am, Nicholas Dionysopoulos <nikosd...@gmail.com>
wrote:

> With all due respect, Andrew, if anyone is running a high traffic website without APC and a transparent CDN he is either stupid or just plain dangerous. When Gazetta.gr ran on Joomla! it could server several millions of unique visitors with a single server. I have not heard of a more heavy traffic website.
>
> Besides, if your are looking for performance bottlenecks, please consider how JModel and, as a result, com_content get to provide pagination. I have seen beefy servers being reduced to a pile of junk just because com_content was trying to paginate the front page of the site. The site only had a few dozen thousand articles.
>
> Well, if you are still looking for a way to save 39 nanoseconds per page load, I would suggest removing comments and whitespace from all PHP files. You will save about 50 nanoseconds doing that. Just because you can doesn't mean you should ;)
>
> Nicholas K. Dionysopoulos
> Lead Developer, AkeebaBackup.com
> Web:https://www.akeebabackup.com
> Blog:http://www.dionysopoulos.me/blog
>
> Sent from my iPad. Please excuse my brevity.
>

[Nicholas K. Dionysopoulos]

Hi Andrew,

OK, I understand that. But instead of "fixing" assign/assignRef why not use magic __set() and __get() methods to store such values to an internal array or a state object? It would make more sense. It would also change the variable assignment paradigm from $this->assign('something', $object) to $this->something = $object. We could keep the assign/assignRef methods (deprecated, of course) to do something like:

public function assign($key, $value) {

$this->$key = $value;

}

I think this would make more sense from an architectural point of view. Moreover, if we also add a magic __call() function we could allow for chaining which in turn would allow us to write something like $this->foo($value1)->bar($value2)->baz($value3)->display(); Maybe I'm just a little biased here; I've been using this kind of code with Nooku Framework for over a year and I find the way the J! FW does value assignments cumbersome. Not to mention that writing less code improves readability and makes spotting and solving bugs just so much easier. OK, I'll stop now as this is most likely not the proper list to discuss that.

Regarding the pagination query, can you point me to this discussion, please? I took a look at Joomla! 1.6.5's code before posting and I'm pretty sure it isn't doing a COUNT query. Was that changed in 1.7? I haven't had the chance to dive into its code yet :)

Cheers,

-- 

Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: http://www.AkeebaBackup.com

Blog: http://www.dionysopoulos.me/blog

[Niels Braczek]

Nicholas K. Dionysopoulos schrieb:

> why
> not use magic __set() and __get() methods to store such values to an
> internal array or a state object?

Because one *never* wants to use magic setters and getters. They are
misleading, because one doesn't see, that the data may be manipulated
during transfer.

And, for clean code's sake, one would always avoid direct access to
another object's properties, since that prohibits refactoring of the
internal data representation.

Ideally, *all* properties are private, and only accessible by getter and
setter methods, following the naming convention setXxxx()/getXxxx().
That way, one gets self-documenting code with better maintainability and
testability.

> I think this would make more sense from an architectural point of
> view. Moreover, if we also add a magic __call() function we could
> allow for chaining which in turn would allow us to write something
> like $this->foo($value1)->bar($value2)->baz($value3)->display();

The names of properties are nouns or adjectives; the names of methods
are verbs. Following this clean code rule, it is prohibited to address a
property 'foo' using a method foo(). It must be setFoo() instead.

Since a modern IDE can generate getter and setter methods on a single
mouse click, using the magic __call() smells ugly. It is more work to
keep the DocBlocks in sync, and will most likely not be updated, when a
new property is introduced. Thus, you will have improper documentation.
Therefor this is a bad idea. Use real getter and setter methods instead.

Regards,
Niels

--
| http://www.kolleg.de · Das Portal der Kollegs in Deutschland |
| http://www.bsds.de · BSDS Braczek Software- und DatenSysteme |
| Webdesign · Webhosting · e-Commerce · Joomla! Content Management |
------------------------------------------------------------------

[Nicholas Dionysopoulos]

I don't think you understood what I wrote. Please read Nooku's code to see what I mean.

Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com
Web: https://www.akeebabackup.com
Blog: http://www.dionysopoulos.me/blog

Sent from my iPad. Please excuse my brevity.

[Andrew Eddie]

Hi Nic

Yeah you could do that, but I think it's also just as easy to replace:

$this->assignRef('items', $this->getItems('items'));

with:

$this->items = $this->get('Items');

Remember those properties are for the benefit of the layout designer
who are not necessarily master PHP coders who know the in's and outs
of magic methods and chaining.

The chat about getTotal is here:

http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_id=8103&tracker_item_id=26291

I've so far found knocking out the joins has negligible impact (on
well indexed joins), but reducing select <all fields> to select
<primary key> makes a huge impact.

Regards,
Andrew Eddie


On Jul 19, 6:57 pm, "Nicholas K. Dionysopoulos" <nikosd...@gmail.com>
wrote:

> > On Jul 19, 7:39 am, Nicholas Dionysopoulos <nikosd...@gmail.com (http://gmail.com)>
> > wrote:
> > > With all due respect, Andrew, if anyone is running a high traffic website without APC and a transparent CDN he is either stupid or just plain dangerous. When Gazetta.gr (http://Gazetta.gr) ran on Joomla! it could server several millions of unique visitors with a single server. I have not heard of a more heavy traffic website.

>
> > > Besides, if your are looking for performance bottlenecks, please consider how JModel and, as a result, com_content get to provide pagination. I have seen beefy servers being reduced to a pile of junk just because com_content was trying to paginate the front page of the site. The site only had a few dozen thousand articles.
>
> > > Well, if you are still looking for a way to save 39 nanoseconds per page load, I would suggest removing comments and whitespace from all PHP files. You will save about 50 nanoseconds doing that. Just because you can doesn't mean you should ;)
>
> > > Nicholas K. Dionysopoulos

> > > Lead Developer, AkeebaBackup.com (http://AkeebaBackup.com)

> > > Web:https://www.akeebabackup.com
> > > Blog:http://www.dionysopoulos.me/blog
>
> > > Sent from my iPad. Please excuse my brevity.
>

> > > 19 Ιουλ 2011, 0:28, ο/η Andrew Eddie <mambob...@gmail.com (http://gmail.com)> έγραψε:
>
> > > > The suggested changes are small for low load sites, but could possibly
> > > > add up in very high traffic sites. Whatever the case, the assignref
> > > > method is a hangover of the php 4 days and we should be standardising
> > > > on not using it. Peter's suggestion about substr is also good. Each
> > > > change in isolation is small, maybe even insignificant, but if people
> > > > are willing to do the work to provide patches, I wouldn't stop them.
> > > > These are quick, good coding practice wins.
>
> > > > Regards
> > > > Andrew Eddie
>

> > > > On Jul 18, 10:52 pm, Yurii Korotia <k...@qwist.org (http://qwist.org)> wrote:
> > > > > fine then. I didn't say 96% slower, but +96% overhead, where +4% gives
> > > > > direct assign.
> > > > > about tracing - it's pretty much the same, same with mysql logging. If
> > > > > you make loop testing near 1kk it makes your result less floating, not
> > > > > because it hasn't random, but because time difference becomes higher.
>
> > > > > about xdebug, here we have sentence like "don't shoot fly with cannon"
>
> > > > > and yes, you're right, I'm going to shut up, will return with
> > > > > something other
>
> > > > > have a nice day
>
> > > > --
> > > > You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.

> > > > To post to this group, send an email to joomla-de...@googlegroups.com (mailto:joomla-de...@googlegroups.com).
> > > > To unsubscribe from this group, send email to joomla-dev-gene...@googlegroups.com (mailto:joomla-dev-gene...@googlegroups.com).

> > > > For more options, visit this group athttp://groups.google.com/group/joomla-dev-general?hl=en-GB.
>
> > --
> > You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.

> > To post to this group, send an email to joomla-de...@googlegroups.com (mailto:joomla-de...@googlegroups.com).
> > To unsubscribe from this group, send email to joomla-dev-gene...@googlegroups.com (mailto:joomla-dev-gene...@googlegroups.com).

[Nicholas K. Dionysopoulos]

Hi Andrew,

Well, yeah, no matter which method you use, you always have to write $this->items = (value). The difference is that with magic getter/setters you end up saving properties in a protected object, without "polluting" the view object. The magic call method could allow the developer (the one dealing with view.html.php) to write less code. The designer (dealing with tmpl/*.php files) needn't know how view.html.php works. He should know that $this->items works. How? That's none of his business. I would never sacrifice the ease of templating just to make the developer's life a tad easier.

Thank you for the link! I had seen that discussion but I thought it was dead. The status is still set to "Confirmed". I have ended up with the same conclusion a couple of months ago, seeing which queries on Akeeba Release System cause the most slowdown. With a main download table in excess of 1.8 million records, performance suffered with JOINs. I was already in the process of killing JOINs. Now I see that it's a good idea to do COUNT(pk) instead of COUNT(*). That's a good tip, thank you!

-- 
Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: http://www.AkeebaBackup.com

Blog: http://www.dionysopoulos.me/blog

To post to this group, send an email to joomla-de...@googlegroups.com.
To unsubscribe from this group, send email to joomla-dev-gene...@googlegroups.com.

[Yurii Korotia]

you are wrong about count(*)

for counting rows in one table without where clause - it is the same
as counting by primary key as PK is index by default.

Even, when you use 1 where clause it is much faster then by id. Look
for explain

explain extended select sql_no_cache count(*) from jos_users where
email like '%%@%%';
id select_type table type possible_keys key key_len ref rows filtered
Extra
1 SIMPLE jos_users index NULL email 302 NULL 3512 100.00 "Using where;
Using index"

explain extended select sql_no_cache count(id) from jos_users where
email like '%%@%%';
id select_type table type possible_keys key key_len ref rows filtered
Extra
1 SIMPLE jos_users ALL NULL NULL NULL NULL 3512 100.00 "Using where"

you may see your technique with count(id) DOESN'T use index as type is
all. So relax, your technique is crap.

Another impact is engine type. You use MyISAM which is crap comparing
to InnoDB, not always of course, but for jos_content you could easily
use InnoDB to make your site operate faster.


using join

explain select sql_no_cache count(*) from jos_content c right join
jos_categories ca on c.catid = ca.id;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ca index NULL PRIMARY 4 NULL 16 "Using index"
1 SIMPLE c ref idx_catid idx_catid 4 test.ca.id 2535 "Using index"



explain select sql_no_cache count(c.id) from jos_content c right join
jos_categories ca on c.catid = ca.id;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ca index NULL PRIMARY 4 NULL 16 "Using index"
1 SIMPLE c ref idx_catid idx_catid 4 test.ca.id 2535

again, second table went with count(c.id) without extra 'using index'.

any arguments? MySQL is 5.5

what queries do you use?

[Nicholas K. Dionysopoulos]

Yuril,

I agree with your comments about COUNT(*), given the MySQL explanation of the query.

Regarding the database engine, I don't think that InnoDB is an option with content. InnoDB doesn't allow full text indices, so search performance would suffer. I know, in the era of Google why are we still using our own search but, apparently, many people find it useful and most site visitors do use it.

Regarding the JOINs, if you try it on very large datasets you can see a dramatic speed difference. Selecting from a table with 1,800,000 rows, JOINED with two other tables having 20 and 5 rows respectively, the top-20 select query took 35 seconds. Removing the JOINs, the query took less than half a second. Well, it's much "cheaper" to load the tiny tables in an indexed array in memory and use PHP to translate IDs to human-readable data on the template than doing that with JOINs in the database.

-- 
Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: http://www.AkeebaBackup.com

Blog: http://www.dionysopoulos.me/blog

[Yurii Korotia]

query example, please.

When I get some time I could generate 2kk rows and try it. You
probably talk about jos_ content

[JM Simonet]

Could you avaoid usig the word c..p in your posts?
This triggers on my mail client an alert for "bad words" and anyway,
is totally useless for a demonstration.
Something like "does'nt work" or "is not good IMHO" would serve the
same purpose.

Thank you.
JM

>--
>You received this message because you are subscribed to the Google
>Groups "Joomla! General Development" group.
>To post to this group, send an email to joomla-de...@googlegroups.com.
>To unsubscribe from this group, send email to
>joomla-dev-gene...@googlegroups.com.
>For more options, visit this group at
>http://groups.google.com/group/joomla-dev-general?hl=en-GB.

--
>Please keep the Subject wording in your answers
This e-mail and any attachments may be confidential. You must not
disclose or use the information contained in this e-mail if you are
not the
intended recipient. If you have received this e-mail in error, please
notify us immediately and delete the e-mail and all copies.
-----------------------------------------------------------
Jean-Marie Simonet / infograf768
Joomla Leadership Team - Production Working group
Joomla! Translation Coordination Team

[Yurii Korotia]

sure thing

well to know your users are blocked from http://en.wikipedia.org/wiki/Craps
too

p.s.
full off topic
ass far ass humans are so close to animals and live based on instincts
more than humanity human won't be understood here as words are used
not in their initial meanings.

[DDoSAttack Hasslehoff]

@JM did you seriously just interrupt the flow of conversation over your irresponsible filter?

On Jul 20, 2011 5:13 AM, "JM Simonet" <infog...@gmail.com> wrote:
> Could you avaoid usig the word c..p in your posts?
> This triggers on my mail client an alert for "bad words" and anyway,
> is totally useless for a demonstration.
> Something like "does'nt work" or "is not good IMHO" would serve the
> same purpose.
>
> Thank you.
> JM
>

>>--
>>You received this message because you are subscribed to the Google
>>Groups "Joomla! General Development" group.
>>To post to this group, send an email to joomla-de...@googlegroups.com.
>>To unsubscribe from this group, send email to
>>joomla-dev-gene...@googlegroups.com.
>>For more options, visit this group at
>>http://groups.google.com/group/joomla-dev-general?hl=en-GB.
>
>

> --
>>Please keep the Subject wording in your answers
> This e-mail and any attachments may be confidential. You must not
> disclose or use the information contained in this e-mail if you are
> not the
> intended recipient. If you have received this e-mail in error, please
> notify us immediately and delete the e-mail and all copies.
> -----------------------------------------------------------
> Jean-Marie Simonet / infograf768
> Joomla Leadership Team - Production Working group
> Joomla! Translation Coordination Team
>

[DDoSAttack Hasslehoff]

@Nicholas I know that PHP provides magic getters and setters, but I would highly discourage their usage.

Using public getSomething() and setSomething() with private fields keeps the internal logic private thus NOBODY needs to know how it works... it just does. Furthermore, it keeps things much more readable throughout the life of the class. You know exactly what data you are handling and manipulating and what it belongs to.

Magic getters and setters are a holdover from when PHP was starting to become more object oriented. I have never had to use them and don't foresee a time where I would, especially when there are better, more clear, easily documented, easily understood alternatives.

One should feel dirty using those things IMO lol

[Nicholas K. Dionysopoulos]

I beg to differ. DDD is certainly not dirty and it's where everyone is moving. __get(), __set() and __call() are the foundation of DDD implementations in PHP AFAIK.

-- 

Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: http://www.AkeebaBackup.com

Blog: http://www.dionysopoulos.me/blog

[Yurii Korotia]

are you talking about this ?

<?php
//...[snip - add your MySQL connection code here]...

class mytable {
public $Name;

public function __construct($Name) {
$this->Name = $Name;
}

public function __set($var, $val) {
mysql_query("UPDATE {$this->Name} SET $var = '$val';");
}

// public $AdminEmail = 'f...@bar.com';
}

$systemvars = new mytable("systemvars");
$systemvars->AdminEmail = 'tel...@somesite.net';
?>

guys, it's suicide to use it. I found this piece of code in internet,
but it's really cool how you can make troubles for yourself from
nothing


by the way, what about hard queries? I can try to fix them

[Nicholas K. Dionysopoulos]

Of course not! That's the most idiotic way to use magic setters that I've seen in a lifetime. Ugh!

-- 
Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: http://www.AkeebaBackup.com

Blog: http://www.dionysopoulos.me/blog

[Yurii Korotia]

but magic is always out of rational borders, you know.
Also I found how we can make from private function a public function
with _call. Same with __get() - we can make from private public
properties. Not by will, but accidentally.
Also __set() cannot set variable by reference

what is DDD?

On 20 Лип, 20:19, "Nicholas K. Dionysopoulos" <nikosd...@gmail.com>
wrote:

>  Of course not! That's the most idiotic way to use magic setters that I've seen in a lifetime. Ugh!
>
> --
> Nicholas K. Dionysopoulos
> Lead Developer, AkeebaBackup.com
> Web:http://www.AkeebaBackup.com
> Blog:http://www.dionysopoulos.me/blog
>
> On Wednesday, 20 July 2011 at 20:05, Yurii Korotia wrote:
> > are you talking about this ?
>
> > <?php
> >  //...[snip - add your MySQL connection code here]...
>
> >  class mytable {
> >  public $Name;
>
> >  public function __construct($Name) {
> >  $this->Name = $Name;
> >  }
>
> >  public function __set($var, $val) {
> >  mysql_query("UPDATE {$this->Name} SET $var = '$val';");
> >  }
>

> >  // public $AdminEmail = '...@bar.com (mailto:f...@bar.com)';

> >  }
>
> >  $systemvars = new mytable("systemvars");

> >  $systemvars->AdminEmail = 'tel...@somesite.net (mailto:tel...@somesite.net)';

> > ?>
>
> > guys, it's suicide to use it. I found this piece of code in internet,
> > but it's really cool how you can make troubles for yourself from
> > nothing
>
> > by the way, what about hard queries? I can try to fix them
>
> > --
> > You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.

> > To post to this group, send an email to joomla-de...@googlegroups.com (mailto:joomla-de...@googlegroups.com).
> > To unsubscribe from this group, send email to joomla-dev-gene...@googlegroups.com (mailto:joomla-dev-gene...@googlegroups.com).

[Nick Savov]

DDD, is perhaps this:
http://en.wikipedia.org/wiki/Domain-driven_design

> joomla-de...@googlegroups.com.

> To unsubscribe from this group, send email to

> joomla-dev-gene...@googlegroups.com.

[Andrew Eddie]

Actually, correction. I didn't use COUNT() at all, I just injected a
getListTotalQuery method into my API and had that clear the select an
do $query->clear('select)->select('a.pk'). Doing a quick test on the
same data, SELECT a.pk is twice as fast as SELECT COUNT(*) or SELECT
COUNT(a.pk) - go figure.

But on a house keeping matter, we should really split all these
respective performance topics up and be talking about them on the CMS
or Framework list(s), not General (it's more for about how you are
using Joomla, not how to improve Joomla).

Regards,
Andrew Eddie

On Jul 20, 4:08 pm, "Nicholas K. Dionysopoulos" <nikosd...@gmail.com>
wrote:
> Hi Andrew,
>

> Well, yeah, no matter which method you use, you always have to write $this->items = (value). The difference is that with magic getter/setters you end up saving properties in a protected object, without "polluting" the view object. The magic call method could allow the developer (the one dealing with view.html.php) to write less code. The designer (dealing with tmpl/*.php files) needn't know how view.html.php works. He should know that $this->items works. How? That's none of his business. I would never sacrifice the ease of templating just to make the developer's life a tad easier.
>
> Thank you for the link! I had seen that discussion but I thought it was dead. The status is still set to "Confirmed". I have ended up with the same conclusion a couple of months ago, seeing which queries on Akeeba Release System cause the most slowdown. With a main download table in excess of 1.8 million records, performance suffered with JOINs. I was already in the process of killing JOINs. Now I see that it's a good idea to do COUNT(pk) instead of COUNT(*). That's a good tip, thank you!
>
> --
> Nicholas K. Dionysopoulos
> Lead Developer, AkeebaBackup.com
> Web:http://www.AkeebaBackup.com
> Blog:http://www.dionysopoulos.me/blog
>
>
>
> On Wednesday, 20 July 2011 at 00:41, Andrew Eddie wrote:
> > Hi Nic
>
> > Yeah you could do that, but I think it's also just as easy to replace:
>
> > $this->assignRef('items', $this->getItems('items'));
>
> > with:
>
> > $this->items = $this->get('Items');
>
> > Remember those properties are for the benefit of the layout designer
> > who are not necessarily master PHP coders who know the in's and outs
> > of magic methods and chaining.
>
> > The chat about getTotal is here:
>

> >http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEd...

>
> > I've so far found knocking out the joins has negligible impact (on
> > well indexed joins), but reducing select <all fields> to select
> > <primary key> makes a huge impact.
>
> > Regards,
> > Andrew Eddie
>

> > On Jul 19, 6:57 pm, "Nicholas K. Dionysopoulos" <nikosd...@gmail.com (http://gmail.com)>

> > wrote:
> > > Hi Andrew,
>
> > > OK, I understand that. But instead of "fixing" assign/assignRef why not use magic __set() and __get() methods to store such values to an internal array or a state object? It would make more sense. It would also change the variable assignment paradigm from $this->assign('something', $object) to $this->something = $object. We could keep the assign/assignRef methods (deprecated, of course) to do something like:
>
> > > public function assign($key, $value) {
> > > $this->$key = $value;
>
> > > }
>
> > > I think this would make more sense from an architectural point of view. Moreover, if we also add a magic __call() function we could allow for chaining which in turn would allow us to write something like $this->foo($value1)->bar($value2)->baz($value3)->display(); Maybe I'm just a little biased here; I've been using this kind of code with Nooku Framework for over a year and I find the way the J! FW does value assignments cumbersome. Not to mention that writing less code improves readability and makes spotting and solving bugs just so much easier. OK, I'll stop now as this is most likely not the proper list to discuss that.
>
> > > Regarding the pagination query, can you point me to this discussion, please? I took a look at Joomla! 1.6.5's code before posting and I'm pretty sure it isn't doing a COUNT query. Was that changed in 1.7? I haven't had the chance to dive into its code yet :)
>
> > > Cheers,
>
> > > --
> > > Nicholas K. Dionysopoulos

> > > Lead Developer, AkeebaBackup.com (http://AkeebaBackup.com)

[Nicholas K. Dionysopoulos]

Andrew,

Just a quick note, based on my (rather limited) experience with large sites, and I promise I will stop the thread here (as you said, it's more proper for the CMS list).

When you do a SELECT instead of COUNT, MySQL has to load in memory the list of all of the keys. With 50,000 records we end up wasting almost a megabyte of memory. The count is slower,yes, but doesn't waste any RAM. I would argue that the count method is better for such sites. I know of at least one site which has a huge RAM issue due to com_content's pagination but has no problem with CPU usage.

Nicholas K. Dionysopoulos
Web Development & IT Services
http://www.dionysopoulos.me

http://www.AkeebaBackup.com

2011/7/21 Andrew Eddie <mamb...@gmail.com>

To post to this group, send an email to joomla-de...@googlegroups.com.
To unsubscribe from this group, send email to joomla-dev-gene...@googlegroups.com.

[Andrew Eddie]

Yuri, how about we move this one to the framework list.

The class to tackle first, as a test case, would be JModelList

I'd add an extra method for getListTotalQuery (defaults to return
$this->getListQuery() to keep the status quo) that the child class can
override and "tune" for the specific circumstances, manipulating the
JDatabaseQuery object in the process.

And we need a big data set to test :)

Just a suggestion.

Thanks.

Regards,
Andrew Eddie

[Nicholas K. Dionysopoulos]

Hi Andrew,

I think it's more than conceivable to create a quick'n'dirty script to create a few dozens nested categories with several hundred articles full of Lorem Ipsum text for testing purposes. So, where do we start? :)

Best regards,

-- 
Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: http://www.AkeebaBackup.com

Blog: http://www.dionysopoulos.me/blog

[Mark Dexter]

Such a script would be very useful to the Bug Squad. There are a number of times when we could use this. I'm attaching a script Hannes wrote to create a bunch of categories, just in case it might be useful. Mark