Performances issues (was Issues using sqlpo)
Fabien Penso
About the performances issues, I just benchmark a call to [Something
allObjects] with 150 rows in the table, on my devices it takes 3
secondes to load ! :( Is that the kind of performances expected (it's
far faster to load from a local plist...) or did I do something wrong?
A previous call to sqlite3 *database = [[SQLiteInstanceManager
sharedManager] database]; is pretty fast, so it's really the SELECT
itself (I think it's the part of SQLitePersistentObject.m that fetch
columns and put it in the properties).
Any hints ?
My Something is really simple :
@interface Something : SQLitePersistentObject {
NSString *name;
int zipcode;
NSString *name2;
NSString *name3;
NSString *name4;
NSString *name5;
NSString *name6;
NSString *name7;
float something1;
float something2;
int something3;
}
Scott Lyons
limit the number of objects you load at a time or using the paired
array system instead. One thing I'm working on that I hope I can
perfect is lazy loading and cached queries, which might help.
Fabien Penso
>
> I think there's two possible ways of dealing with this. One is to
> limit the number of objects you load at a time or using the paired
> array system instead. One thing I'm working on that I hope I can
> perfect is lazy loading and cached queries, which might help.
As the datas are really simple (simple tables) I have the feeling the
issue is not about loading 150 rows (the AdressBook has much more and
loads fast) but more about the code itself that loads the rows.
I'll digg into this, I would not be surprised if using something like
fmdb and manual calls would make a similar allObjects methods much
much faster.
Fabien Penso
> allObjects] with 150 rows in the table, on my devices it takes 3
> secondes to load ! :(
Guys,
I used FMDB to develop my own allObjects method to compare, on my
iPhone it takes 0.20 sec instead of >3secondes to load
150 rows. which is far more what I expect from such a device.
I guess I would recommand NOT using sqlpo if you store more than 10
rows in your table (or I missed something). I would have appreciated
to use a higher library for such a need, but the current performance
of sqlpo is really too slow.
I would recommand removing thoses for() loops which runs for every
column for every rows in SQLitePersistentObject.m, they feel like
performance killers.
Cheers.
Hunter Hillegas
solution to SQLPO was similar. I had to use paired arrays to get
anywhere near the same performance.
That said though, instead of just bailing on SQLPO, what can we do to
improve it and get some sort of parity?
Who knows, Apple may add CoreData to iPhone OS 3.0 but if they don't
or if you want to support 2.x, having a nice persistence framework is
important and a real help.
Fabien Penso
<li...@lastonepicked.com> wrote:
>
> My experience moving from a custom (but somewhat similar) FMDB
> solution to SQLPO was similar. I had to use paired arrays to get
> anywhere near the same performance.
Ah I'm not alone then. Maybe it should be in the README so people like
me don't spend hours trying to have something working, when it's
actually not working anyway.
> That said though, instead of just bailing on SQLPO, what can we do to
> improve it and get some sort of parity?
Sadly looking at the code I have the file I specified should be
rewritten. But the whole project is not so much line of code, I don't
get why nobody has written something like this.
Anyone tried the one from OmniGroup ? (I just checkout their source
but did not try it yet)
I came here as a user and not as a potential contributor (I know it
sounds like assh*le talk).
I'll just implement my own lib for my own use later if I need to, and
release it if it helps anyone.
> Who knows, Apple may add CoreData to iPhone OS 3.0 but if they don't
> or if you want to support 2.x, having a nice persistence framework is
> important and a real help.
I agree.
Scott Lyons
machine. Loading 150 rows means creating 150 objects at once. Yes, the
mechanics may not be perfect yet, but use a little sense here. Your
feedback is appreciated, but you started several threads about
problems you've had and all the time you wasted trying to fix them,
complaining about lack of unit tests for certain cases, then turn
around and say the you're just a user, not a contributor. This is a
hobby project for most of us. It was started by Jeff, who spends most
of his time trying not to fall asleep while coding and writing his
book. I'm not sure how anyone else feels, but you could have saved
time just by filing a damn bug report and kept your backseat
developing to yourself. Or at least present it in a manner that
doesn't undermine our efforts.
Hunter Hillegas
I was the one that started the original thread on performance problems
a few weeks ago, though I was not the one that posted about the
different loading methods and test cases, et al.
Let me first say that I think this project and what Jeff has done is
great - he's the spark (well, more than that - he wrote a lot of code)
but at a certain point, a project like this is about us running with
it. Let's do that. Let's improve it.
At the same time, what I've said and what others have said is that
there seems to be a wide differential between other DB methods and
SQLPO when it comes to performance. To some degree that doesn't
surprise me since SQLPO does more but that said, the differential
seems to be a bit wider than what I and some others expected.
Based on that previous experience, it would seem that performance
improvements are possible in SQLPO which is the best of all worlds for
all of us - an easy to use, Open Source framework for persistence on
the iPhone and other Cocoa platforms (who knows what Apple will add to
the mix on Tuesday). I don't think that reasonable performance with
medium sized sets of objects is unrealistic - if custom FMDB+hydration
methods can do it, why not SQLPO? I doubt it's impossible - how far
off could these two approaches be?
Paired arrays are basically explicit lazy loading minus the magic,
right? Just seems we could improve PO with some of these techniques
that have been explored in a multitude of persistence frameworks from
Hibernate to Core Data to EOF.
Maybe for your project, these optimizations are not important. To a
large degree Open Source is about scratching your own itch and unlike
the other poster, I *am* willing to contribute.
What I'd like to do is fine some like minded individuals and see if we
can make a dent on the performance delta. I'm less than six months
into Cocoa but if no one more senior is interested, I'll do my own
research, that's totally cool.
I agree that efforts are best spent via bug reports, writing code,
etc... I think we're on the same page, just those that are being bit
by performance issues maybe need to get together and donate some time.
Cheers,
Hunter
Scott Lyons
and after I'm done implementing the new structure I had some designs
for more effective querying.
The idea I had was to use a proxy object for the results. For large
collections it would load say 10 at a time. That way any sorting that
needed to be done could just be done invisibly by re-issuing the query
to the database.
The other thing I was thinking about was a magic query object, similar
to Django's Q object. The query object would create a special instance
of a SQLPO or something, and you could use it as such:
Query *q = [[Query alloc] initWithClass:[BasicData class]];
q.b1=10; // would return all objects with b1 of 1
q.b2.order_by = @"DESC";
You get the idea. It would do more of the hardcore lazy loading by not
actually touching the database until something is actually used.
Fabien Penso
Le 15 mars 09 à 05:34, Scott Lyons a écrit :
>
> You know, I'm just going to say it. The iPhone isn't a desktop
> machine. Loading 150 rows means creating 150 objects at once. Yes, the
> mechanics may not be perfect yet, but use a little sense here.
My code does create 150 objects at once, still is faster from a
magnitude.
Because it's not a desktop machine, you want the code to be superfast
and optimized.
Fabien Penso
Le 15 mars 09 à 06:20, Scott Lyons a écrit :
>
> I absolutely agree. I've made a few attempts at improving performance,
> and after I'm done implementing the new structure I had some designs
> for more effective querying.
>
> The idea I had was to use a proxy object for the results. For large
> collections it would load say 10 at a time. That way any sorting that
> needed to be done could just be done invisibly by re-issuing the query
> to the database.
You should first fix the findByCritera method and make it much faster,
quickly looking at it it
goes through every column for every returned rows, to see column's
type, etc. That should
be done only on the first returned column.
And that method is far too long and too complex to not be buggy and
slow.
I don't see how you're testing your few attemps of improving
performance without performance ... tests. I have not found them
anywhere.
Can I suggest to change the README on github to add a :
** WARNING **
The current state of the code should not be used for real
applications, its slow.
That would definitly save time for the upcoming users.