Trouble in understanding cost for Projection Query

[NAyAN Jain]

I am trying to reduce the datastore cost by using Projection. I have read that a Projection Query costs only 1 Read Operation but in my case the Projection cost goes more than 1. Here is the code:

Query<Finders> q = ofy().load().type(Finders.class).project("Password","Country");
for(Finders finder:q)
{
resp.getWriter().println(finder.getCountry()+" "+finder.getPassword());
}
On executing this, the q object contains 6 items and to retrieve these 6 items it takes 6 Read operations as shown in Appstats. Can anyone tell me what's wrong over here ?

[Christian F. Howes]

i suspect the docs meant to say "1 Read Operation per row"  i don't know how you could ever get to less then 1 read operation per row.

[Jeff Schnitzer]

A projection query should produce 1 read op for the query and 1 small op per row fetched.

Possibly Objectify is trying to hybridize the query (convert to keys-only + batch get) even though it's a projection. If this entity has the @Cache annotation, try running the query like this:

ofy().load().type(Finders.class).hybrid(false).project("Password","Country")

If that solves the problem, this is an Objectify bug and I'll fix it today. If it doesn't solve the problem, verify behavior with the low-level API - Objectify just passes the projection information on down.

Jeff

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/14da2fa4-115f-484e-8cdc-c319706009a5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[NAyAN Jain]

No I haven't used the cache annotation but still the cost and datastore read are same as a normal query.
I am looking for a way to reduce read operations.

[Jeff Schnitzer]

Run the equivalent query with the low-level API. If it produces the same unexpected results, it's Google's issue. If it fixes the problem, it's mine. Other than a potential hybridization issue I can't imagine what Objectify could be doing wrong since it's just adding projections to an underlying low-level Query object.

Jeff

On Fri, Dec 18, 2015 at 5:50 PM, NAyAN Jain <gamern...@gmail.com> wrote:

No I haven't used the cache annotation but still  the cost and datastore read are same as a normal query.
I am looking for a way to reduce read operations.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/604573fe-8f5b-4763-a742-358eef9f06ca%40googlegroups.com.

[Nick (Cloud Platform Support)]

Jeff's analysis is on-point here. It seems certainly possible that Ofy could be transforming the query. The only way is to test, and the methods to do so have been given. If this turns out to be a Datastore problem, rather than just Ofy, a post should be made to the Public Issue Tracker. Someone from Cloud Platform Community Support will triage the issue shortly after it appears.

On Saturday, December 19, 2015 at 10:01:06 AM UTC-5, Jeff Schnitzer wrote:

Run the equivalent query with the low-level API. If it produces the same unexpected results, it's Google's issue. If it fixes the problem, it's mine. Other than a potential hybridization issue I can't imagine what Objectify could be doing wrong since it's just adding projections to an underlying low-level Query object.

Jeff

On Fri, Dec 18, 2015 at 5:50 PM, NAyAN Jain <gamern...@gmail.com> wrote:

No I haven't used the cache annotation but still  the cost and datastore read are same as a normal query.
I am looking for a way to reduce read operations.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.

Visit this group at https://groups.google.com/group/google-appengine.

[Rahul Tongia]

Hi Jeff and Nick,

I am facing the same problem regarding projection query costs. I had performed the same projection call on my local machine with appstats using both Objectify (with hybrid as false) and native Datastore API. In both cases, I got same result in terms of Read counts. It was 896 reads where as I was expecting 1 read with 895 small ops. 

To me, the problem looks like either in Appstats for Java where it is not able to estimate the projection query entity reads as small ops or the Projection query itself is broken in terms of costs. Atleast, Objectify seems to be not causing this issue.

I will check in production, how my read costs get affected with this change.

Nick could you please check with Appstats team (if any one is maintaining it) if this is an Appstats issue?

Thanks,
Rahul

On Tuesday, December 22, 2015 at 7:07:55 PM UTC+1, Nick (Cloud Platform Support) wrote:

Jeff's analysis is on-point here. It seems certainly possible that Ofy could be transforming the query. The only way is to test, and the methods to do so have been given. If this turns out to be a Datastore problem, rather than just Ofy, a post should be made to the Public Issue Tracker. Someone from Cloud Platform Community Support will triage the issue shortly after it appears.

On Saturday, December 19, 2015 at 10:01:06 AM UTC-5, Jeff Schnitzer wrote:

Run the equivalent query with the low-level API. If it produces the same unexpected results, it's Google's issue. If it fixes the problem, it's mine. Other than a potential hybridization issue I can't imagine what Objectify could be doing wrong since it's just adding projections to an underlying low-level Query object.

Jeff

On Fri, Dec 18, 2015 at 5:50 PM, NAyAN Jain <gamern...@gmail.com> wrote:

No I haven't used the cache annotation but still  the cost and datastore read are same as a normal query.
I am looking for a way to reduce read operations.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.

Visit this group at https://groups.google.com/group/google-appengine.

[Nick (Cloud Platform Support)]

Hey Jeff,

A post like yours should be made to the Public Issue Tracker (linked in a previous post), along with technical details of the exact queries, what you expected, what you observed instead, and any information (code snippets, a complete repository link) needed to allow reproduction and observation on another machine (one of our machines). This would be the best way to make sure the issue is seen by the Appstats maintainers or someone close to such.

Regards,

Nick

[Jeff Schnitzer]

I think your message was for Rahul (added to cc just in case) - yes, it sounds like a great item for the issue tracker :)

Jeff

[Rahul Tongia]

Thanks both. I will post in the tracker.

Regards,

Rahul

Sent from my iPhone

[Nick (Cloud Platform Support)]

Hey Jeff and Rahul,

Yes, I had replied to the wrong individual message. At any rate, thanks for going to the appropriate forum, and best of luck in getting a quick resolution.