Filtering a list property for items NOT in the list

[j]

I'm trying to figure out if there's a way to query a list property to
find only entities that do NOT contain the value in question. Went
through the docs and the group and so far I've found nothing that says
you can.

So I know you can do this:

Query<Game> q = ofy().query(Game.class).filter("players",
someplayer)

To get me a list of all the games that contain a given player in the
player list. What I'm wanting to do is this:

Query<Game> q = ofy().query(Game.class).filter("players !=",
someplayer)

Except of course that means "not equal to" instead of "does not
contain." Is there some way to do this with filters, or am I going to
have to use a relationship? Retrieving all the games and looking
through the player list of each seems like a non-scalable solution.

Thanks in advance.

[Didier Durand]

Hi,

The way I see is the low-level API of GAE datastore: this way, you
have a separate access to each property as an object and you can check
if it's here or not.

I look forward to hearing how we can also do it with Objectify.

regards

didier

[Jeff Schnitzer]

I'm pretty sure what this query (find entities with a list property
devoid of a value) is not possible on the appengine datastore.

Querywise, there shouldn't be anything you can do with the low level
API that you can't do with Objectify (and vice-versa). The mapping is
quite thin.

Jeff

[Didier Durand]

Hi Jeff,

I thought what j is looking for is to see if a property is present or
not in the datastore for a given Entity

So, I thought that low-level API Entity.getProperties() would do the
trick: you don't get those properties mappeds in properties of a pojo
(and set to null if they don't exist in ds) as with ofy but you get a
list of those really present or not. Then you can check if they were
really used or not.

Am I wrong?

regards

didier

On Feb 5, 8:51 am, Jeff Schnitzer <j...@infohazard.org> wrote:
> I'm pretty sure what this query (find entities with a list property
> devoid of a value) is not possible on the appengine datastore.
>
> Querywise, there shouldn't be anything you can do with the low level
> API that you can't do with Objectify (and vice-versa).  The mapping is
> quite thin.
>
> Jeff
>

[Jeff Schnitzer]

It sounds to me like the OP was asking about queries...

Jeff

[Didier Durand]

Hi,

@Jeff: then, what about querying if property is null and for the
resulting entities, do what I suggest to see if the property was
created or not ?

regards

didier

On Feb 6, 3:45 pm, Jeff Schnitzer <j...@infohazard.org> wrote:
> It sounds to me like the OP was asking about queries...
>
> Jeff
>

[Jeff Schnitzer]

If you make a query like this...

ofy.query(Thing.class).filter("listProperty", null);

...you'll get back all Things that have a null in the list property.
Probably not what the OP wanted.

GAE does not make it easy to find the absence of things in list
properties. If there's a magic way of doing this, I don't know about
it.

Jeff

[j]

Just to confirm, that's not what I want. I basically want all
entities where listProperty.contains(value) == false.

Thanks for all the comments. I figured Objectify might not do it
because the underlying framework doesn't support it, but I just
thought I'd check.

On Feb 6, 10:38 am, Jeff Schnitzer <j...@infohazard.org> wrote:
> If you make a query like this...
>
> ofy.query(Thing.class).filter("listProperty", null);
>
> ...you'll get back all Things that have a null in the list property.
> Probably not what the OP wanted.
>
> GAE does not make it easy to find the absence of things in list
> properties.  If there's a magic way of doing this, I don't know about
> it.
>
> Jeff
>

[Didier Durand]

Hi,

Entity.getProperties() lets you define easily if a property is missing
of not if you have the list of the names you are checking against:
either thoses names are in the returned map or not.

regards

didier

[j]

Thanks for the reply, Didier. Unfortunately, I think it is a little
too advanced for me right now. Could you explain where
Entity.getProperties() is found and how you would use it to find out
if a key was in a property (that is a list of keys)?

[Jeff Schnitzer]

I think there is a big communication impedance mismatch here. Correct
me if I'm wrong but:

* j wants to query for entities which lack a certain value in a list
property (not possible on GAE, afaik).
* Didier is discussing how you get raw property data once you have an
entity (presumably fetched or queried for in some way).

Unless you're doing an exhaustive search through all your entities,
these are wholly unrelated.

Jeff

[j]

Right, Jeff. I wanted to query the datastore to get all entities that
lack a certain value in a list. Fetching all entities and then
checking for the value in each list for each entity wouldn't really
scale.

Thanks for helping clarify.

On Feb 7, 1:37 am, Jeff Schnitzer <j...@infohazard.org> wrote:
> I think there is a big communication impedance mismatch here.  Correct
> me if I'm wrong but:
>
>  * j wants to query for entities which lack a certain value in a list
> property (not possible on GAE, afaik).
>  * Didier is discussing how you get raw property data once you have an
> entity (presumably fetched or queried for in some way).
>
> Unless you're doing an exhaustive search through all your entities,
> these are wholly unrelated.
>
> Jeff
>

[Jeff Schnitzer]

Depending on how wide the range of possible values are, you might
consider storing a list property of all the values that each entity
*doesn't* have...

Jeff

[Didier Durand]

Hi,

My solution works is the density of entities having the missing
properties is low:

a) you query the entities with null value for a certain property (or a
set of them)
b) for each of the resulting entity, you check via low-level api is
property is missing or not.

Then, it's clear that it doesn't scale if the density of entities with
null property(ies) is high. But that was a proposal in case the
context is appropriate.

regards

didier

[j]

Yeah, not really the case. The real example here is that I have a
bunch of players. Players can start games. Each game has a list of
players that have joined it. I want to list all the games you can
join. You cannot join a game you have already joined, so it should
not show in the list. This is why I want to say "show me all the
games that do not have Player 123 in the list of players".

On Feb 7, 2:32 am, Jeff Schnitzer <j...@infohazard.org> wrote:
> Depending on how wide the range of possible values are, you might
> consider storing a list property of all the values that each entity
> *doesn't* have...
>
> Jeff
>

[Jeff Schnitzer]

Sounds pretty reasonable to iterate all the games and manually skip
the ones you can't join - should be a small number, no?

Jeff

[j]

Well, yeah, unless my site every actually took off and there were
thousands of people playing worldwide. That's the hope, at least. :D

On Feb 9, 1:49 am, Jeff Schnitzer <j...@infohazard.org> wrote:
> Sounds pretty reasonable to iterate all the games and manually skip
> the ones you can't join - should be a small number, no?
>
> Jeff
>

[Josh Landin]

Even still, how many games could one person reasonably play at a time? 5, 10, 100, even 500 would still perform well being removed from the result. It sounds more like a grey-out feature than anything. That is "greying out" the play button for the ones I can't join *again*.

--
Josh

[Jeff Schnitzer]

...and I would expect the problem is not how to deal with the 50 games
that a user might have already joined, but presenting the 50,000 games
that the user could join.

Jeff

[j]

All good points from both Jeff and Josh. Thanks for the replies. And
in reality, there would be some matchmaking system that would further
filter them down based on skill level, etc.

So this is probably not a great instance of an argument for needing
the functionality (in GAE) to avoid scalability problems. I'm sure
they exist, though. :D

On Feb 9, 3:07 pm, Jeff Schnitzer <j...@infohazard.org> wrote:
> ...and I would expect the problem is not how to deal with the 50 games
> that a user might have already joined, but presenting the 50,000 games
> that the user could join.
>
> Jeff
>