Bulbs property index

134 views
Skip to first unread message

keith

unread,
Jun 7, 2012, 11:57:13 AM6/7/12
to Neo4j
If I'm doing something wrong here please let me know what; if the
feature is not implemented is there a timeline for it, or should the
docs note such? A suggested workaround?

I'm running with yesterday's PyPI download (bulbs 0.3-20120508) and
the default config, attempting to create a non-default index:

class Person(Node):
element_type = "person"
django_id = Integer(indexed=True, nullable=False)
g.add_proxy("person", Person)
james = g.person.create(name="James", django_id=1)

I don't see any sign of a relevant index in Bulbs:
(Pdb) james.get_property_keys()
['django_id']
(Pdb) james.get_index_keys()
(Pdb)
Or in the web admin:
gremlin> g.indices
==> AUTOMATIC[vertices:Vertex][autoIndexKeys:null]
==> MANUAL[person:Vertex]
==> AUTOMATIC[edges:Edge][autoIndexKeys:null]

Source code for bulbs.property look suspicious, but perhaps the
comment just refers to the unique, not the indexed:
# These aren't implemented yet.
# TODO: unique creates an index
self.indexed = indexed

django_id is a reference to a SQL primary key that ties into the
Django auth system user table, so I will be often using it to lookup
the graph node for traversal start.

Thanks,

Keith

James Thornton

unread,
Jun 7, 2012, 12:22:46 PM6/7/12
to ne...@googlegroups.com
Bulbs automatically creates a "person" index for you and adds it to the model proxy:

>>> g.person.index.lookup(name="James")

...it indexes all keys by default.

- James

keith

unread,
Jun 8, 2012, 1:22:53 PM6/8/12
to Neo4j
Yes, you just helped clear up a basic misunderstanding of indexing in
Neo4J on my part. Thanks for the quick reply.

James Thornton

unread,
Jun 8, 2012, 4:44:37 PM6/8/12
to ne...@googlegroups.com


On Friday, June 8, 2012 12:22:53 PM UTC-5, keith wrote:
Yes, you just helped clear up a basic misunderstanding of indexing in
Neo4J on my part.  Thanks for the quick reply. 

Note that Neo4j does not auto index by default nor does it index all keys by default -- Bulbs does this for you and uses Neo4j manual indexes under the hood. 

Here's how a basic model create works, which creates and indexes a Node:

>>> g.people.create(name="James")  

# you call the create() method on NodeProxy "people":

# which calls _create() on Node model (you can override this to change behavior): 

# which calls create_indexed_vertex() on Neo4jClient:

# which grabs the create_indexed_vertex Gremlin script (manually indexes all keys by default):

# which then executes the Gremlin script on Neo4j Server, passing in the params:

The create_indexed_vertex Gremlin script is where all the indexing occurs, and it works the the same way for relationships.

- James
 

Peter Neubauer

unread,
Jun 9, 2012, 2:19:17 AM6/9/12
to ne...@googlegroups.com

Thanks James for the explanation!

Send from mobile.

s...@kill.io

unread,
Jun 26, 2012, 3:52:39 AM6/26/12
to ne...@googlegroups.com
An ugly workaround would be:

james2 = g.people.create(name="James")
g.vertices.index.put(james2.eid,"name","James")

using python 2.7, neo4j 1.7.1 community and bulbs 0.3_20120625 (forgot to post yesterday)

regards

Am Montag, 25. Juni 2012 23:31:47 UTC+2 schrieb s...@kill.io:
Hi,

i'm having trouble with exactly this feature.

Illustration on the quickstart example:

class Person(Node):

    element_type = "person"
    name = String(nullable=False)
    age = Integer()


class Knows(Relationship):

    label = "knows"
    created = DateTime(default=current_datetime, nullable=False)
    

test:
print "start"
g = Graph()

print "before config"
#g.config.set_logger(DEBUG)

#std example
james = g.vertices.create(name="James")
julie = g.vertices.create(name="Julie")
g.edges.create(james, "knows", julie)

print "add proxy"
g.add_proxy("people", Person)
g.add_proxy("knows", Knows)

print "add persons"
james2 = g.people.create(name="James2")
julie2 = g.people.create(name="Julie2")
print "add relation"
g.knows.create(james2, julie2)
g.knows.create(james, julie2)

print "result:"
print g.V
print len(list(g.vertices.index.lookup(name="James")))
print len(list(g.vertices.index.lookup(name="James2")))

normally the ouptut of the first and second len() statement should be equal. but instead the output is (debug off for readability):

start
before config
add proxy
add persons
add relation
result:
1
0

can you point me sth that i am probably doing wrong? system is ubuntu 12.04 64bit and atm i dont have gremmlin standalone installed 

cheers and regards,

ben

output with debug on:
start
before config
POST body: {"params":{"keys":null,"index_name":"vertex","data":{"name":"James"}},"script":"neo4j = g.getRawGraph()\n  manager = neo4j.index()\n  g.setMaxBufferSize(0)\n  g.startTransaction()\n  try {\n    index = manager.forNodes(index_name)\n    vertex = neo4j.createNode()\n    for (entry in data.entrySet()) {\n      if (entry.value == null) continue;\n      vertex.setProperty(entry.key,entry.value)\n      if (keys == null || keys.contains(entry.key))\n\tindex.add(vertex,entry.key,String.valueOf(entry.value))\n    }\n    g.stopTransaction(TransactionalGraph.Conclusion.SUCCESS)\n    return vertex\n  } catch (e) {\n    g.stopTransaction(TransactionalGraph.Conclusion.FAILURE)  \n    return e\n  }"} 
POST body: {"params":{"keys":null,"index_name":"vertex","data":{"name":"Julie"}},"script":"neo4j = g.getRawGraph()\n  manager = neo4j.index()\n  g.setMaxBufferSize(0)\n  g.startTransaction()\n  try {\n    index = manager.forNodes(index_name)\n    vertex = neo4j.createNode()\n    for (entry in data.entrySet()) {\n      if (entry.value == null) continue;\n      vertex.setProperty(entry.key,entry.value)\n      if (keys == null || keys.contains(entry.key))\n\tindex.add(vertex,entry.key,String.valueOf(entry.value))\n    }\n    g.stopTransaction(TransactionalGraph.Conclusion.SUCCESS)\n    return vertex\n  } catch (e) {\n    g.stopTransaction(TransactionalGraph.Conclusion.FAILURE)  \n    return e\n  }"} 
POST body: {"params":{"label_var":"label","outV":53,"keys":null,"data":{},"inV":54,"index_name":"edge","label":"knows"},"script":"import org.neo4j.graphdb.DynamicRelationshipType;\n  neo4j = g.getRawGraph()\n  manager = neo4j.index()\n  vertex = neo4j.getNodeById(outV)\n  relationshipType = DynamicRelationshipType.withName(label)\n  g.setMaxBufferSize(0)\n  g.startTransaction()\n  try {\n    index = manager.forRelationships(index_name)\n    edge = vertex.createRelationshipTo(neo4j.getNodeById(inV),relationshipType)\n    for (entry in data.entrySet()) {\n      if (entry.value == null) continue;\n      edge.setProperty(entry.key,entry.value)\n      if (keys == null || keys.contains(entry.key))\n\tindex.add(edge,entry.key,String.valueOf(entry.value))\n    }\n    index.add(edge,label_var,String.valueOf(label))\n    g.stopTransaction(TransactionalGraph.Conclusion.SUCCESS)\n    return edge\n  } catch (e) {\n    g.stopTransaction(TransactionalGraph.Conclusion.FAILURE)\n    return e\n  }"} 
add proxy
GET body: None 
POST body: {"config":{"type":"exact","provider":"lucene"},"name":"person"} 
GET body: None 
POST body: {"name":"knows"} 
add persons
POST body: {"params":{"keys":null,"index_name":"person","data":{"element_type":"person","name":"Jamesxxx"}},"script":"neo4j = g.getRawGraph()\n  manager = neo4j.index()\n  g.setMaxBufferSize(0)\n  g.startTransaction()\n  try {\n    index = manager.forNodes(index_name)\n    vertex = neo4j.createNode()\n    for (entry in data.entrySet()) {\n      if (entry.value == null) continue;\n      vertex.setProperty(entry.key,entry.value)\n      if (keys == null || keys.contains(entry.key))\n\tindex.add(vertex,entry.key,String.valueOf(entry.value))\n    }\n    g.stopTransaction(TransactionalGraph.Conclusion.SUCCESS)\n    return vertex\n  } catch (e) {\n    g.stopTransaction(TransactionalGraph.Conclusion.FAILURE)  \n    return e\n  }"} 
POST body: {"params":{"keys":null,"index_name":"person","data":{"element_type":"person","name":"Julie500"}},"script":"neo4j = g.getRawGraph()\n  manager = neo4j.index()\n  g.setMaxBufferSize(0)\n  g.startTransaction()\n  try {\n    index = manager.forNodes(index_name)\n    vertex = neo4j.createNode()\n    for (entry in data.entrySet()) {\n      if (entry.value == null) continue;\n      vertex.setProperty(entry.key,entry.value)\n      if (keys == null || keys.contains(entry.key))\n\tindex.add(vertex,entry.key,String.valueOf(entry.value))\n    }\n    g.stopTransaction(TransactionalGraph.Conclusion.SUCCESS)\n    return vertex\n  } catch (e) {\n    g.stopTransaction(TransactionalGraph.Conclusion.FAILURE)  \n    return e\n  }"} 
add relation
POST body: {"params":{"label_var":"label","outV":55,"keys":null,"data":{"created":1340659699},"inV":56,"index_name":"knows","label":"knows"},"script":"import org.neo4j.graphdb.DynamicRelationshipType;\n  neo4j = g.getRawGraph()\n  manager = neo4j.index()\n  vertex = neo4j.getNodeById(outV)\n  relationshipType = DynamicRelationshipType.withName(label)\n  g.setMaxBufferSize(0)\n  g.startTransaction()\n  try {\n    index = manager.forRelationships(index_name)\n    edge = vertex.createRelationshipTo(neo4j.getNodeById(inV),relationshipType)\n    for (entry in data.entrySet()) {\n      if (entry.value == null) continue;\n      edge.setProperty(entry.key,entry.value)\n      if (keys == null || keys.contains(entry.key))\n\tindex.add(edge,entry.key,String.valueOf(entry.value))\n    }\n    index.add(edge,label_var,String.valueOf(label))\n    g.stopTransaction(TransactionalGraph.Conclusion.SUCCESS)\n    return edge\n  } catch (e) {\n    g.stopTransaction(TransactionalGraph.Conclusion.FAILURE)\n    return e\n  }"} 
result:
POST body: {"params":null,"script":"g.getVertices()"} 
GET body: None 
1
GET body: None 
0

James Thornton

unread,
Jun 26, 2012, 3:58:47 AM6/26/12
to ne...@googlegroups.com


On Monday, June 25, 2012 4:31:47 PM UTC-5, s...@kill.io wrote:

print "add persons"
james2 = g.people.create(name="James2")
julie2 = g.people.create(name="Julie2")
print "add relation"
g.knows.create(james2, julie2)
g.knows.create(james, julie2)

print "result:"
print g.V
print len(list(g.vertices.index.lookup(name="James")))
print len(list(g.vertices.index.lookup(name="James2")))


Hi Ben -

Each model has its own index so the model-specific elements are stored in the generic vertices index.

For example, in your case, james2 is in the person index, and you can look it up like this:

>>> g.people.index.lookup(name="James2")

- James
 

James Thornton

unread,
Jun 26, 2012, 3:59:34 AM6/26/12
to ne...@googlegroups.com

Hi Ben -



For example, in your case, james2 is in the person index, and you can look it up like this:

>>> g.people.index.lookup(name="James2")

- James
 

Correction: Meant to say, "Each model has its own index so the model-specific elements are NOT stored in the generic vertices index.".

- James
 

s...@kill.io

unread,
Jun 26, 2012, 4:07:01 AM6/26/12
to ne...@googlegroups.com
oh dammit :) this happens if you don't read examples carefully enough... 

Sorry and thank you very very much!
Reply all
Reply to author
Forward
0 new messages