How to create lucene index with Python Bulbs framework?

[Kevin I]

I have the Lucene index plugin installed and active. I just can't figure out how to create indices. I tried this:

g.gremlin.execute('create index Student.name on Student (name) fulltext engine lucene')

but it doesn't work. Returns the following error:

SystemError: ({'status': '500', 'transfer-encoding': 'chunked', 'server': 'grizzly/2.2.16', 'connection': 'close', 'date': 'Tue, 14 Apr 2015 20:54:50 GMT', 'access-control-allow-origin': '*', 'content-type': 'application/json'}, '{"message":"","error":"javax.script.ScriptException: groovy.lang.MissingPropertyException: No such property: index for class: Script5","api":{"description":"evaluate an ad-hoc Gremlin script for a graph.","parameters":{"returnTotal":"when set to true, the full result set will be iterated and the results returned (default is false)","rexster.returnKeys":"an array of element property keys to return (default is to return all element properties)","rexster.showTypes":"displays the properties of the elements with their native data type (default is false)","load":"a list of \'stored procedures\' to execute prior to the \'script\' (if \'script\' is not specified then the last script in this argument will return the values","rexster.offset.end":"end index for a paged set of data to be returned","rexster.offset.start":"start index for a paged set of data to be returned","params":"a map of parameters to bind to the script engine","language":"the gremlin language flavor to use (default is groovy)","script":"the Gremlin script to be evaluated"}},"success":false}')

What am I missing here?

[Kevin I]

I just realized that I'm trying to interpret the SQL query with the Gremlin interpreter.

Still I don't know how to execute it.

[ky...@ovguideinc.com]

I'm pretty certain you can't create these fancy orientdb indices via bulbs.

----

This open issue is about adding support to directly talk to orientdb via the REST api.

https://github.com/espeed/bulbs/issues/128

I'd very much like this feature!  

----

you can use pyorient to talk to orientdb in python: 

https://github.com/mogui/pyorient

----

Additionally, there are serialization issues with custom orientdb stuff.

e.g, a list of 

EMBEDDED SETS/LISTS get serialized as:

u'kind': u'[tv_tv_program, film_film]'

I've noticed DATE fields getting serialized as:

u'modified': u'Wed Apr 01 15:58:46 PDT 2015',

which can cause problems (e.g, schema violation when not turned back into date) when trying to save via bulbs

LINKLIST/SET properties are serialized as

u'cast': u'com.tinkerpop.blueprints.impls.orient.OrientElementIterable@3949de91',

which is completely unworkable. The only workaround I know is judicious use of server side gremlin scripts.

Bulbs doesn't seem to have access to the "@class" property so you can't know the class of a record without adding another field to record that, I think this is a problem at the rexster level though I am unsure.

The date and embedded set/list problems can easily be fixed by subclassing the Property class in

https://github.com/espeed/bulbs/blob/master/bulbs/property.py

and creating custom conversion to/from python/orientdb.

[Kevin I]

Thank you very much ky...! I've installed pyorient.

But now, what is this class name here :

CREATE INDEX <name> ON <class-name> (prop-names) FULLTEXT ENGINE LUCENE

as found in this wiki here? I tried using the Bulbs model class name and obviously it didn't work. Do I have to define separate classes for this? If so, can you please show how to do it?

Thanks!

[ky...@ovguideinc.com]

<class-name> refers to classes defined in Orientdb.

http://orientdb.com/docs/last/orientdb.wiki/Tutorial-Classes.html

Useful default classes are "V" for vertices, and "E" for edges.

You can define a new classes like:

create class MyV extends V

create class MyE extends E

you can do

create property V.element_name STRING

select from MyV set element_name = "myv"

(all above are osql console commands)

to create an element name property, and populate all records of that class with a string representing that property (this relates to what I mentioned before about not seeing any way to access the "@class" field in bulbs).  

You can now create bulbs model classes that correspond to the classes in Orientdb itself. "element_name" is the default field name to associate graphdb records with bulbs python model classes, it can be changed if something else works better for you.

[Kevin I]

@kyle,

Sorry to get back so late. I can't quite wrap my head around this. From what you've said and I understand from the docs and from fiddling with the console, this is what I've understood:

1. In orientdb, all classes are identified with the `@CLASS` property just like the classes in Bulbs are identified with `element_type` property.

2. A Class cannot be changed in orientdb, once it is created.

I have the following classes in Bulbs for example : 

Student, Staff, Subject, Branch

Now, I need to create lucene indexes for which I cannot use Bulbs. So, I get into the Orientdb console. As you said, I tried creating the class `Student` in orientdb too, like this:

`create class Student extends V`

But, still all the above 4 bulbs class type objects are shown as `V` objects in orientdb. ie. the number of `Student` records in orientDB is 0.

So, in short, what I want to do is to convert the `V` type records to the `Student`, `Staff`, `Subject` and `Branch` type records in OrientDB, so that I can create indices in Lucene for my search module.

Also, I tried your command : 

`select from MyV set element_name = "myv"`

but it throwed a syntax error.

Thank you for your help so far. Looking ahead for your reply!

--

---
You received this message because you are subscribed to a topic in the Google Groups "OrientDB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/orient-database/Oi34bWhARAU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to orient-databa...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Always remember that the world around you is made by people that are no smarter than you and me.

[Kyle]

You are right, 

select from MyV set element_name = "myv"

should be 

update MyV set element_name = "myv"

sorry.

From within bulbs you can create a vertex of type "MyV" by doing 

In [9]: data = {"@class":"MyV","ovgid":"test1", "element_type":"MyV"}
In [10]: v=g.vertices.create(data)

[Kevin I]

Thanks kyle!

Unfortunately, when I try to create the property `@class` like this :

`data = {"@class":"MyV","ovgid":"test1", "element_type":"MyV"}`,

I get the following error:

`

SystemError: ({'status': '500', 'transfer-encoding': 'chunked', 'server': 'grizzly/2.2.16', 'connection': 'close', 'date': 'Wed, 22 Apr 2015 07:09:12 GMT', 'access-control-allow-origin': '*', 'content-type': 'application/json'}, '{"message":"","error":"javax.script.ScriptException: com.orientechnologies.orient.core.exception.OSchemaException: Cannot change the schema while a transaction is active. Schema changes are not transactional","api":{"description":"evaluate an ad-hoc Gremlin script for a graph.","parameters":{"returnTotal":"when set to true, the full result set will be iterated and the results returned (default is false)","rexster.returnKeys":"an array of element property keys to return (default is to return all element properties)","rexster.showTypes":"displays the properties of the elements with their native data type (default is false)","load":"a list of \'stored procedures\' to execute prior to the \'script\' (if \'script\' is not specified then the last script in this argument will return the values","rexster.offset.end":"end index for a paged set of data to be returned","rexster.offset.start":"start index for a paged set of data to be returned","params":"a map of parameters to bind to the script engine","language":"the gremlin language flavor to use (default is groovy)","script":"the Gremlin script to be evaluated"}},"success":false}')

`

But, if I change `@class` to something else, it works. Any solution?

And also what is that `ovgid` property?

[Kevin I]

Ok! I tried `@CLASS` instead of `@class` and it didn't show up any error. But even now, the new vertices do not have the value `MyV` but `V`.

I mean,  I did `@CLASS = MyV` but after creation, if I see the vertices via rexster, all those records have `@CLASS = V`.

I even tried `_type = myv` and `_type = MyV` but the same problem. This is depressing. Any solution?

Thanks.

[Kyle]

Errors happen when you change the schema after starting rexster and connecting in bulbs.

If you restart rexster that specific error will probably stop.

I haven't figured out this problem completely, I think it has something to do with bulbs creating a transaction when you connect?? 

If you make any progress on figuring out how to deal with this I would appreciate you sharing that info.

-------------

You won't be able to change the class of the vertex from bulbs. 

move vertex from the console could possibly work.

------------

walkthrough:

orientdb> create database plocal:/tmp/foodb admin admin
Creating database [plocal:/tmp/foodb] using the storage type [plocal]...
Database created successfully.
Current database is: plocal:/tmp/foodb
orientdb {db=foodb}>
orientdb {db=foodb}> create class MyV extends V
Class created successfully. Total classes in database now: 11

change the path to something that makes sense for you.

configure rexster to see that database.

create a model in python code:

from bulbs.model import Node
class MyV(Node):
    element_type = "myv"

get a bulbs graph object from rexster somehow (I'll use the name "g") 

use add_proxy so that the bulbs graph object g knows about your models: 

g.add_proxy("myv", MyV)

20:51:31 In [38]: g.vertices.create({"element_type":"myv", "@class":"MyV", "myid":1})
20:51:55Out [38]: <MyV: http://localhost:8182/graphs/v2/vertices/#11:4>

checking back on console:

orientdb> connect remote:localhost/foodb admin admin
Connecting to database [remote:localhost/foodb] with user 'admin'...OK
orientdb {db=foodb}> select from V where myid = 1
----+-----+------+----+------------
#   |@RID |@CLASS|myid|element_type
----+-----+------+----+------------
0   |#11:4|MyV   |1   |myv
----+-----+------+----+------------

[Kevin I]

@Kyle,

Yeah, it did stop throwing the error, probably since I restarted Rexster. I don't think `@class` (lowercase) has anything to do with the OrientDB class property, if I'm not wrong. I think its `@CLASS`.

I followed all your steps. I created the `MyV` class in the console, and recreated the vertices like this:

`g.vertices.create({"element_type":"myv", "@class":"MyV", "@CLASS":"MyV"})`    // Just to make sure.

Still, the vertex looks like this:

{

  "element_type": "myv",

  "@CLASS": "V"                      // And not `MyV`

}                                              // There is no property named `@class` created

It is like I can't override the value of `@CLASS`. And I'm sure that I have the class `MyV` (subclass of `V`) created via the OrientDB console.

Has anyone ever done it before?

[Kevin I]

It's also weird that a couple (out of 20) edges Models (in Bulbs) are automatically shown as classes in orientDB:

CLASSES

----------------------------------------------+------------------------------------+------------+----------------+

 NAME                                         | SUPERCLASS                         | CLUSTERS   | RECORDS        |

----------------------------------------------+------------------------------------+------------+----------------+

 _studio                                      |                                    | 11         |              2 |

 E                                            |                                    | 10         |              0 |

 has_author                                   | E                                  | 13         |              0 |                           --> This one

 has_user_level                               | E                                  | 12         |            103 |                        --> And this one

 OFunction                                    |                                    | 6          |              0 |

 OIdentity                                    |                                    | -          |              0 |

 ORestricted                                  |                                    | -          |              0 |

 ORIDs                                        |                                    | 8          |              0 |

 ORole                                        | OIdentity                          | 4          |              3 |

 OSchedule                                    |                                    | 7          |              0 |

 OTriggered                                   |                                    | -          |              0 |

 OUser                                        | OIdentity                          | 5          |              4 |

 Room                                         | V                                  | 15         |              0 |

 Student                                      | V                                  | 14         |              0 |

 V                                            |                                    | 9          |            814 |

----------------------------------------------+------------------------------------+------------+----------------+

 TOTAL = 15                                                                                                  926 |

----------------------------------------------+------------------------------------+------------+----------------+

If this is possible, why isn't possible for the vertices?

But why are the other edges not shown as OrientDB classes here? It would be nice if @Luca can help with this!

[Kyle]

I am using orientdb 2.0.7, rexster 2.6

> I don't think `@class` (lowercase) has anything to do with the OrientDB class property,

All through the documentation the class property is shown as "@class".

If you do the query 

select from V where @class = "MyV"

it will return the same as 

select from MyV 

> // There is no property named `@class` created

this is the problem I mentioned earlier of not knowing how to access the orientdb @class from bulbs, I think it is not exposed to rexster.

> I followed all your steps. I created the `MyV` class in the console, and recreated the vertices like this:

To make sure everything is correct you should redo all those steps on a completely new database.

> And I'm sure that I have the class `MyV` (subclass of `V`) created via the OrientDB console

Your 

list classes

output in your second post does not show a "MyV" class.

that 

> "@CLASS": "V"    

is very strange. 

> It's also weird that a couple (out of 20) edges Models (in Bulbs) are automatically shown as classes in orientDB:

I'm not sure why this is, my first guess would be those classes were auto-created when you created some edges.