Titan 0.3.0

Showing 1-36 of 36 messages
Titan 0.3.0 Matthias 3/7/13 2:27 AM
Hey guys,

just wanted to share with you an update on the Titan roadmap. We re-prioritized a bunch of features and decided that it was about time to remove some technical debt in the Titan core module. This turned out into a major rewrite Titan's internals which opened the door to adding some great new features. With that many changes, Titan 0.3.0 will be backwards incompatible, so we decided to do a 0.2.1 release first, which includes a bunch of bugfixes, the multi-module refactoring and other changes that we have added to master over the last two months. Titan 0.2.1-SNAPSHOT has been deployed to sonatype and will be released in two weeks.

Titan 0.3.0-SNAPSHOT currently lives in the "indexing" branch which indicates one of the major new features that will be coming in Titan 0.3.0: full-text indexing, numeric range indexing, and geospatial indexing for both vertices and edges. These advanced indexing capabilities are provided by ElasticSearch (http://www.elasticsearch.org/) and Lucene (http://lucene.apache.org/) which are now integrated into Titan and available as Titan modules. Similarly to storage backends, Titan abstract external indexes which allows it to interface with arbitrary indexing solutions. We chose Lucene for this initial release because its the most popular and most mature indexing system in the open source domain. Like BerkeleyDB, it is designed for single machine use. ElasticSearch is a fairly young but quickly maturing open source project build on top of Lucene that scales to multiple servers and is robust against failure. Hence, it is an ideal partner for Cassandra or Hbase.

To include one of the two external indexes into your project, you simply configure them in your properties file. For instance:

storage.index.search.backend=com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
storage.index.search.hosts=[LIST_OF_HOST_IPs]

Here, "search" is the name you give to your index. Since you can register multiple external indices with Titan, each is uniquely identified by its name.

Now, you can tell Titan to index certain keys for you:

graph.makeType().name('time').dataType(Long.class).indexed('search', Edge.class).indexed(Vertex.class).unique(Direction.OUT).makePropertyKey();
graph.makeType().name('location').dataType(Geoshape.class).indexed('search', Edge.class).indexed('search',Vertex.class.unique(Direction.OUT).group(PROPG).makePropertyKey();

So, you use "indexed" to tell Titan that you want a particular key to be indexed and then you specify the name of the index you want to use ("search") and whether you want to index vertices or edges. That's right, Titan supports edge indexing now. If you leave out the name of the index (as we do for the 'time' key and vertices), then it defaults to Titan's "standard" indexing mechanism, namely using the underlying storage backend for indexing. This is what Titan provided all along for equality lookups.
Note, that Titan has its own geo-representation (Geoshape) which makes it easy for you to indexes vertices and edges by their location.

To query, you do:

g.query().has('time',Cmp.GREATER_THAN,System.currentTimeMillis()-1000000).has('location',Geo.WITHIN,Geoshape.circle(33.0,22.0,100)).edges()

which constitutes an "AND" query for all edges that have a timestamp larger than 1000 seconds ago and are within 100 km around that random geo point I inserted there.

I hope that gives you an idea for how this works. As with storage backends, all indexing handling is done by Titan and you can do everything at the level of the graph.

Since a lot of people have asked for this feature, I thought you might want to take a look at Titan 0.3.0-SNAPSHOT and play around with it to give us some feedback on this new feature. Note, that Titan 0.3.0 is not yet stable as we are still tinkering with the interface and sorting out some hyper threading issues.
Other things that are new in 0.3.0:
- use "unique" in type definitions to mark labels and keys as functional (i.e. unique(Direction.OUT)). That allows us to remove that mathematical "functional".
- complete rewrite of the caching engine which is now much better about caching vertex centric query results
- better byte representation and lazy de-serialization for better performance
- better query optimization and query rewriting for both vertex centric queries and global graph queries
- Edge now longer extends Vertex. Access to unidirectional edges through get/setProperty
- Properties on vertices can have properties on them (mind boggling...) which is very useful for version, timestamping, etc

So, lots of new stuff. Please check it out and let us know if you find any issues.

Thanks,
Matthias



--
Matthias Broecheler
http://www.matthiasb.com
Re: [Aurelius] Titan 0.3.0 Pablo Pareja 3/7/13 3:38 AM
Great!

Really looking forward to the 0.3.0 version ;)

Pablo

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Pablo Pareja Tobes 
Re: Titan 0.3.0 subhankar 3/7/13 4:00 AM
awesome......when is the projected time to release 0.3....eagerly waiting for it.......

and congrats for the release...or should i say thanks ... :)
Re: [Aurelius] Re: Titan 0.3.0 Matthias 3/7/13 11:20 AM
Planning to release one week after 0.2.1 to avoid confusion. So, in about 3 weeks.

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 


Re: [Aurelius] Titan 0.3.0 Marko Rodriguez 3/7/13 3:32 PM
Hi,

TinkerPop 2.3.0 (release in ~1.5 weeks) will have Graph.query() … It will slowly consume KeyIndexableGraph.

Marko.


On Mar 7, 2013, at 3:46 PM, Bryn Cooke <bryn...@gmail.com> wrote:

This looks great. I can't wait to start using it.

Out of interest, do you think that the query API will eventually make it into Blueprints?

Bryn

Re: Titan 0.3.0 Eric Scrivner 3/7/13 4:45 PM
Sounds fantastic. Can't wait to start experimenting with 0.2.1 / 0.3.0.
Re: Titan 0.3.0 Jonathan Haddad 3/7/13 4:54 PM
This sounds pretty amazing.  Since the new version will be incompatible with .2, does that mean you'll be including a migration script, or will it require a dump & reload of the graph?

On Thursday, March 7, 2013 2:27:57 AM UTC-8, Matthias wrote:
Re: [Aurelius] Titan 0.3.0 defermat 3/7/13 4:07 PM
Awesome stuff.  Looking forward to a lot of the new features.  Is the
indexing going to allow for additional indices to be added after the
graph has been created?

Charlie
Re: [Aurelius] Re: Titan 0.3.0 Matthias 3/7/13 5:15 PM
We are working on making Faunus capable to doing upgrades like that. Titan 0.3 will have a different disk representation of edges so a migration script is not feasible for distributed deployments. The upgrade process will not be available with Titan 0.3 immediately, hence the dual release.

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 


Re: [Aurelius] Titan 0.3.0 Matthias 3/7/13 5:20 PM
So, you can add new index providers but types will continue to be unmodifiable (i.e. cannot change their indexing properties) until a future version. Updating types is a tricky business that will keep us busy for a little while longer....
Re: [Aurelius] Titan 0.3.0 defermat 3/7/13 5:40 PM
Got it, sounds good.  Thanks.
Re: Titan 0.3.0 subhankar 3/8/13 4:14 AM
could not start titan+cas embbedded while adding ES plugin
my titan config is---

storage.backend=embeddedcassandra
storage.cassandra-config-dir=file:///home/justdial/work/TITAN/titan-indexing/config/cassandra.yaml
storage.keyspace=GraphTitan
storage.connection-pool-size=96
storage.read-consistency-level=QUORUM
storage.write-consistency-level=QUORUM
storage.replication-factor=1
ids.block-size=100000
storage.index.search.backend=com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
storage.index.search.hosts=127.0.0.1

and error is 
13/03/08 17:49:05 INFO es.ElasticSearchIndex: Configured remote host: 127.0.0.1 : 9300
13/03/08 17:49:05 ERROR service.CassandraDaemon: Exception in thread Thread[main,5,main]
java.lang.IllegalArgumentException: Could not instantiate implementation: com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
at com.thinkaurelius.titan.diskstorage.Backend.getImplementationClass(Backend.java:268)
at com.thinkaurelius.titan.diskstorage.Backend.getIndexes(Backend.java:239)
at com.thinkaurelius.titan.diskstorage.Backend.<init>(Backend.java:98)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:413)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:62)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
at com.thinkaurelius.titan.tinkerpop.rexster.RexsterTitanServer.start(RexsterTitanServer.java:59)
at com.thinkaurelius.titan.tinkerpop.rexster.RexsterTitanServer.startDaemon(RexsterTitanServer.java:69)
at com.thinkaurelius.titan.tinkerpop.rexster.RexsterTitanServer.main(RexsterTitanServer.java:106)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at com.thinkaurelius.titan.diskstorage.Backend.getImplementationClass(Backend.java:257)
... 8 more
Caused by: org.elasticsearch.client.transport.NoNodeAvailableException: No node available
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:202)
at org.elasticsearch.client.transport.support.InternalTransportClusterAdminClient.execute(InternalTransportClusterAdminClient.java:85)
at org.elasticsearch.client.support.AbstractClusterAdminClient.health(AbstractClusterAdminClient.java:79)
at org.elasticsearch.action.admin.cluster.health.ClusterHealthRequestBuilder.doExecute(ClusterHealthRequestBuilder.java:87)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:62)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:57)
at com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex.<init>(ElasticSearchIndex.java:146)
... 13 more
13/03/08 17:49:05 INFO thrift.ThriftServer: Stop listening to thrift clients
13/03/08 17:49:05 INFO gms.Gossiper: Announcing shutdown
13/03/08 17:49:06 INFO net.MessagingService: Waiting for messaging service to quiesce
13/03/08 17:49:06 INFO net.MessagingService: MessagingService shutting down server thread.
 
Re: [Aurelius] Re: Titan 0.3.0 Matthias 3/8/13 10:39 AM
Hey,

with this configuration, you need to run ElasticSearch before starting titan because you configured it for remote connection but there is no elastic search server running locally.

Change the configuration to:

storage.index.search.backend=com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
storage.index.search.local-mode=true
storage.index.search.client-only=false
storage.index.search.directory=/tmp/es

This should run it embedded.
HTH,
Matthias

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Matthias Broecheler
http://www.matthiasb.com
Re: [Aurelius] Re: Titan 0.3.0 subhankar 3/8/13 11:21 AM
i thought it would start es as it runs cassandra ....
it is working now
which is advisable local or remote mode....
anyway we r planning to deploy ES for our other search applications in near future then is it better to deploy ES in cluster mode from now.....

and thanks for help.....
Re: [Aurelius] Re: Titan 0.3.0 Matthias 3/8/13 11:45 AM
ElasticSearch is a pretty new component for us, so my advice on that is not yet solid.
For now, I would advise you to run elastic search in remote mode, in particular if you are running with cassandra embedded. There might be resource interference between cassandra and ES.
Re: [Aurelius] Re: Titan 0.3.0 subhankar 3/8/13 11:48 AM
thanks.....
Re: [Aurelius] Re: Titan 0.3.0 Zack Maril 3/9/13 1:37 AM
Woah. Did you guys get rid of TitanInMemoryBlueprintsGraph? Trying to update hermes and I got blindsided by all the changes to the internals. Could you put up the javadoc for the 0.3.0-SNAPSHOT someplace? 
-Zack
Re: [Aurelius] Re: Titan 0.3.0 Zack Maril 3/9/13 6:10 AM
When you say 0.2.1-SNAPSHOT has been deployed but not released, do you mean that the poms are there and not the jars? Because the jars aren't there currently and I wanted to check to see if that's right (0.3.0-SNAPSHOTS jars are there already).
-Zack
Re: [Aurelius] Re: Titan 0.3.0 Matthias 3/9/13 3:59 PM
Hey Zack,

what I meant to say is that we have deployed both 0.2.1 and 0.3.0 to sonatype so you guys can play with it but its not yet officially released (i.e. still snapshot) as things are still changing.

Yes, a lot of things have changed in 0.3.0 and one of the things that I dropped was InMemoryGraph. I plan on adding a storage backend which is just a sorted hashmap which would give you the same behavior as an InMemoryGraph (but with transactions) and is super trivial to implement (just an implementation of KeyValueStore that uses a ConcurrentSkipListMap), however, I haven't gotten around to that. InMemoryGraph was this special class floating around which made the code unpretty.

Cheers
Mathtias
Re: [Aurelius] Re: Titan 0.3.0 Jonathan Haddad 3/9/13 6:50 PM
Gotcha.  We only deployed our first Titan install about a month ago, and are only working with about 1/2 a gig of data - it should still be possible for us to dump our graph as a graphson file, then reload into titan 0.3, right?

Jon
Re: [Aurelius] Re: Titan 0.3.0 Matthias 3/9/13 6:58 PM
Yes, absolutely.
Re: [Aurelius] Re: Titan 0.3.0 Marko Rodriguez 3/9/13 7:20 PM

Hi,

Few things on this.

  1. Make sure you don't have non-primitive property values as Faunus doesn't support them yet. If you do, you can always transfer them over afterwards using this technique: https://github.com/thinkaurelius/faunus/wiki/Distributed-Graph-Computing-with-Gremlin
  2. Make use of a SequenceFile instead of GraphSON for speed. Though, with GraphSON you have an easily parseable backup to use with other tools if need be. Also, with GraphSON, ints are longs.
  3. Make sure to fully set up your indices in Titan 0.3.0 prior to bulk load and don't rely on schema inferencing. Faunus schema inferencing is not type-smart.
  4. Always check the statistics of the graph afterwards to make sure everything came across. Counters to watch for on the final BlueprintsOutputMapReduce is NULL_VERTICES and NULL_VERTEX_EDGES.

Have been dealing with this lately...

Good luck,
Marko.

http://markorodriguez.com

Re: [Aurelius] Re: Titan 0.3.0 Zack Maril 3/10/13 1:02 AM
Re: [Aurelius] Re: Titan 0.3.0 Matthias 3/10/13 9:14 PM
My intuition would be that those should both have jars. The process for deploying is exactly the same...
Not sure why there is a difference.
Re: [Aurelius] Re: Titan 0.3.0 Zack Maril 3/11/13 8:39 AM
Yeah, I'm having trouble getting the 0.2.1-SNAPSHOT to download at all. Maven can find the pom's no problem but it gets stuck trying to find the jar's on sonatype. I'm using this as the artifact identifier: 
[com.thinkaurelius.titan/titan "0.2.1-SNAPSHOT"] (Clojure way of doing maven) 

I don't think the jar's have been deployed anywhere for 0.2.1-SNAPSHOT. 
-Zack 
Re: [Aurelius] Re: Titan 0.3.0 Matthias 3/13/13 9:39 PM
Mmh, I will try to redeploy this.

On Mon, Mar 11, 2013 at 8:39 AM, Zack Maril <thewi...@gmail.com> wrote:
Yeah, I'm having trouble getting the 0.2.1-SNAPSHOT to download at all. Maven can find the pom's no problem but it gets stuck trying to find the jar's on sonatype. I'm using this as the artifact identifier: 
[com.thinkaurelius.titan/titan "0.2.1-SNAPSHOT"] (Clojure way of doing maven) 

I don't think the jar's have been deployed anywhere for 0.2.1-SNAPSHOT. 
-Zack 


On Monday, March 11, 2013 8:14:43 AM UTC+4, Matthias wrote:
My intuition would be that those should both have jars. The process for deploying is exactly the same...
Not sure why there is a difference.

On Sun, Mar 10, 2013 at 1:02 AM, Zack Maril <thewi...@gmail.com> wrote:
Okay! So, my main question is should there be jar's in this folder:
The way there are jar's in this folder:
-Zack


On Sunday, March 10, 2013 3:59:05 AM UTC+4, Matthias wrote:
Hey Zack,

what I meant to say is that we have deployed both 0.2.1 and 0.3.0 to sonatype so you guys can play with it but its not yet officially released (i.e. still snapshot) as things are still changing.

Yes, a lot of things have changed in 0.3.0 and one of the things that I dropped was InMemoryGraph. I plan on adding a storage backend which is just a sorted hashmap which would give you the same behavior as an InMemoryGraph (but with transactions) and is super trivial to implement (just an implementation of KeyValueStore that uses a ConcurrentSkipListMap), however, I haven't gotten around to that. InMemoryGraph was this special class floating around which made the code unpretty.

Cheers
Mathtias


On Sat, Mar 9, 2013 at 6:10 AM, Zack Maril <thewi...@gmail.com> wrote:
When you say 0.2.1-SNAPSHOT has been deployed but not released, do you mean that the poms are there and not the jars? Because the jars aren't there currently and I wanted to check to see if that's right (0.3.0-SNAPSHOTS jars are there already).
-Zack

On Saturday, March 9, 2013 1:37:29 PM UTC+4, Zack Maril wrote:
Woah. Did you guys get rid of TitanInMemoryBlueprintsGraph? Trying to update hermes and I got blindsided by all the changes to the internals. Could you put up the javadoc for the 0.3.0-SNAPSHOT someplace? 
-Zack

On Friday, March 8, 2013 11:48:20 PM UTC+4, subhankar biswas wrote:
thanks.....

On Saturday, 9 March 2013 01:15:35 UTC+5:30, Matthias wrote:
ElasticSearch is a pretty new component for us, so my advice on that is not yet solid.
For now, I would advise you to run elastic search in remote mode, in particular if you are running with cassandra embedded. There might be resource interference between cassandra and ES.

On Fri, Mar 8, 2013 at 11:21 AM, subhankar biswas <neo20...@gmail.com> wrote:
i thought it would start es as it runs cassandra ....
it is working now
which is advisable local or remote mode....
anyway we r planning to deploy ES for our other search applications in near future then is it better to deploy ES in cluster mode from now.....

and thanks for help.....


On Saturday, 9 March 2013 00:09:49 UTC+5:30, Matthias wrote:
Hey,

with this configuration, you need to run ElasticSearch before starting titan because you configured it for remote connection but there is no elastic search server running locally.

Change the configuration to:

storage.index.search.backend=com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
storage.index.search.local-mode=true
storage.index.search.client-only=false
storage.index.search.directory=/tmp/es

This should run it embedded.
HTH,
Matthias

On Fri, Mar 8, 2013 at 4:14 AM, subhankar biswas <neo20...@gmail.com> wrote:
could not start titan+cas embbedded while adding ES plugin
my titan config is---

storage.backend=embeddedcassandra
storage.cassandra-config-dir=file:///home/justdial/work/TITAN/titan-indexing/config/cassandra.yaml
storage.keyspace=GraphTitan
storage.connection-pool-size=96
storage.read-consistency-level=QUORUM
storage.write-consistency-level=QUORUM
storage.replication-factor=1
ids.block-size=100000
storage.index.search.backend=com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
storage.index.search.hosts=127.0.0.1

and error is 
13/03/08 17:49:05 INFO es.ElasticSearchIndex: Configured remote host: 127.0.0.1 : 9300
13/03/08 17:49:05 ERROR service.CassandraDaemon: Exception in thread Thread[main,5,main]
java.lang.IllegalArgumentException: Could not instantiate implementation: com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
at com.thinkaurelius.titan.diskstorage.Backend.getImplementationClass(Backend.java:268)
at com.thinkaurelius.titan.diskstorage.Backend.getIndexes(Backend.java:239)
at com.thinkaurelius.titan.diskstorage.Backend.<init>(Backend.java:98)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:413)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:62)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
at com.thinkaurelius.titan.tinkerpop.rexster.RexsterTitanServer.start(RexsterTitanServer.java:59)
at com.thinkaurelius.titan.tinkerpop.rexster.RexsterTitanServer.startDaemon(RexsterTitanServer.java:69)
at com.thinkaurelius.titan.tinkerpop.rexster.RexsterTitanServer.main(RexsterTitanServer.java:106)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at com.thinkaurelius.titan.diskstorage.Backend.getImplementationClass(Backend.java:257)
... 8 more
Caused by: org.elasticsearch.client.transport.NoNodeAvailableException: No node available
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:202)
at org.elasticsearch.client.transport.support.InternalTransportClusterAdminClient.execute(InternalTransportClusterAdminClient.java:85)
at org.elasticsearch.client.support.AbstractClusterAdminClient.health(AbstractClusterAdminClient.java:79)
at org.elasticsearch.action.admin.cluster.health.ClusterHealthRequestBuilder.doExecute(ClusterHealthRequestBuilder.java:87)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:62)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:57)
at com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex.<init>(ElasticSearchIndex.java:146)
... 13 more
13/03/08 17:49:05 INFO thrift.ThriftServer: Stop listening to thrift clients
13/03/08 17:49:05 INFO gms.Gossiper: Announcing shutdown
13/03/08 17:49:06 INFO net.MessagingService: Waiting for messaging service to quiesce
13/03/08 17:49:06 INFO net.MessagingService: MessagingService shutting down server thread.
 

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Matthias Broecheler
http://www.matthiasb.com

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Matthias Broecheler
http://www.matthiasb.com

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Matthias Broecheler
http://www.matthiasb.com

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Matthias Broecheler
http://www.matthiasb.com

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Matthias Broecheler
http://www.matthiasb.com
Re: [Aurelius] Re: Titan 0.3.0 Zack Maril 3/18/13 5:32 AM
Was rereading this message:
- Properties on vertices can have properties on them (mind boggling...) which is very useful for version, timestamping, etc

I've been considering how to make Titan immutable at some level. While it is currently possible within an application with very careful management of how you interact with Titan, I have a feeling the above would allow someone to really easily write some general code that would allow anyone to interact with an effectively immutable graph database (which seems counterintuitive at first, but ends up being quick useful). It would be very similar to datomic:

Neat!
-Zack
Re: [Aurelius] Re: Titan 0.3.0 Matthias 3/18/13 8:59 AM
Hey Zack,

I think of Titan as an immutable database but in a slightly different sense. Edges and properties in Titan cannot be modified once they are created but can be deleted. Hence, every graph mutation is translated into a deletion and addition operation.

Imho, being able to delete is an important aspect of any serious database system. There are many applications where being able to delete is an important requirement. Furthermore, "forgetting" is an important feature of our brain (and not a limitation as some might think) and hence I believe that any semi-intelligent system must have the ability to "forget", i.e. delete things.
That's why I think Titan is immutable.

Cheers,
Matthias
Re: Titan 0.3.0 Sergey L 3/18/13 11:52 AM
Why yor choose ES instead Solr - Solr more stable and sync release with Lucene?
Re: [Aurelius] Re: Titan 0.3.0 Matthias 3/18/13 12:23 PM
Hey Sergey,

check out:

That said, note that it is fairly simple to write a Solr adapter for Titan - requires only one class implementation.

Best,
Matthias

On Mon, Mar 18, 2013 at 11:52 AM, Sergey L <fknis...@gmail.com> wrote:
Why yor choose ES instead Solr - Solr more stable and sync release with Lucene?

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Matthias Broecheler
http://www.matthiasb.com
Re: [Aurelius] Re: Titan 0.3.0 Sergey L 3/19/13 2:13 PM
Ok, thanks.
Re: Titan 0.3.0 AN 3/25/13 8:20 AM
I am very interested in integrating ElasticSearch's faceted search and had found a previous thread on this list that deferred it to a future release of Titan that would integrate ES.

It sounds like ES is very close and just wanted to confirm that the current snapshot of Titan has some minimal ES search that I could build upon.  

You mention, "We chose Lucene for this initial release because its the most popular and most mature indexing system in the open source domain," which sounds to me that only Lucene is currently supported?

Also, I do not have any direct experience with ES's faceted search (this will be my first project with it) so please let me know if there might be some inherit issue in pursuing this path when integrating with Titan.

Thanks,
Andrew

On Thursday, March 7, 2013 2:27:57 AM UTC-8, Matthias wrote:
Hey guys,

just wanted to share with you an update on the Titan roadmap. We re-prioritized a bunch of features and decided that it was about time to remove some technical debt in the Titan core module. This turned out into a major rewrite Titan's internals which opened the door to adding some great new features. With that many changes, Titan 0.3.0 will be backwards incompatible, so we decided to do a 0.2.1 release first, which includes a bunch of bugfixes, the multi-module refactoring and other changes that we have added to master over the last two months. Titan 0.2.1-SNAPSHOT has been deployed to sonatype and will be released in two weeks.

Titan 0.3.0-SNAPSHOT currently lives in the "indexing" branch which indicates one of the major new features that will be coming in Titan 0.3.0: full-text indexing, numeric range indexing, and geospatial indexing for both vertices and edges. These advanced indexing capabilities are provided by ElasticSearch (http://www.elasticsearch.org/) and Lucene (http://lucene.apache.org/) which are now integrated into Titan and available as Titan modules. Similarly to storage backends, Titan abstract external indexes which allows it to interface with arbitrary indexing solutions. We chose Lucene for this initial release because its the most popular and most mature indexing system in the open source domain. Like BerkeleyDB, it is designed for single machine use. ElasticSearch is a fairly young but quickly maturing open source project build on top of Lucene that scales to multiple servers and is robust against failure. Hence, it is an ideal partner for Cassandra or Hbase.

To include one of the two external indexes into your project, you simply configure them in your properties file. For instance:

storage.index.search.backend=com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
storage.index.search.hosts=[LIST_OF_HOST_IPs]

Here, "search" is the name you give to your index. Since you can register multiple external indices with Titan, each is uniquely identified by its name.

Now, you can tell Titan to index certain keys for you:

graph.makeType().name('time').dataType(Long.class).indexed('search', Edge.class).indexed(Vertex.class).unique(Direction.OUT).makePropertyKey();
graph.makeType().name('location').dataType(Geoshape.class).indexed('search', Edge.class).indexed('search',Vertex.class.unique(Direction.OUT).group(PROPG).makePropertyKey();

So, you use "indexed" to tell Titan that you want a particular key to be indexed and then you specify the name of the index you want to use ("search") and whether you want to index vertices or edges. That's right, Titan supports edge indexing now. If you leave out the name of the index (as we do for the 'time' key and vertices), then it defaults to Titan's "standard" indexing mechanism, namely using the underlying storage backend for indexing. This is what Titan provided all along for equality lookups.
Note, that Titan has its own geo-representation (Geoshape) which makes it easy for you to indexes vertices and edges by their location.

To query, you do:

g.query().has('time',Cmp.GREATER_THAN,System.currentTimeMillis()-1000000).has('location',Geo.WITHIN,Geoshape.circle(33.0,22.0,100)).edges()

which constitutes an "AND" query for all edges that have a timestamp larger than 1000 seconds ago and are within 100 km around that random geo point I inserted there.

I hope that gives you an idea for how this works. As with storage backends, all indexing handling is done by Titan and you can do everything at the level of the graph.

Since a lot of people have asked for this feature, I thought you might want to take a look at Titan 0.3.0-SNAPSHOT and play around with it to give us some feedback on this new feature. Note, that Titan 0.3.0 is not yet stable as we are still tinkering with the interface and sorting out some hyper threading issues.
Other things that are new in 0.3.0:
- use "unique" in type definitions to mark labels and keys as functional (i.e. unique(Direction.OUT)). That allows us to remove that mathematical "functional".
- complete rewrite of the caching engine which is now much better about caching vertex centric query results
- better byte representation and lazy de-serialization for better performance
- better query optimization and query rewriting for both vertex centric queries and global graph queries
- Edge now longer extends Vertex. Access to unidirectional edges through get/setProperty
- Properties on vertices can have properties on them (mind boggling...) which is very useful for version, timestamping, etc

So, lots of new stuff. Please check it out and let us know if you find any issues.

Thanks,
Matthias



--
Matthias Broecheler
http://www.matthiasb.com
Re: [Aurelius] Re: Titan 0.3.0 Matthias 3/25/13 10:18 AM
Hey Andrew,

ElasticSearch and Lucene integration will come with Titan 0.3.0 which has been released as SNAPSHOT and is currently in the "indexing" branch of the repo. Both index providers will be supported.
For now, we will only support boolean search through Titan - that is, faceted search is not yet supported.

Best,
Matthias

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Matthias Broecheler
http://www.matthiasb.com
Re: Titan 0.3.0 Robo Beans 4/9/13 12:54 PM
Re: [Aurelius] Re: Titan 0.3.0 Robo Beans 4/9/13 12:58 PM
Hi Matthias,

If possible, could you please share more details about the class implementation for this solr adapter for titan? Thanks!
Re: [Aurelius] Re: Titan 0.3.0 Matthias 4/10/13 8:21 AM
Hey,

we don't have a solr adapter for Titan yet. Only ElasticSearch and Lucene are supported at this point.

Best,
Matthias
More topics »