Web application using tinkerpop

[Mike Roberts]

Hi,

I repeatedly ask the same question every few months/years because I cannot find an answer:

I need some guidance about the best design patterns to encapsulate/wrap/hide the tinkerpop specific implementation.

I used Tinkerpop 2 + Frames and I don't know how to migrate to Tinkerpop 3.

is there a java open-source (web) applications that uses a graph database thru Tinkerpop?

Or would anyone give me some examples how to properly use tinkerpop in a well-designed and structured solution?

Thank you

[HadoopMarc]

Hi,

Can you first check the query languages section of the TinkerPop main page and say what you are still missing?

This contains various links to ORM frameworks that could easily be combined with a web framework of your choice. In addition, I think many graph systems build their own java API end points that simply embed TinkerPop implementations like janusgraph and convert API request into gremlin queries directly.

Best wishes,     Marc

Op woensdag 7 april 2021 om 09:50:32 UTC+2 schreef mik...@gmail.com:

[pieter gmail]

HI,

" I think many graph systems build their own java API end points..."

This is what we have, 

UI javascript (in our case Mithril) <--> Rest via Spark java, and from there its java, i.e. TinkerPop Graph API and whatever...

For our part, we avoid opening the backend, logic or data, directly to the UI, so no GraphQL, or TinkerPop JS. To my mind it would be a leaking of responsibility and a security risk.

Cheers

Pieter

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/0414eb7e-7270-4b97-961b-3b3a4a23a9cbn%40googlegroups.com.

[Mike Roberts]

Hi,

thank you for your reply.

I already checked out all the options listed in that page. All the Java implementations seem to be abandoned, incomplete or deprecated. Besides, after the unfortunate experience with Frames that we stuck with after it was brutally abandoned, I'd rather avoid to lock to another third-party "custom" implementation.

I'd prefer to learn how to use the standard Tinkerpop api in the most flexible, maintainable and decoupled way. We have a quite complex domain and I wouldn't like to write tons of g.V() 

There's something I cannot grasp, I am quite confused.

[Mike Roberts]

Hi,

thanks for the suggestion.

How do you wrap and expose the results from Tinkerpop? Do you map them to a DTO? How do you support graph traversal?

[David Bechberger]

What you are asking here is a bit unclear as I am unsure what answer you are looking for? 

To provide some specifics though I'll try and address your question on best design patterns to encapsulate the TP implementation.  From an architectural perspective, TinkerPop serves as the data persistence layer of your application in the same manner as other databases such as Postgres would.  This means that all standard best architectural practices that apply for those systems also apply for a TP enabled backend.  What exactly those practices are dependent on if you are using a tiered architecture, a monolith, microservices, etc.  

In general, the best practices for other databases apply here such as abstraction of the Tinkerpop specifics such as connection management, queries, data transformation to/from query results using something like the Repository Design pattern (or similar).  A few TP-specific items to consider are:

• TinkerPop drivers are built around an architecture of having a persistent connection.  Opening and closing a connection should be done at the lifespan of the application instead of per request.  Using something like a Singleton pattern for the Client object is an option here.
• For complex queries, like the one you mentioned, building up traversals to cover multiple actions and then iterating them only one time tends to provide better performance. (e.g. here)
• There are no fully complete OGMs for any language other than .NET (i.e. Gremlinq) so data mapping to/from result sets needs to be handled by the application
• Best practices for how to optimize connection parameters are going to vary based on the underlying database being used so I suggest checking the documentation for the underlying TP implementation.

I hope this helps answer some of your questions above.

Regards,

Dave

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/73b7b87f-dc5b-45fa-9e6a-83c50ca0e310n%40googlegroups.com.

[Stephen Mallette]

> All the Java implementations seem to be abandoned, incomplete or deprecated.

> There are no fully complete OGMs for any language other than .NET (i.e. Gremlinq) so data mapping to/from result sets needs to be handled by the application

I thought Ferma was pretty solid for Java in this context. It may not have a recent commit history but it's been around a while and i thought it more mature than abandoned. I can't recall if it supports remoting though.

I also thought Goblin was pretty strong for Python in the OGM space - that code sample seems pretty nice and of course it must support remoting so that's a plus:

https://goblin-ogm.com/

Dave, where there specific things you thought were lacking in these OGMs?

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CA%2BTXmsRZ7ag9afmdWOvdeEe14kJ3LxS62UzNiqVqtPnCdUphMQ%40mail.gmail.com.

[pieter gmail]

Hi,

Here is more on how we do it,

For our own application's domain we use UMLG, it is generated entities from UML class diagrams. I won't suggest using it though, it was a ok idea but alas it needs a new implementation that learnt all the lessons from the first one.

That said we use it extensively, probably about 200 classes which translates to 200 entities. Nice and type safe which makes a massive difference. 

So if there is some DTO framework out there its probably a good idea. Having labels hanging around as strings will probably turn into a mess and hardcoding the shape of the graph is even worse.

Outside our own domain we import endless external domains. We do not own this data and it comes in on the fly. TinkerPop's NoSql (lazy on the fly schema creation) comes in super handy. We do however maintain, in java, meta data that we persist to flat files. We use this meta data to know what vertex/edge labels and properties exists in the graph. The graph actually knows its own meta data, but we need additional information so we persist the meta data outside the graph.

Regarding exposing all this, we have nothing fancy.

graph.traversal.V().hasLabel("StartHere").out()....values("property1")....toList();

loop the list, place the data in jackson json and send it somewhere over http,

If its domain objects then it looks more like hibernate

User user = findUser(nameFromRequest); 

json.put(user.getUsername())

json.put(user.getEmail())

Address address = user.getAddress()

...

return json.toString()

Cheers

Pieter

[Mike Roberts]

Thank you David for the exaustive reply.

I know I am not very good in  exposing my doubts.

What I try to figure out is how to get rid of Tinkerpop 2 Frames that is the way we implemented our domain model interfaces, with the new Tinkerpop 3 (or should I wait for version 4? LOL).

Here is a simple example of what we have:

public interface UserEntity extends NameableDomainEntity {

    String getUsername();

    void setUsername(String username);

    Iterable<GroupEntity> getGroups();

}

public class UserEntityImpl extends AbstractNameableFramedDelegate<UserFrame> implements UserEntity {

    protected UserEntityImpl(UserFrame f) {

        super(f);

    }

    

    public String getUsername() {

        return (getDelegate() != null) ? getDelegate().getUsername() : null;

    }

    

    public Iterable<GroupEntity> getGroups() {

        return (getDelegate() != null) ? IdentityMapper.mapToIterableGroupEntity(getDelegate().getGroups()) : null;

    }

}

@TypeValue(UserFrame._TYPEVALUE)

public interface UserFrame extends NameableDomainFrame {

    final String _TYPEVALUE = "user";

    final String USERNAME = "username";

    final String IS_MEMBER_OF = "is_member_of";

    

    @Property(USERNAME)

    String getUsername;

    

    @Adjacency(label=IS_MEMBER_OF)

    public Iterable<GroupFrame> getGroups();

}

Now, I would like to replace the class UserFrame with a Tinkerpop 3 implementation and my doubts are: should I write plain gremlin queries for each method? Should I go with the Remote Server, create DSL o something else? What about mutations and transactions? Will I eventually end up writing code as we used to do back to the ugly JDBC days?

[Mike Roberts]

I did a POC with Ferma some years ago. There were some issues with the implementation that the developers never fixed and right now it seems no longer maintained.

I liked Microsoft spring-data-gremlin project, but they are going to deprecate it.

[Mike Roberts]

Thank you Pieter, I'll think about what you suggest

[Stephen Mallette]

Could you please be more specific about the Ferma issues? What was not working for you?

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/39459e75-68da-4117-b6a7-43ffe62b0c93n%40googlegroups.com.

[Mike Roberts]

Gosh, it happened several years ago and I'm very bad in remembering things ;P

I'll try to find the messages I exchanged with the lead developer.

Anyhow in the github project there are very old issues still open.

[Stephen Mallette]

well, perhaps I don't need specifics if it's a bother to figure out. i'd just offer that you consider reaching out to Jeffrey Freeman again if it's been several years since you have looked into Ferma and to not consider old issues or a relatively quiet commit log to mean definite abandonment. 

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/9b4a153f-7d63-424f-b601-0157f3ce3349n%40googlegroups.com.