Accessing datastore from microservice

[Richard Cheesmar]

Are there any docs on accessing the datastore from a microservice that lives in the same app as other non service modules?

[Richard Cheesmar]

Standard Env, python2.7

[George (Cloud Platform Support)]

App Engine's Python standard runtime connects to Cloud Datastore using the NDB Client Library. The NDB Client Library provides persistent storage in a schemaless object datastore. It supports automatic caching, sophisticated queries, and atomic transactions.

You cannot use the Cloud Datastore client library with Python applications in the App Engine standard environment. You may look for more details in the "Cloud Datastore Overview" online document, as well as Using Cloud Datastore. 

[Richard Cheesmar]

I know how to use the ndb Client Library, that was not the question. The question was is there any documentation for accessing the datastore from a microservice using standard and python 2.7. It seemslike it can't be done without calling endpoints on my API modules which are all non worker python2.7 standard modules, which would then negate the point of having a microservice for some heavy lifting in the first place.

[Richard Cheesmar]

Ok, symlinks is the answer if you need to share the model definitions.

[sami...@google.com]

Hi Richard,

I am clarifying the point that George has made in his earlier post on the thread. Basically, Modules are services, so are microservices, they are all the same. I have to mention that the way to access Datastore from a microservice or let alone any service through App Engine Standard and Python 2.7 is to use the NDB client library. It is not necessary to call endpoints on your other services as any other service can also call the NDB client library. As for the reference to Symlinks, it is not very clear on how symlinks are related to your earlier questions.

[Richard Cheesmar]

Hi, Sami, how do I do stuff on say a model class called users that is defined outside of the micro service inside of the micro service using just the NDB client library? I have to have access to the model definition inside of the micro service to query or save an entity to that particular ndb.Model class.

After ~Googling and using stackoverflow to research this The only way isto symlink the model definitions into the microservice locally and when uploading to app engine those symlinks turn into copies of the class itself. Thus I can access the class directly from within the micro service.

If there is a better way, please do tell.

Thanks

[Julie (cloud platform support)]

This seems like a design/code issue which would be best posted on Stackoverflow. It would also be helpful to clarify why redefining your user class in GAE code when you are using NDB to get / put the data would not be sufficient. 

[Richard Cheesmar]

Ok, I'dd like to clear this up:

Without having a copy of the ndb.Model in your microservice you cannot access it from the ndb client library.

The point of the symlinks is to provide that access without having to copy the model definition in everytime you make a change to it.  Ok, you can cut and paste, but it's easier to symlink it. Of course when it is uploaded it is a static copy of the model definition, not a link...

Perhaps my question should have been more specific up front...

However, I think there is a clear use case for sharing models amongst micro services in the same application, without having to share the model amongst all those services that use it. This is problematic as far as maintainence is concerned.

Does that clear my thinking up?

[Olu]

Quite rightly, with NDB Client Library, it would be necessary to have the NDB model in your Service. And I agree with you about the validity of a use case for sharing models among services without having to share the model among all the services that use the model. However, there is presently no other implementation that allows such access to your Model definitions outside of your Application code (apart from using Symlinks locally and uploading to App Engine, that is). 

I have created a Feature Request with the product engineers and you can add more description to the feature request for better detailing of the request. Be sure to Star the issue tracker link for regular updates on the implementation of the feature, however there is no ETA available for this feature at this time.