how to manage datastore.client in my API

69 views
Skip to first unread message

Gregg Murray

unread,
Mar 19, 2020, 2:34:18 PM3/19/20
to google-appengine-go
Hello!

I am using the App Engine in datastore mode.   I am wondering about how to manage datastore.client in my functions and test functions.  
In the Below Example, is it correct to keep the context and client and keep passing them to datastore methods?  
Or should the client be created in each of the datastore methods, or is there some other best practice for re-using context and client?

FYI I'm aware this code doesn't function, it's just pseudocode to put my question in context.

func main() {
    http.HandleFunc("/", indexHandler)
    http.HandleFunc("/dosomething", dosomething)
...........
}

func dosomething(w http.ResponseWriter, r *http.Request) {

    //Create Context
    ctx := context.Background()

    // Creates a client.
    clierr := datastore.NewClient(ctx, ProjectID) //projectID
    if err != nil {
        log.Fatalf("Failed to create client: %v", err)
    }

e, err := NewObjectFromData(ctx, cli, r.FormValue("parameter"))
    if err != nil {
        log.Fatalf("Error Running NewObjectFromData %v", err)
    }

c, err:= e.FindSomeOfTheChildren(ctx, cli)
if err != nil {
        log.Fatalf("Error Running FindSomeOfTheChildren %v", err)
} //Do Something to e

//Save Changes to e
err = e.DBSave(ctx, cli)
if err != nil {
        log.Fatalf("Error Saving Object back to datastore %v", err)
}

    ro := Returnobject {}
   ro.Status = OK

        jerr := json.Marshal(ro)
        if err != nil {
            log.Fatalf("Error Parsing Json")
        }
    w.Write(j)
    return 
}

Chaoming Li

unread,
Mar 19, 2020, 6:51:58 PM3/19/20
to google-appengine-go
I was in a similar situation and I created my own Datastore Client struct to initialise the my client and wrap the functions I need inside, so I don't have to pass ctx and the actual client on every function call.

Gregg Murray

unread,
Mar 20, 2020, 1:46:50 PM3/20/20
to google-appengine-go
I like this solution but it does not answer the question of whether persisting a client is better than to keep creating a new one as needed ( I don't know enough about datastore/app engine load balancing to answer that one)

Also, How do you extract the Context from a Datastore.Client?  Was there an existing property or method that you utilized, or did you make the context a new property of your custom Client struct using composition?

type CustomClient struct  {
     Context context.Context
     datastore.Client

Prateek Malhotra

unread,
Mar 20, 2020, 4:27:54 PM3/20/20
to google-appengine-go
To answer your question, you just need to setup the datastore client once, not per call (unless you are using the deprecated appengine package)

Here's what I've been doing:
  1. Always start my server with
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()
    // Use ctx to initialize any clients
  2. Setup each server as a struct with clients for each service it needs to interact with
    type server struct {
        datastoreClient *datastore.Client // I actually never do this, this is usually part of any service clients I create, see below
        myServiceClient *myservice.Client // Here I'd set this up with something like myservice.NewClient(ctx, myservice.WithDatastoreClient(dsClient))
        // etc...
    }
  3. ALWAYS pass in the `http.Request.Context()` for all calls, including datastore calls! You can leverage middleware, tracing, and so many other useful things this way!
  4. I typically use gRPC w/ OpenAPI extensions to generate my server-code and use a server struct as above.
To modify your example:

func main() {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
s := &server{}
ds, err := datastore.NewClient(ctx, projectID)
if err != nil {
log.Fatalf("Failed to create client: %v", err)
}

{
cli, err := someservice.NewClient(ctx, someservice.WithDatastoreClient(ds))
if err != nil {
log.Fatalf("Failed to create client: %v", err)
}
s.myClient = cli
}
{
cli, err := otherserivce.NewClient(ctx, someservice.WithDatastoreClient(ds))
if err != nil {
log.Fatalf("Failed to create client: %v", err)
}
s.myOtherClient = cli
}

http.HandleFunc("/", s.indexHandler)
http.HandleFunc("/dosomething", s.dosomething)
}

type server struct {
myClient      *somservice.Client
myOtherClient *otherserivce.Client
}

func (s *server) dosomething(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()

e, err := s.myClient.NewObjectFromData(ctx, r.FormValue("parameter"))
if err != nil {
log.Fatalf("Error Running NewObjectFromData %v", err)
}

c, err := s.myClient.FindSomeOfTheChildren(ctx)
if err != nil {
log.Fatalf("Error Running FindSomeOfTheChildren %v", err)
}

//Do Something to e

//Save Changes to e
err = s.myClient.SaveObject(ctx, e)
if err != nil {
log.Fatalf("Error Saving Object back to datastore %v", err)
}

ro := Returnobject{}
ro.Status = OK

j, err := json.Marshal(ro)
if err != nil {
log.Fatalf("Error Parsing Json")
}
w.Write(j)
return
}


Thanks,
Prateek
Reply all
Reply to author
Forward
0 new messages