Cloud Datastore Emulator support?

[Jon Grover]

First, my apologies: I realize this is a bit off-topic. I'm running into a mysterious issue w/ the Cloud Datastore Emulator during a migration from NDB to Cloud NDB. I don't think the issue is related to Cloud NDB in particular, but I'm not sure where to turn for help. I posted a more detailed description in the Google Cloud Datastore Googlegroup here, but it appears to be much less active than this group.

The nutshell version is that I am calling requests.post('http://localhost:8089/reset') in the setUp() method for each of my tests which involve NDB in order to clear the datastore . This works when I run any one test file individually, but when I try to run our full test suite of 580 tests we start seeing an error about halfway through. The error indicates that it wasn't able to create any new connections to the URL, and once it first appears all subsequent tests fail with the same error. 

Traceback (most recent call last):
  File "test/my_test.py", line 11, in setUp
  File "/Projects/tilegames/test/_base_test.py", line 50, in setUp
  File "/Projects/tilegames/lib/requests/api.py", line 119, in post
  File "/Projects/tilegames/lib/requests/api.py", line 61, in request
  File "/Projects/tilegames/lib/requests/sessions.py", line 530, in request
  File "/Projects/tilegames/lib/requests/sessions.py", line 643, in send
  File "/Projects/tilegames/lib/requests/adapters.py", line 516, in send
ConnectionError: HTTPConnectionPool(host='localhost', port=8089): Max retries exceeded with url: /reset (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x11286e590>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',))

My initial efforts to troubleshoot what's going on here have all failed and I'm not sure where the best place is to turn for support. I suppose Stack Overflow is always an option. Anyone here have thoughts?

[David Do]

Hello Jon,

Unfortunately, since this is not the correct group for providing support for the Datastore Emulator, I suggest that you focus on monitoring the thread you have already opened in the Google Cloud Datastore group.

Alternatively, I agree that it would be a good idea to post your issue on Stack Overflow as you have mentioned, and you may also consider using Issue Tracker. From there, you would either consult existing Datastore issues or create your own entry describing the Datastore emulator test issue you are encountering.

[Jon Grover]

Those links are very helpful. Thank you, David!

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/SL8Cmm-304k/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/dacf6ad8-386e-42c7-ac79-136e13e7ef09%40googlegroups.com.

[Ryan B]

hi again, jon! silly question, but are you also overriding tearDown() and calling self.ndb_context.__exit__(None, None, None) there? if not, i wonder if all of the previous tests' contexts are holding onto open connections to the datastore emulator and exhausting a connection pool somewhere. seems unlikely, but maybe possible.

[Jon Grover]

Not a silly question at all! Yes, I'm exiting the context. My base test code looks like this:

import requests
import unittest
import webapp2

from google.cloud import ndb

from test._testhelper import TestHelper

NDB_CLIENT = ndb.Client(project='tilegames2')
NDB_CLIENT.host = 'localhost:8089'
NDB_CLIENT.secure = False

class MyTestCase(unittest.TestCase):
 def setUp(self):
   self.testapp = webapp2.WSGIApplication([])
   self.testHelper = TestHelper()
   # clear datastore

   requests.post('http://localhost:8089/reset')

   self.ndb_context = NDB_CLIENT.context()
   self.ndb_context.__enter__()

 def tearDown(self):
   self.ndb_context.__exit__(None, None, None)