Tilestache Config on S3 - ExternalConfigServer Help

48 views
Skip to first unread message

Alex King

unread,
May 21, 2017, 11:01:17 AM5/21/17
to Tilestache
I'm trying to throw the Tilestache config on S3 but am having issues.

I'm using elastic beanstalk and point the wsgi server to the following code:

import os, TileStache
application = TileStache.Goodies.ExternalConfigServer.WSGIServer('https://s3.amazonaws.com/omitted/tilestache.cfg')


However this returns "AttributeError: 'module' object has no attribute 'Goodies'"

When I try to do TileStache.Goodies.ExternalConfigServer:WSGIServer(), with a colon before WSGIServer, as is referenced in the docs I get an error right at the colon. 

Clearly this is probably a syntax error, but do you guys have any ideas?

Thanks,
Alex

jban...@deltageo.ca

unread,
Jul 12, 2017, 10:23:36 PM7/12/17
to Tilestache
You need to import TileStache.Goodies as well

import os
import TileStache
import TileStache.Goodies

application = TileStache.Goodies.ExternalConfigServer.WSGIServer('https://s3.amazonaws.com/omitted/tilestache.cfg')

Cheers,
James

Alex King

unread,
Jul 31, 2017, 7:44:46 PM7/31/17
to tiles...@googlegroups.com
Thanks for the help, James. Clearly a silly error on my part!

Question for you - now that I've corrected that, I get the following error which I don't get while running the config straight from the EC2 instance. Thoughts on what could be triggering it? It's definitely valid JSON, and it's not a permissions issue.
File "/opt/python/current/app/tilestache.py", line 4, in <module>
application = TileStache.Goodies.ExternalConfigServer.WSGIServer('https://s3.amazonaws.com/XXXXXXXX/config/tilestache.cfg')
File "/opt/python/run/venv/lib/python2.7/site-packages/TileStache/Goodies/ExternalConfigServer.py", line 127, in __init__
cache_dict = json_load(urlopen(url_root + "/cache"))
File "/usr/lib64/python2.7/json/__init__.py", line 291, in load
**kw)
File "/usr/lib64/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Best,
Alex


--
You received this message because you are subscribed to a topic in the Google Groups "Tilestache" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tilestache/8zNjOGKZEOY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tilestache+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

isbhod

unread,
Jul 2, 2018, 9:58:29 AM7/2/18
to Tilestache
Alex, 

Sorry to drag up the past, but did you ever get this to work and if so how?  I am working on something very similar and am getting the same errors:

$ gunicorn --bind localhost:8888 "TileStache.Goodies.ExternalConfigServer:WSGIServer('http://s3.amazonaws.com/*MYBUCKET*/tilestache.cfg')"
[2018-07-02 13:48:52 +0000] [26004] [INFO] Starting gunicorn 19.4.5
[2018-07-02 13:48:52 +0000] [26004] [INFO] Listening at: http://127.0.0.1:8888 (26004)
[2018-07-02 13:48:52 +0000] [26004] [INFO] Using worker: sync
[2018-07-02 13:48:52 +0000] [26008] [INFO] Booting worker with pid: 26008
[2018-07-02 13:48:52 +0000] [26008] [ERROR] Exception in worker process:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 515, in spawn_worker
    worker.init_process()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 122, in init_process
    self.load_wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 130, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/lib/python2.7/dist-packages/gunicorn/util.py", line 377, in import_app
    app = eval(obj, mod.__dict__)
  File "<string>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python2.7/site-packages/TileStache/Goodies/ExternalConfigServer.py", line 128, in __init__
    cache_dict = json_load(urlopen(url_root + "/cache"))
  File "/usr/lib/python2.7/json/__init__.py", line 291, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 515, in spawn_worker
    worker.init_process()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 122, in init_process
    self.load_wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 130, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/lib/python2.7/dist-packages/gunicorn/util.py", line 377, in import_app
    app = eval(obj, mod.__dict__)
  File "<string>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python2.7/site-packages/TileStache/Goodies/ExternalConfigServer.py", line 128, in __init__
    cache_dict = json_load(urlopen(url_root + "/cache"))
  File "/usr/lib/python2.7/json/__init__.py", line 291, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
[2018-07-02 13:48:52 +0000] [26008] [INFO] Worker exiting (pid: 26008)
[2018-07-02 13:48:52 +0000] [26004] [INFO] Shutting down: Master
[2018-07-02 13:48:52 +0000] [26004] [INFO] Reason: Worker failed to boot.

So then I tried loading it form a local file:

$ gunicorn --bind localhost:8888 "TileStache.Goodies.ExternalConfigServer:WSGIServer('http://s3.amazonaws.com/cimscloudbeta/N8test/tilestache.cfg')"
[2018-07-02 13:48:52 +0000] [26004] [INFO] Starting gunicorn 19.4.5
[2018-07-02 13:48:52 +0000] [26004] [INFO] Listening at: http://127.0.0.1:8888 (26004)
[2018-07-02 13:48:52 +0000] [26004] [INFO] Using worker: sync
[2018-07-02 13:48:52 +0000] [26008] [INFO] Booting worker with pid: 26008
[2018-07-02 13:48:52 +0000] [26008] [ERROR] Exception in worker process:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 515, in spawn_worker
    worker.init_process()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 122, in init_process
    self.load_wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 130, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/lib/python2.7/dist-packages/gunicorn/util.py", line 377, in import_app
    app = eval(obj, mod.__dict__)
  File "<string>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python2.7/site-packages/TileStache/Goodies/ExternalConfigServer.py", line 128, in __init__
    cache_dict = json_load(urlopen(url_root + "/cache"))
  File "/usr/lib/python2.7/json/__init__.py", line 291, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 515, in spawn_worker
    worker.init_process()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 122, in init_process
    self.load_wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 130, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/lib/python2.7/dist-packages/gunicorn/util.py", line 377, in import_app
    app = eval(obj, mod.__dict__)
  File "<string>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python2.7/site-packages/TileStache/Goodies/ExternalConfigServer.py", line 128, in __init__
    cache_dict = json_load(urlopen(url_root + "/cache"))
  File "/usr/lib/python2.7/json/__init__.py", line 291, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
[2018-07-02 13:48:52 +0000] [26008] [INFO] Worker exiting (pid: 26008)
[2018-07-02 13:48:52 +0000] [26004] [INFO] Shutting down: Master
[2018-07-02 13:48:52 +0000] [26004] [INFO] Reason: Worker failed to boot.
ubuntu@ip-172-31-11-17:~$ gunicorn --bind localhost:8888 "TileStache.Goodies.ExternalConfigServer:WSGIServer('/home/ubuntu/tilestache.cfg')"
[2018-07-02 13:50:42 +0000] [26009] [INFO] Starting gunicorn 19.4.5
[2018-07-02 13:50:42 +0000] [26009] [INFO] Listening at: http://127.0.0.1:8888 (26009)
[2018-07-02 13:50:42 +0000] [26009] [INFO] Using worker: sync
[2018-07-02 13:50:42 +0000] [26013] [INFO] Booting worker with pid: 26013
[2018-07-02 13:50:42 +0000] [26013] [ERROR] Exception in worker process:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 515, in spawn_worker
    worker.init_process()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 122, in init_process
    self.load_wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 130, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/lib/python2.7/dist-packages/gunicorn/util.py", line 377, in import_app
    app = eval(obj, mod.__dict__)
  File "<string>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python2.7/site-packages/TileStache/Goodies/ExternalConfigServer.py", line 128, in __init__
    cache_dict = json_load(urlopen(url_root + "/cache"))
  File "/usr/lib/python2.7/urllib.py", line 87, in urlopen
    return opener.open(url)
  File "/usr/lib/python2.7/urllib.py", line 213, in open
    return getattr(self, name)(url)
  File "/usr/lib/python2.7/urllib.py", line 469, in open_file
    return self.open_local_file(url)
  File "/usr/lib/python2.7/urllib.py", line 483, in open_local_file
    raise IOError(e.errno, e.strerror, e.filename)
IOError: [Errno 20] Not a directory: '/home/ubuntu/tilestache.cfg/cache'
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 515, in spawn_worker
    worker.init_process()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 122, in init_process
    self.load_wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 130, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/lib/python2.7/dist-packages/gunicorn/util.py", line 377, in import_app
    app = eval(obj, mod.__dict__)
  File "<string>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python2.7/site-packages/TileStache/Goodies/ExternalConfigServer.py", line 128, in __init__
    cache_dict = json_load(urlopen(url_root + "/cache"))
  File "/usr/lib/python2.7/urllib.py", line 87, in urlopen
    return opener.open(url)
  File "/usr/lib/python2.7/urllib.py", line 213, in open
    return getattr(self, name)(url)
  File "/usr/lib/python2.7/urllib.py", line 469, in open_file
    return self.open_local_file(url)
  File "/usr/lib/python2.7/urllib.py", line 483, in open_local_file
    raise IOError(e.errno, e.strerror, e.filename)
IOError: [Errno 20] Not a directory: '/home/ubuntu/tilestache.cfg/cache'
[2018-07-02 13:50:42 +0000] [26013] [INFO] Worker exiting (pid: 26013)
[2018-07-02 13:50:42 +0000] [26009] [INFO] Shutting down: Master
[2018-07-02 13:50:42 +0000] [26009] [INFO] Reason: Worker failed to boot.

/home/ubuntu/tilestache.cfg/cache??? whats this "cache" file??? oh look up at this line for the first attempt "File "/home/ubuntu/.local/lib/python2.7/site-packages/TileStache/Goodies/ExternalConfigServer.py", line 128, in __init__
    cache_dict = json_load(urlopen(url_root + "/cache"))" seems to be adding "/cache" to the url... interesting... so what if I change the name of the local file to 'cache' and see what happens...

$ gunicorn --bind localhost:8888 "TileStache.Goodies.ExternalConfigServer:WSGIServer('/home/ubuntu/')"
[2018-07-02 13:34:55 +0000] [25963] [INFO] Starting gunicorn 19.4.5
[2018-07-02 13:34:55 +0000] [25963] [INFO] Listening at: http://127.0.0.1:8888 (25963)
[2018-07-02 13:34:55 +0000] [25963] [INFO] Using worker: sync
[2018-07-02 13:34:55 +0000] [25967] [INFO] Booting worker with pid: 25967
[2018-07-02 13:34:55 +0000] [25967] [ERROR] Exception in worker process:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 515, in spawn_worker
    worker.init_process()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 122, in init_process
    self.load_wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 130, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/lib/python2.7/dist-packages/gunicorn/util.py", line 377, in import_app
    app = eval(obj, mod.__dict__)
  File "<string>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python2.7/site-packages/TileStache/Goodies/ExternalConfigServer.py", line 132, in __init__
    config = ExternalConfiguration(url_root, cache_dict, cache_responses, dirpath)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/TileStache/Goodies/ExternalConfigServer.py", line 105, in __init__
    self.cache = TileStache.Config._parseConfigCache(cache_dict, dirpath)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/TileStache/Config.py", line 322, in _parseConfigCache
    raise Exception('Missing required cache name or class: %s' % json_dumps(cache_dict))

I don't know enough about python to make much about this and was hoping to not have to dig into the TileStache code to work this out. Were you able to successfully load an external tilestache.cfg file?

Thank you,
Nathan
To unsubscribe from this group and all its topics, send an email to tilestache+...@googlegroups.com.

Alex King

unread,
Jul 14, 2018, 11:55:42 AM7/14/18
to tiles...@googlegroups.com
Sorry for the delay on this. Just had a kid so haven't had much time.

It's been a while, but I never got that "External Config Server" to work.  Instead, here's the code I ended up using. Good luck!

import os, TileStache
import boto3

bucket = "tiles_bucket_s3"
key = "config/tilestache.cfg"
dl_path = '/tmp/tilestache.cfg'

def retrieveFile(bucket, key, dl_path):

   s3 = boto3.resource('s3')
   s3.Bucket(bucket).download_file(key, dl_path)

   return dl_path

application = TileStache.WSGITileServer(retrieveFile(bucket, key, dl_path))

Reply all
Reply to author
Forward
0 new messages