An Amazon AMI in the works?

[Steve Nelson]

Good morning everyone - my first post so a quick introduction...

 

My name is Steve Nelson, I'm the GIS Coordinator here at the Town of Banff, Alberta, Canada.

 

A few weeks back I met with our Town Manager to discuss Open Data, and long story short, he's very supportive of the initiative, so I've been reading through as much information as possible regarding Open Data, Policies, Data Catalogs etc.

 

As part of this reading I've come across mention of a possible AMI for Amazon to get the Data Catalog started, and so I just wanted to start a post asking for more information about the possibility of this becoming reality.  I've had success setting up our GIS on Amazon's cloud, so having a preconfigured AMI for your data catalog would be great in getting our town set up with the Code for America catalog.

 

Any insight/details you can provide would be appreciated!

All the best,

Steve

 

[Jeffrey Johnson]

Hi Steve,

I proposed to do this quite a while ago, but have been very busy with
other stuff, and never got around to it. If you are interested, I will
put it on my todo list for this week.

Also, curious what other parts of your GIS are up on AWS. Would be
nice to hear about successful implementations from others.

Cheers,

Jeff

[Todd Willey]

Steve, if you run an Ubuntu 12.04 image you can use the fabric script
to install quite painlessly. You'll just be prompted for a password
for the postgres user you create.

[Steve Nelson]

Hi Jeff,

Thanks for your reply - I would definitely be interested in this AMI for Amazon, however there's no huge rush as it will be a few weeks before I'm able to flush out some processes, identify data sets (low hanging fruit), etc..

 

We have a portion of our GIS data on Amazon along with quite a few applications and services which haven't yet been launched publicly.  They're still under development with the intention of launching these Sept 1st, along with our new Towns website.  however we'll have a couple FLEX apps, some JS apps ,and a couple mobile services for field staff to collect field data etc.

 

I should probably qualify my statement re. setting up GIS on Amazon - since it was an AMI, I really just followed a few simple steps and I also had a how-to video from ESRI explaining what/why I was doing things.  This helped out a lot since I would have been pretty lost without that.

 

Thanks again for the reply Jeff, I'll keep checking in on this forum over the next few weeks.
Steve

[Steve Nelson]

Hi Todd,

Thanks for the details below.  Could you tell me where I find the 'fabric script' you mention below?  Is there a link that would walk me through a typical setup - pretty new to all this so still a little gun shy.

Cheers

Steve

[Adam Hinz]

This is a pretty good starting point:

https://github.com/azavea/Open-Data-Catalog#non-python-dependencies

--

Adam Hinz

Software Engineer, Azavea

hinz...@gmail.com | c: 608-445-8391 | Twitter: @ahinz

[Jeffrey Johnson]

Adam,

Should we set aside some time to work on this together?

Jeff

[Adam Hinz]

You should also be able to do something like this:

sudo apt-get install python-pip

sudo pip install fabric

curl "https://raw.github.com/azavea/Open-Data-Catalog/master/fabfile.py" > fabfile.py

fabric catalog

(all thanks to Todd!)

On Mon, Jul 16, 2012 at 3:17 PM, Steve Nelson <steve....@banff.ca> wrote:

[Adam Hinz]

Oops:

"fab catalog" not "fabric"

[Adam Hinz]

Sure

-Adam

[Todd Willey]

Something like

# fab --host=my-host.com catalog

is best. You can get fancy with specifying identity files (private
keys) with the -i flag as well.

[Jeffrey Johnson]

Just a note that here is my fab code that creates an AMI for geonode
(my day job project). It should give you most of what you need to do
the same for ODC if you want to take a stab at it before I get to it.

https://github.com/jj0hns0n/geonode-cloud/blob/master/fabfile.py#L375

Jeff

[Steve Nelson]

Morning all, thanks for the details/posts so far.  Really appreciate all the insight.

 

However, being new here and not a DB administrator etc, I was hoping for some clarification.

 

I've logged into AWS management console and literally am on the second window of starting a new instance - this is (sadly) where I'm getting lost.

 

So I'm searching in there for AMIs under the community tab for "ubuntu 12.04" and have found 5 possible AMIs.

Is there a specific AMI I should be choosing here?

And, once I choose an AMI and have it set up, where/how do I run these setup commands posted below?

 

Thanks all,
Steve

[Jeffrey Johnson]

Here is the Ubuntu Guide to this.

https://help.ubuntu.com/community/EC2StartersGuide

For 12.04, I would use ami-82fa58eb which is 64 bit and EBS backed.
You will be prompted to create a key as you launch this. When you
finish and it launches. You can use that key as you execute those
commands.

All that said, Im guessing you are on windows? In which case, Im just
not sure how you would use fabric :/ Anyone else have any bright ideas
on that?

Jeff


> Thanks all,
> Steve

[Steve Nelson]

Thanks again Jeff.

Any suggestions on instance size for a data catalog?  We're a small town and since our datasets will also be small - I thought a small instance would suffice?  Or is a large required to run Ubunto and the catalog?
Steve

[Adam Hinz]

Hey Steve,

ODC doesn't store the actual data on the server, instead it stores links to the data. Todd's data warehouse that was recently merged in allows you to store data on Amazon S3. So a small instance would be fine. All of my local VMs run with 300mb of RAM or less and I haven't had any problems at all.

-Adam

[Jeffrey Johnson]

Im running our san diego one on a micro instance.

[Steve Nelson]

Found this link regarding conneting to 12.04 from Windows - is this the issue you were referring to?  The connecting to Ubuntu from Windows?
Steve

http://www.liberiangeek.net/2012/05/connect-to-ubuntu-12-04-precise-pangolin-via-windows-remote-desktop/

 

[Jeffrey Johnson]

Yeah, the problem is that you will have to get connected to the ec2
virtual machine the first time in order to install the package for
this (xrdp).

What you are trying to do is execute the fabric command on your local
workstation and have it execute against the remote machine over SSH.
Looks like there is some guidance on how to setup fabric on windows in
the install docs though.

http://docs.fabfile.org/en/latest/installation.html

In any case, I will see what I can do to get that AMI baked ASAP so
this isnt so hard for ya.

Jeff