Has anyone created an Amazon EC2 AMI?

[Paul Esling]

I am attempting to install the TransitDataFeeder on an Amazon EC2
server.

Has anyone done this and have an AMI that they could share? I'm not a
Java developer so this has been a bit challenging, and I have not yet
been able to get this up and running.

I have a server running JBOSS 5.1 on Ubuntu 10.04, with PostgreSQL
8.4.

I checked out the TransitDataFeeder code through subversion and
attempted to build the application by editing build.properties and
running ant.

I can now get to a TransitDataFeeder login page at http://[my IP]:8080/
datafeeder/, however there was no database created in PostgreSQL
(where I intended to look for a username and password).

Any advice is appreciated.

Regards,
Paul

[Nicolás Pace]

Hi Paul,

I haven't done this in an EC2 AMI, but have already installed in my own VM.

I've followed the instructions indicated in:

# https://groups.google.com/forum/?fromgroups=#!topic/transitdatafeeder/BxCFVFRefqk

I customized them... this is my working version:

svn checkout http://transitdatafeeder.googlecode.com/svn/trunk/ transitdatafeeder-read-only

cd transitdatafeeder-read-only/datafeeder

======================

#Some pre-requisites for this process: 

#You'll need: 

# - Java 1.5 or newer 

#Instalar Java

echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list

echo "deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886

apt-get update

apt-get install -y oracle-java6-installer

# En ubuntu server

# primero, reparar locale: http://desarrolla2.com/post/reparando-perl-warning-setting-locale-failed-en-ubuntu-server

locale-gen en_US.UTF-8

dpkg-reconfigure locales

#  - Ant (A java build tool, http://ant.apache.org/, installed on many 

#Mac and Linux distros by default) 

apt-get install -y ant1.7

#  - a Subversion (svn) client 

apt-get install -y subversion 

apt-get install -y zip screen

#  - JBoss 5.1 (note: the installation will fail on JBoss 6.0!) 

wget "http://downloads.sourceforge.net/project/jboss/JBoss/JBoss-5.1.0.GA/jboss-5.1.0.GA-jdk6.zip?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fjboss%2Ffiles%2FJBoss%2FJBoss-5.1.0.GA%2F&ts=1360276452&use_mirror=ufpr" -O jboss-5.1.0.GA-jdk6.zip

#  - postgresql (tested with 8.4 and 9.0) 

apt-get install -y postgresql

#1) checkout tdf 

#--------------------------------------------------------------------- 

#1a) see http://code.google.com/p/transitdatafeeder/source/checkout 

#    You will need subversion for this. 

#    At the time of this writing, r6 was the newest revision in the 

# repository. 

svn checkout http://transitdatafeeder.googlecode.com/svn/trunk/ transitdatafeeder-read-only

#2) prepare the database 

#--------------------------------------------------------------------- 

#2a) install postgresql. This actually depends on the operating system 

#your using. I got tdf working with postgresql 8.4.x and 9.0. 

#2b) create a postgres-user named "ideauser" with password "ideapass". 

#    The easiest way is to use a GUI like pgadmin (http:// 

#www.pgadmin.org/) to do this. Otherwise, you can also log on into psql 

#as the postgres user and create the user on the psql-command line: 

#    $ psql -u postgres 

#    postgres# CREATE USER ideauser PASSWORD 'ideapass' 

su postgres

psql

# CREATE USER ideauser PASSWORD 'ideapass'

#2c) create a database "tdf" with "ideauser" as owner 

#    Again, use pgadmin to do this, or via psql: 

#    postgres# CREATE DATABASE tdf OWNER ideauser

#CREATE DATABASE tdf OWNER ideauser  

# configurar para que se puedan conectar desde todos lados (fuente http://blog.deliciousrobots.com/2011/12/13/get-postgres-working-on-ubuntu-or-linux-mint/)

# Editar /etc/postgresql/9.1/main/pg_hba.conf cambiando:

#local   all             all                                     peer

#por:

#local   all             all                                     md5

#2d) create the initial database-structure. 

#    Load the file /path/to/tdf/datafeeder/resources/sql/DDL/postgresql/ 

#create_tables.sql into the database. On a linux command line, this can 

#be done like this: 

psql tdf ideauser < /path/to/tdf/datafeeder/resources/sql/DDL/postgresql/create_tables.sql 

# It shows 4 warnings at the end which I ignored (I think the script 

# assumes to be imported as the postgres user, and not as ideauser) 

# 2e) log in to psql, either on the command line or using pgadmin, and 

# modify the search-path for the ideauser as specified. 

psql tdf ideauser 

#  postgres$ ALTER USER ideauser SET search_path TO idea, pg_catalog; 

# 2f) edit the file /path/to/tdf/datafeeder/resources/sql/alter/postgresql/r27_shapes.sql: 

#   the tablename user must be quoted: user -> "user" (2x) 

#   so, the ends on line 5 and line 26: 

#     REFERENCES user(user_id); 

#   should become 

#     REFERENCES "user"(user_id); 

# 2g) load the modified sql-script into the database (similar to 2d) 

psql tdf ideauser < /path/to/tdf/datafeeder/resources/sql/alter/postgresql/r27_shapes.sql 

#     It will show 2 errors at the beginning, ignore them. 

# 2h) load the next sql-script into the database. 

psql tdf ideauser < /path/to/tdf/datafeeder/resources/sql/alter/postgresql/r27_transfers.sql 

# 2i) edit the file /path/to/tdf/datafeeder/resources/sql/alter/postgresql/r28_users.sql: 

#     the tablename user must be quoted: user -> "user" (three times, 

# similar to 2f) 

# 2j) load the modified file 

psql tdf ideauser < /path/to/tdf/datafeeder/resources/sql/alter/postgresql/r28_users.sql 

# 2k) load all of the remaining sql-files in /path/to/tdf/datafeeder/resources/sql/alter/postgresql/, one after the other. The don't have to be edited. 

# 2l) edit /path/to/tdf/datafeeder/resources/import-test.sql 

#     replace "CURDATE()" with "current_timestamp" 

# 2m) laod the test-data: 

psql tdf ideauser < /path/to/tdf/datafeeder/resources/import-test.sql 

# 3) compile and install tdf 

# --------------------------------------------------------------------- 

# 3a) change to the checkout-directory of tdf 

cd path/to/tdf 

# 3b) create a file "build.properties" in this directory with the following 3 lines as content: 

#        jboss.home=/root/jboss-5.1.0.GA 

#        jboss.domain=default 

#        profile=prod

# 3c) edit the file resources/datafeeder-prod-ds.xml 

#     change the database-name from idea-sandbox to tdf 

#     and set the correct password (i.e. "ideapass" in our example) 

# 3d) compile and install/deploy tdf 

ant 

# download and install the jdbc-driver for postgres. 

#     download the jar-file matching your postgres installation from 

# http://jdbc.postgresql.org/download.html. Put this jar-file into path/ 

# to/jboss/common/lib/ 

wget http://jdbc.postgresql.org/download/postgresql-9.2-1002.jdbc3.jar -O /root/jboss-5.1.0.GA/common/lib/postgresql-9.2-1002.jdbc3.jar

# 3f) start jboss. This is dependent on you system, how you start JBoss. 

#     in my installation, it could start it directly 

#     $ cd /root/jboss-5.1.0.GA/ ; ./bin/run.sh 

#     on Ubuntu and probably other linux distros, it's typically: 

/etc/xinit.d/jboss start 

# 3g) wait until jboss has completely started, then you should be able 

# to log in to tdf: http://localhost:8080/datafeeder/ 

#     login as admin, with an empty password 

# 3h) change passwords in the web gui. Then create additional users, 

# agencies etc and start entering data. 

# Some additional notes / experiences: 

# - I first tried to deploy/install tdf on JBoss AS 6.0. First, the 

# deployment failed due to some problem sounding similar to <https:// 

# issues.jboss.org/browse/JBAS-7594>. This could be fixed by downloading 

# "JBoss Seam 2.2.1.Final" from http://seamframework.org/Seam2/Downloads, 

# unzip it, and then copy jboss-seam-2.2.1.Final/lib/jboss-seam-ui.jar 

# to path/to/tdf/datafeeder/lib/ 

# With this change, tdf got correctly deployed ($ ant clean; ant) and 

# loaded by JBoss, but the tdf-GUI did not work properly, likely due to 

# other incompabilities, resulting in strange error messages and data 

# not being able to be saved. Thus, one has to use JBoss 5.1 for tdf r6. 

# - The installation could be simplified if e.g. all the sql-files would 

# be updated to work with postgresql 8.4 and newer (e.g. steps 2f, 2i 

# and 2l could be done once in the repository). Also, the installation 

# process would be simplified if the many single sql-updates (steps 2f - 

# 2k) could be merged into one file. 

# - In addition, the sql-updates assume that a tablespace/schema "idea" 

# is used, making step 2e necessary. By cleaning up the sql-files, this 

# step could probably also be omitted. 

# - The warnings produced in step 2d are due to sql-commandos only 

# available to the postgresql super-user, but (as far as I understand 

# them) not even necessary for the correct working of tdf. So they could 

# be removed as well. 

# - Despite its length, this documentation may be far from complete. 

# Especially the parts involving postgresql assume you have a properly 

# configured and working postgresql-installation. Thus, instructions 

# involving postgresql are rather short, as these notes focus more on 

# getting tdf itself installed. Depending on your actual installation, 

# you might need some additional documentation on how to do some things 

# in postgresql on your system.