Installation instructions

648 views
Skip to first unread message

mrieser

unread,
May 3, 2011, 4:22:30 PM5/3/11
to TransitDataFeeder
Hi all,

I recently tried to install TransitDataFeeder (tdf). As the
documentation on how to do this is rather scarce, I thought I'd share
my notes. I installed in on Linux (Ubuntu) as well as on Mac OS X
10.6, the notes thus assume some knowledge of navigating and working
with a command line (terminal). All lines starting with a '$' are
meant as shell-commandos on a command line prompt. I sometimes use "/
path/to/tdf" which should be replaced with the actual path to where
tdf is located on you file system.


Some pre-requisites for this process:
You'll need:
- Java 1.5 or newer
- Ant (A java build tool, http://ant.apache.org/, installed on many
Mac and Linux distros by default)
- a Subversion (svn) client
- JBoss 5.1 (note: the installation will fail on JBoss 6.0!)
- postgresql (tested with 8.4 and 9.0)


1) checkout tdf
---------------------------------------------------------------------

1a) see http://code.google.com/p/transitdatafeeder/source/checkout
You will need subversion for this.
At the time of this writing, r6 was the newest revision in the
repository.


2) prepare the database
---------------------------------------------------------------------

2a) install postgresql. This actually depends on the operating system
your using. I got tdf working with postgresql 8.4.x and 9.0.

2b) create a postgres-user named "ideauser" with password "ideapass".
The easiest way is to use a GUI like pgadmin (http://
www.pgadmin.org/) to do this. Otherwise, you can also log on into psql
as the postgres user and create the user on the psql-command line:
$ psql -u postgres
postgres# CREATE USER ideauser PASSWORD 'ideapass'

2c) create a database "tdf" with "ideauser" as owner
Again, use pgadmin to do this, or via psql:
postgres# CREATE DATABASE tdf OWNER ideauser

2d) create the initial database-structure.
Load the file /path/to/tdf/datafeeder/resources/sql/DDL/postgresql/
create_tables.sql into the database. On a linux command line, this can
be done like this:
$ psql tdf ideauser < /path/to/tdf/datafeeder/resources/sql/DDL/
postgresql/create_tables.sql

It shows 4 warnings at the end which I ignored (I think the script
assumes to be imported as the postgres user, and not as ideauser)

2e) log in to psql, either on the command line or using pgadmin, and
modify the search-path for the ideauser as specified.
$ psql tdf ideauser
postgres$ ALTER USER ideauser SET search_path TO idea, pg_catalog;

2f) edit the file /path/to/tdf/datafeeder/resources/sql/alter/
postgresql/r27_shapes.sql:
the tablename user must be quoted: user -> "user" (2x)
so, the ends on line 5 and line 26:
REFERENCES user(user_id);
should become
REFERENCES "user"(user_id);

2g) load the modified sql-script into the database (similar to 2d)
$ psql tdf ideauser < /path/to/tdf/datafeeder/resources/sql/alter/
postgresql/r27_shapes.sql

It will show 2 errors at the beginning, ignore them.

2h) load the next sql-script into the database.
$ psql tdf ideauser < /path/to/tdf/datafeeder/resources/sql/alter/
postgresql/r27_transfers.sql

2i) edit the file /path/to/tdf/datafeeder/resources/sql/alter/
postgresql/r28_users.sql:
the tablename user must be quoted: user -> "user" (three times,
similar to 2f)

2j) load the modified file
$ psql tdf ideauser < /path/to/tdf/datafeeder/resources/sql/alter/
postgresql/r28_users.sql

2k) load all of the remaining sql-files in /path/to/tdf/datafeeder/
resources/sql/alter/postgresql/, one after the other.
The don't have to be edited.

2l) edit /path/to/tdf/datafeeder/resources/import-test.sql
replace "CURDATE()" with "current_timestamp"

2m) laod the test-data:
$ psql tdf ideauser < /path/to/tdf/datafeeder/resources/import-
test.sql



3) compile and install tdf
---------------------------------------------------------------------

3a) change to the checkout-directory of tdf
$ cd path/to/tdf

3b) create a file "build.properties" in this directory with the
following 3 lines as content:
jboss.home=/path/to/jboss-5.1.0.GA
jboss.domain=default
profile=prod

3c) edit the file resources/datafeeder-prod-ds.xml
change the database-name from idea-sandbox to tdf
and set the correct password (i.e. "ideapass" in our example)

3d) compile and install/deploy tdf
$ ant

3e) download and install the jdbc-driver for postgres.
download the jar-file matching your postgres installation from
http://jdbc.postgresql.org/download.html. Put this jar-file into path/
to/jboss/common/lib/

3f) start jboss. This is dependent on you system, how you start JBoss.
in my installation, it could start it directly
$ cd /path/to/jboss/ ; ./bin/run.sh
on Ubuntu and probably other linux distros, it's typically:
$ /etc/xinit.d/jboss start

3g) wait until jboss has completely started, then you should be able
to log in to tdf: http://localhost:8080/datafeeder/
login as admin, with an empty password

3h) change passwords in the web gui. Then create additional users,
agencies etc and start entering data.



Some additional notes / experiences:
- I first tried to deploy/install tdf on JBoss AS 6.0. First, the
deployment failed due to some problem sounding similar to <https://
issues.jboss.org/browse/JBAS-7594>. This could be fixed by downloading
"JBoss Seam 2.2.1.Final" from http://seamframework.org/Seam2/Downloads,
unzip it, and then copy jboss-seam-2.2.1.Final/lib/jboss-seam-ui.jar
to path/to/tdf/datafeeder/lib/
With this change, tdf got correctly deployed ($ ant clean; ant) and
loaded by JBoss, but the tdf-GUI did not work properly, likely due to
other incompabilities, resulting in strange error messages and data
not being able to be saved. Thus, one has to use JBoss 5.1 for tdf r6.

- The installation could be simplified if e.g. all the sql-files would
be updated to work with postgresql 8.4 and newer (e.g. steps 2f, 2i
and 2l could be done once in the repository). Also, the installation
process would be simplified if the many single sql-updates (steps 2f -
2k) could be merged into one file.

- In addition, the sql-updates assume that a tablespace/schema "idea"
is used, making step 2e necessary. By cleaning up the sql-files, this
step could probably also be omitted.

- The warnings produced in step 2d are due to sql-commandos only
available to the postgresql super-user, but (as far as I understand
them) not even necessary for the correct working of tdf. So they could
be removed as well.

- Despite its length, this documentation may be far from complete.
Especially the parts involving postgresql assume you have a properly
configured and working postgresql-installation. Thus, instructions
involving postgresql are rather short, as these notes focus more on
getting tdf itself installed. Depending on your actual installation,
you might need some additional documentation on how to do some things
in postgresql on your system.



I hope these notes help other people to get TransitDataFeeder
installed as well.

Regards,
Marcel


-------------------------------

Marcel Rieser
senozon - understanding mobility. http://senozon.com/
MATSim - large-scale, agent-based transport simulation: http://matsim.org/

Karel Novotny

unread,
May 3, 2011, 10:09:39 PM5/3/11
to transitd...@googlegroups.com
Thanks Marcel, this is great! And extremely useful because I don't think
much information on this is available.

So thanks for open-sourcing tdf deployment ;)

karel

Smartrip

unread,
Jun 14, 2011, 3:56:06 AM6/14/11
to transitd...@googlegroups.com
Hi Marcel,

Thanks for sharing this informations. 
I'll evaluate your manual within next few days. After all, I'll share my lesson learned with all of you.

My other question is: did you ever tried to install and deploy the TDF on the Amazon EC2 platform (the similar question arise in one of the latest posts)?

Filip

Michael Dillon

unread,
Jul 14, 2011, 1:22:02 PM7/14/11
to TransitDataFeeder
Marcel,

I just followed your installation guide completely on a fresh ubuntu
server install and everything worked with out a hitch!
Thanks so much for this guide, great work!

- Michael

xabier perez

unread,
Oct 9, 2012, 6:46:12 PM10/9/12
to transitd...@googlegroups.com
hi i had some problems with jboss, when starting the server and going to localhost:8080/datafeeder/ jboss says that cant find the resource, any idea what might cause this?
thank you so much!!

xabier perez

unread,
Oct 20, 2012, 2:18:20 PM10/20/12
to transitd...@googlegroups.com
got it working, there´s a FINAL STEP missing:
move your recently created "default" folder´s deploy files (a .war and a xml) to the "default" folder of jboss.

André Silva

unread,
Oct 22, 2012, 8:31:32 PM10/22/12
to transitd...@googlegroups.com
Hi all,

I followed this installation guide and managed to deploy TDF successfully, but I'm unable to do the last step:

"3h) change passwords in the web gui. Then create additional users, 
agencies etc and start entering data."

After login with admin and an empty password, I see the page where the agencies will appear (empty) and at the top left I see "Admin User | My Account | Logout".
In My account, if I set a new password, I get the following Exception in the UI:

  • Transaction failed
  • <br/>org.hibernate.validator.event.ValidateEventListener.validate(ValidateEventListener.java:148)<br/>org.hibernate.validator.event.ValidateEventListener.onPreUpdate(ValidateEventListener.java:177)<br/>org.hibernate.action.EntityUpdateAction.preUpdate(EntityUpdateAction.java:237)<br/>org.hibernate.action.EntityUpdateAction.execute(EntityUpdateAction.java:88)<br/>org.hibernate.engine.ActionQueue.execute(ActionQueue.java:279)<br/>org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:263)<br/>org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:168)<br/>org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)<br/>org.hibernate.event.def.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:64)<br/>org.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:996)<br/>org.hibernate.impl.SessionImpl.list(SessionImpl.java:1141)<br/>org.hibernate.impl.QueryImpl.list(QueryImpl.java:102)<br/>org.hibernate.ejb.QueryImpl.getSingleResult(QueryImpl.java:88)<br/>org.jboss.seam.security.management.JpaIdentityStore.lookupUser(JpaIdentityStore.java:860)<br/>org.jboss.seam.security.management.JpaIdentityStore.getGrantedRoles(JpaIdentityStore.java:651)<br/>org.jboss.seam.security.management.IdentityManager.getGrantedRoles(IdentityManager.java:243)<br/>org.jboss.seam.security.management.action.UserAction.saveExistingUser(UserAction.java:115)<br/>org.jboss.seam.security.management.action.UserAction.save(UserAction.java:65)<br/>sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)<br/>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)<br/>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)<br/>java.lang.reflect.Method.invoke(Method.java:616)<br/>org.jboss.seam.util.Reflections.invoke(Reflections.java:22)<br/>org.jboss.seam.intercept.RootInvocationContext.proceed(RootInvocationContext.java:32)<br/>org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:56)<br/>org.jboss.seam.transaction.RollbackInterceptor.aroundInvoke(RollbackInterceptor.java:28)<br/>org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)<br/>org.jboss.seam.core.BijectionInterceptor.aroundInvoke(BijectionInterceptor.java:77)<br/>org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)<br/>org.jboss.seam.core.ConversationInterceptor.aroundInvoke(ConversationInterceptor.java:65)<br/>org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)<br/>org.jboss.seam.core.MethodContextInterceptor.aroundInvoke(MethodContextInterceptor.java:44)<br/>org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)<br/>org.jboss.seam.intercept.RootInterceptor.invoke(RootInterceptor.java:107)<br/>org.jboss.seam.intercept.JavaBeanInterceptor.interceptInvocation(JavaBeanInterceptor.java:185)<br/>org.jboss.seam.intercept.JavaBeanInterceptor.invoke(JavaBeanInterceptor.java:103)<br/>org.jboss.seam.security.management.action.UserAction_$$_javassist_seam_7.save(UserAction_$$_javassist_seam_7.java)<br/>org.ideaproject.action.entityhome.UserHome$SaveUserRunAsOperation.execute(UserHome.java:277)<br/>org.jboss.seam.security.Identity.runAs(Identity.java:743)<br/>org.jboss.seam.security.RunAsOperation.run(RunAsOperation.java:75)<br/>org.ideaproject.action.entityhome.UserHome.update(UserHome.java:176)<br/>sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)<br/>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)<br/>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)<br/>java.lang.reflect.Method.invoke(Method.java:616)<br/>org.jboss.seam.util.Reflections.invoke(Reflections.java:22)<br/>org.jboss.seam.intercept.RootInvocationContext.proceed(RootInvocationContext.java:32)<br/>org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:56)<br/>org.jboss.seam.transaction.RollbackInterceptor.aroundInvoke(RollbackInterceptor.java:28)<br/>org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)<br/>org.jboss.seam.core.BijectionInterceptor.aroundInvoke(BijectionInterceptor.java:77)<br/>org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)<br/>org.jboss.seam.transaction.TransactionInterceptor$1.work(TransactionInterceptor.java:97)<br/>org.jboss.seam.util.Work.workInTransaction(Work.java:47)<br/>org.jboss.seam.transaction.TransactionInterceptor.aroundInvoke(TransactionInterceptor.java:91)<br/>org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)<br/>org.jboss.seam.core.MethodContextInterceptor.aroundInvoke(MethodContextInterceptor.java:44)<br/>org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)<br/>org.jboss.seam.security.SecurityInterceptor.aroundInvoke(SecurityInterceptor.java:163)<br/>org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)<br/>org.jboss.seam.intercept.RootInterceptor.invoke(RootInterceptor.java:107)<br/>org.jboss.seam.intercept.JavaBeanInterceptor.interceptInvocation(JavaBeanInterceptor.java:185)<br/>org.jboss.seam.intercept.JavaBeanInterceptor.invoke(JavaBeanInterceptor.java:103)<br/>org.ideaproject.action.entityhome.UserHome_$$_javassist_seam_6.update(UserHome_$$_javassist_seam_6.java)<br/>sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)<br/>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)<br/>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)<br/>java.lang.reflect.Method.invoke(Method.java:616)<br/>org.jboss.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:335)<br/>org.jboss.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:348)<br/>org.jboss.el.parser.AstPropertySuffix.invoke(AstPropertySuffix.java:58)<br/>org.jboss.el.parser.AstValue.invoke(AstValue.java:96)<br/>org.jboss.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:276)<br/>com.sun.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:68)<br/>javax.faces.component.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:88)<br/>com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:102)<br/>javax.faces.component.UICommand.broadcast(UICommand.java:387)<br/>org.ajax4jsf.component.AjaxViewRoot.processEvents(AjaxViewRoot.java:321)<br/>org.ajax4jsf.component.AjaxViewRoot.broadcastEvents(AjaxViewRoot.java:296)<br/>org.ajax4jsf.component.AjaxViewRoot.processPhase(AjaxViewRoot.java:253)<br/>org.ajax4jsf.component.AjaxViewRoot.processApplication(AjaxViewRoot.java:466)<br/>com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:82)<br/>com.sun.faces.lifecycle.Phase.doPhase(Phase.java:100)<br/>com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)<br/>javax.faces.webapp.FacesServlet.service(FacesServlet.java:265)<br/>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)<br/>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)<br/>org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:83)<br/>org.jboss.seam.web.IdentityFilter.doFilter(IdentityFilter.java:40)<br/>org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)<br/>org.jboss.seam.web.MultipartFilter.doFilter(MultipartFilter.java:90)<br/>org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)<br/>org.jboss.seam.web.ExceptionFilter.doFilter(ExceptionFilter.java:64)<br/>org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)<br/>org.jboss.seam.web.RedirectFilter.doFilter(RedirectFilter.java:45)<br/>org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)<br/>org.ajax4jsf.webapp.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:178)<br/>org.ajax4jsf.webapp.BaseFilter.handleRequest(BaseFilter.java:290)<br/>org.ajax4jsf.webapp.BaseFilter.processUploadsAndHandleRequest(BaseFilter.java:368)<br/>org.ajax4jsf.webapp.BaseFilter.doFilter(BaseFilter.java:495)<br/>org.jboss.seam.web.Ajax4jsfFilter.doFilter(Ajax4jsfFilter.java:56)<br/>org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)<br/>org.jboss.seam.web.LoggingFilter.doFilter(LoggingFilter.java:60)<br/>org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)<br/>org.jboss.seam.servlet.SeamFilter.doFilter(SeamFilter.java:158)<br/>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)<br/>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)<br/>org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)<br/>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)<br/>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)<br/>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)<br/>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)<br/>org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)<br/>org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)<br/>org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)<br/>org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)<br/>org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)<br/>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)<br/>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)<br/>org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)<br/>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)<br/>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)<br/>org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:829)<br/>org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:598)<br/>org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)<br/>java.lang.Thread.run(Thread.java:679)<br/>

Do you know what's causing this?

And how can I add other users and agencies and ..... There's nothing in the UI to add it.

Thanks

Thorbjorn Nielsen

unread,
Feb 19, 2013, 5:43:02 PM2/19/13
to transitd...@googlegroups.com
I can confirm this. Furthermore I was not able to insert new stops. Running "ALTER SEQUENCE idea.stop_stop_id_seq OWNER TO ideauser;" somehow made a difference to me. 

(postgres 9.1, java version "1.6.0_37", jboss 5.1, OSX 10.7.5)
Reply all
Reply to author
Forward
0 new messages