Problems configuring The Fascinator dev install

[Peter Sefton]

Hi all,

I have updated the install guide  for Linux to use the new Git repo rather than Subversion: https://sites.google.com/site/fascinatorhome/home/documentation/technical/install/linux---osx

This seems to work OK using the default config. But when I try to change the thing to run on port 80 I run into a problem, where the "./tf.sh start " script will not run properly.

Steps to reproduce:

* Follow the guide to install

* Use .tf.sh to start the server

* Confirm that it is working on port 9997

* Stop the server

* Change all instances of http://localhost:9997 to http://ip-address/

* Start the server (as root 'cos it is using port 80)

The server fails to start, with errors like this in the log:

2013-01-04 02:02:24,349 main     WARN   DerbyAccessControl   Using previously specified data directory: '/home/ubuntu/.fascinator/database', provided value has been ignored: '/home/ubuntu/.fascinator/database'

2013-01-04 02:02:24,420 main     DEBUG  DerbyAccessControl   Derby security database online!

2013-01-04 02:02:24,469 main     INFO   IngestQueueConsumer  Starting ingest

2013-01-04 02:02:24,788 main     WARN   MessageBroker        Message queues startup failed. Shutting them down...

com.googlecode.fascinator.api.subscriber.SubscriberException: org.apache.solr.client.solrj.SolrServerException: java.net.ConnectException: Connection refused

at com.googlecode.fascinator.subscriber.solrEventLog.SolrEventLogSubscriber.setConfig(SolrEventLogSubscriber.java:269) ~[plugin-subscriber-solrEventLog-1.1.3-SNAPSHOT.jar:na]

at com.googlecode.fascinator.subscriber.solrEventLog.SolrEventLogSubscriber.init(SolrEventLogSubscriber.java:223) ~[plugin-subscriber-solrEventLog-1.1.3-SNAPSHOT.jar:na]

at com.googlecode.fascinator.messaging.SubscriberQueueConsumer.init(SubscriberQueueConsumer.java:180) ~[fascinator-core-1.1.3-SNAPSHOT.jar:na]

at com.googlecode.fascinator.messaging.MessageBroker.startMessageQueues(MessageBroker.java:227) [fascinator-core-1.1.3-SNAPSHOT.jar:na]

at com.googlecode.fascinator.messaging.MessageBroker.access$000(MessageBroker.java:47) [fascinator-core-1.1.3-SNAPSHOT.jar:na]

at com.googlecode.fascinator.messaging.MessageBroker$1.run(MessageBroker.java:168) [fascinator-core-1.1.3-SNAPSHOT.jar:na]

at java.util.TimerThread.mainLoop(Timer.java:534) [na:1.6.0_24]

at java.util.TimerThread.run(Timer.java:484) [na:1.6.0_24]

Caused by: org.apache.solr.client.solrj.SolrServerException: java.net.ConnectException: Connection refused

at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:483) ~[solr-solrj-3.3.0.jar:3.3.0 1139785 - rmuir - 2011-06-26 09:25:01]

at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:244) ~[solr-solrj-3.3.0.jar:3.3.0 1139785 - rmuir - 2011-06-26 09:25:01]

at org.apache.solr.client.solrj.request.SolrPing.process(SolrPing.java:60) ~[solr-solrj-3.3.0.jar:3.3.0 1139785 - rmuir - 2011-06-26 09:25:01]

at org.apache.solr.client.solrj.SolrServer.ping(SolrServer.java:114) ~[solr-solrj-3.3.0.jar:3.3.0 1139785 - rmuir - 2011-06-26 09:25:01]

at com.googlecode.fascinator.subscriber.solrEventLog.SolrEventLogSubscriber.setConfig(SolrEventLogSubscriber.java:247) ~[plugin-subscriber-solrEventLog-1.1.3-SNAPSHOT.jar:na]

... 7 common frames omitted

Caused by: java.net.ConnectException: Connection refused

at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.6.0_24]

at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327) ~[na:1.6.0_24]

at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193) ~[na:1.6.0_24]

at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180) ~[na:1.6.0_24]

at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384) ~[na:1.6.0_24]

at java.net.Socket.connect(Socket.java:546) ~[na:1.6.0_24]

at java.net.Socket.connect(Socket.java:495) ~[na:1.6.0_24]

at java.net.Socket.<init>(Socket.java:392) ~[na:1.6.0_24]

at java.net.Socket.<init>(Socket.java:266) ~[na:1.6.0_24]

at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80) ~[commons-httpclient-3.1.jar:na]

at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122) ~[commons-httpclient-3.1.jar:na]

at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) ~[commons-httpclient-3.1.jar:na]

at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) ~[commons-httpclient-3.1.jar:na]

at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) ~[commons-httpclient-3.1.jar:na]

at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) ~[commons-httpclient-3.1.jar:na]

at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) ~[commons-httpclient-3.1.jar:na]

at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) ~[commons-httpclient-3.1.jar:na]

at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:427) ~[solr-solrj-3.3.0.jar:3.3.0 1139785 - rmuir - 2011-06-26 09:25:01]

... 11 common frames omitted

--

Peter Sefton +61410326955 p...@ptsefton.com http://ptsefton.com
Gmail, Twitter & Skype name: ptsefton

[Vaughan Hobbs]

Peter, were you starting as root or a privileged user?

On *nix systems regular users can't start sockets on ports < 1024.

--

Vaughan Hobbs

email: vaugha...@gmail.com

--
You received this message because you are subscribed to the Google Groups "The Fascinator Developer's List" group.
To post to this group, send an email to the-fasci...@googlegroups.com.
To unsubscribe from this group, send email to the-fascinator-...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Vaughan Hobbs]

Oops, just re-read and noticed you were 'root'.

[Andrew Brazzatti]

Hi Peter,

I know you've mentioned you changed all instances of localhost:9997 to http://<ip address> but the error is coming from the SolrEventLogSubscriber where its testing that it can connect to Solr. This is specified in the system-config.json:

"subscriber": {

        "solr-event-log": {

            "uri": "http://localhost:9997/solr/eventlog",

            "buffer": {

                "docLimit": "200",

                "sizeLimit": "204800",

                "timeLimit": "30"

            }

        }

    },

Assuming that this particular entry hasn't been missed, perhaps a firewall rule is preventing the fascinator instance to "connect to itself" via the machines ip address? What happens if you change that uri to be "http://localhost/solr/eventlog"?

[Peter Sefton]

Thanks Andrew,

That didn't work. Looking back through this list the simplest solution seems to be something that Tim McCallum posted ages ago about using iptables to redirect port 80 to 9997 but if you do that then you run into the problem that came up recently that The Fascinator remaps all the URLS it severs to be fully qualified - so they end up as http://localhost/. 

Has anyone figured out how to turn off this behaviour in a simple way?  

pt

[Greg Pendlebury]

It's not simple, but the real answer would be to list Windows + ISS front-end as non-supported and remove the functionality entirely. Unfortunately it is somewhat embedded these days.

[Greg Pendlebury]

That would be IIS... but I doubt I confused anyone

On 4 January 2013 16:03, Greg Pendlebury <greg.pe...@gmail.com> wrote:

Windows + ISS

[Greg Pendlebury]

On the original issue though, what happens if you leave it run for a while? Sometimes the message queues try to start before Solr has finished coming online and you get the same errors. The message queues should try to start 15 seconds later.

Ta,
Greg

On 4 January 2013 13:34, Peter Sefton <ptse...@gmail.com> wrote:

[Greg Pendlebury]

On 4 January 2013 13:03, Peter Sefton <ptse...@gmail.com> wrote:

Steps to reproduce:

* Follow the guide to install

* Use .tf.sh to start the server

* Confirm that it is working on port 9997

* Stop the server

* Change all instances of http://localhost:9997 to http://ip-address/

* Start the server (as root 'cos it is using port 80)

It only just occurred to me that your subject line was saying you used the dev install. In that case Maven is starting Jetty on your behalf and you are going to need to look in your Maven POM for the Portal module. There's a block like this that configure's the Jetty plugin for Maven:

<connectors>
    <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
        <port>9997</port>
        <maxIdleTime>60000</maxIdleTime>
    </connector>
</connectors>

[Peter Sefton]

Coming back to this thread. Lloyd Harris at UWS is trying to get a new build of The Fascinator up and running and we want to be able to demo it on a real server on port 80 so we need to solve this problem.

What would be involved in what Greg mentions "list Windows + ISS front-end as non-supported and remove the functionality entirely"? Or is there a way to make the thing run on port 80 that I have missed ?

Peter

--
You received this message because you are subscribed to the Google Groups "The Fascinator Developer's List" group.
To post to this group, send an email to the-fasci...@googlegroups.com.
To unsubscribe from this group, send email to the-fascinator-...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Greg Pendlebury]

>> What would be involved in what Greg mentions "list Windows + ISS front-end as non-supported and remove the functionality entirely"?

I haven't thought about the specifics for a while. My gut reaction for doing it properly, is that is would be an annoying amount of work. A lot of web facing content renders absolute URLs using the 'baseUrl' parameter. You would need to reverse this practice throughout the code and instead render relative URLs.

The other option I was considering though, would be to fudge the whole thing and edit the Java layer where 'baseUrl' is bound into the the Velocity content. If you changed this to be something like '/' it might just solve your problem completely with very minimal work... although that is of course nothing more than an untested theory.

Lines like these would be options on where to do that:
http://code.google.com/p/the-fascinator/source/browse/the-fascinator/trunk/portal/src/main/java/com/googlecode/fascinator/portal/services/impl/CachingDynamicPageServiceImpl.java#145 (reading from config)
or
http://code.google.com/p/the-fascinator/source/browse/the-fascinator/trunk/portal/src/main/java/com/googlecode/fascinator/portal/services/impl/CachingDynamicPageServiceImpl.java#246 (binding to velocity)

>> is there a way to make the thing run on port 80 that I have missed ?

Not sure if you saw my last post regarding the specific case of the developer build? In this scenario it is Maven starting Jetty for you, and nothing to do with your config anyway. ReDBox and Mint (for example) don't work this way, so it depends on how you start the app as to whether this is relevant to you.

Ta,
Greg

To unsubscribe from this group and stop receiving emails from it, send an email to the-fascinator-...@googlegroups.com.

To post to this group, send an email to the-fasci...@googlegroups.com.

[Peter Sefton]

Thanks Greg,

What seems to work is changing the base path to "/porta/" and getting rid of all instances of :9997 in the config file, but this means you have Solr running on port 80 so more work would be needed to make sure that people can't access the solr index directly and bypass the security.

[Duncan Dickinson]

Hi Peter,

I may be missing something here so please excuse ignorance etc.

Are you looking to run TF under Windows with IIS as the web server? If so, would AAR (http://www.iis.net/downloads/microsoft/application-request-routing) help you? I thought we were using it with the public memory server back in the day. IIRC it was a pain to setup but it seemed to work - maybe you have a Windows SysAdmin handy - IIS certainly makes things so difficult that I respect Win SysAdmins....

I defer to Greg though - I may be mistaken.

Cheers,


Duncan


Duncan Dickinson
QCIF Project Manager 
Central Queensland University

Contact me:

monday to thursday

ph: 07 3138 2084
m: 0432 402 511
skype: de.dickinson

website | calendar | LinkedIn

[Greg Pendlebury]

If I recall correctly, AAR re-writing HTTP responses as they went back to the client was the problem we were trying to overcome with the absolute links.

Ta,
Greg

[Peter Sefton]

HI all,

Just wanted to post a solution to getting The Fascinator trunk (as of now) to run on port 80 on Linux. The solution is (as per what Greg said):

In System config:

Change URL base to the address of your machine:

 "urlBase": "http://example.com/portal/"

Change all instances of localhost:9997 to localhost

In  $installdir/portal/pom.xml remove the one instance of :9997

That seems to do it - and (obviously) remember to start it as root.

pt