Bob's Browser Isn't Putting

[Kevin Kleinfelter]

My browser isn't sending PUTs for new tiddlers when connected to a TW using the Bob plugin v 1.2.2.

I have an Apache reverse proxy in front of multiple web servers.  Apache handles SSL and authentication and then forwards the traffic.  It forwards to plain TW on port 8084.  It forwards to Bob on port 8088.

Both sites load properly in their browser tab.  When I add a tiddler to the plain-TW site, the browser sends a DELETE and two PUT requests, as expected.  When I add a tiddler to the Bob site, the browser sends no requests.

If I "rm -rf" the Bob plugin and I change command line which start the formerly-Bob wiki so that it starts a plain old TW, reads and writes work fine.  

Any suggestions?

[Jed Carty]

Bob uses websockets, not http requests, to save so you wont see any PUT or DELETE requests from the browser.

[Jed Carty]

If I remember correctly you have to tell Apache to forward websockets, so if you are having trouble that may be why.

[Kevin Kleinfelter]

It gets curiouser and curiouser.  When I connect to my Bob wiki via https://my.site.com, which passes through the Apache reverse proxy:

• The browser does not send PUTs on updates.
• Updates to tiddlers don't persist.
• 'Auto-populate list from Wikis folder' doesn't show any wikis.
• Existing wiki content is rendered correctly.

But when I connect to the same wiki by connecting to internalhost:8088, 

• PUTs happen
• Updates to tiddlers persist
• 'Available WIkis' lists all 3 of my wikis
• Existing wiki content is rendered correctly.

Does Bob have a problem with HTTPS?

[Kevin Kleinfelter]

Ah, that makes sense.  If there's supposed to be a second network connection from the browser to the server, it too will have to be forwarded.

To make the main web-page connection happen via HTTPS, I have Apache handle the authentication and pass incoming port 443 to internal port 8088.

If Bob needs another connection for automatic communication, not only does it have to be forwarded, it has to pass through authentication.  

I see that if the main web server is listening on port 8080, Bob will listen on 8081.  How does that work out on the browser when the main connection is on port 443? Is it going to try to reach the websocket listener on 444 or on 8081?  And if it is running on a different connection, how is it going to negotiate HTTPS authentication?

If there is anyone who has successfully set up Bob behind a web server which handles the HTTPS auth, please share how you did it.  Otherwise, I may have to give up on Bob and go back to plain Tiddlywiki.

[Jed Carty]

The same port is used for both http and websockets with Bob. The part about 8081 is no longer used. You don't need to set up separate rules for handling that.

If you are changing the URL with Apache you also have to set that up in Bob.

If you have your external url as:

http://blah.com/some/path/to/root/wiki/

with all of the wikis hosted under that than you have to set the proxyprefix in the settings like this:

proxyprefix: 'some/path/to/root/wiki'

for more on that see the 'Using Proxies' part of the documentation.

For the things I do I use node for authentication and proxy connections so I don't know about setting up Bob with Apache past what I said above.

[Kevin Kleinfelter]

Jed - thank you for the information. You've shared what you know and are clear where your knowledge ends, and that's all too rare.

All - I'm going to continue to try and work out here how to make this work, since I can't believe I'm the only person on the planet who wants to access TW-Bob from the Internet via an Apache front-end.  Maybe I'll get a pointer from someone.  If not, then at least there's a record for the next guy (if I succeed).  I need to stick with Apache because I have one IP address and I have a bunch of Apache-driven sites using subdomains and vhosts, and I want to add Bob to that mix.

I server the wiki on https://my.domain.com/ (from "/"). I put Fiddler on it.  (I'm on a Mac. Fiddler-Mac exists but it's lame compared with real Fiddler.)  I see the request:

GET https://my.domain.com/ HTTP/1.1

Upgrade: websocket

Connection: Upgrade

and the response should include:

HTTP/1.1 101 Switching Protocols

Upgrade: websocket

Connection: Upgrade

but the actual response is is simply a status 200 with a re-download of the full TW web page.  Clearly, the request to create the WebSocket isn't recognized somewhere on the end-to-end path.

The trace is with HTTPS authentication turned off on my.domain.com, so I'm not dealing with an authentication problem.

My apache config for this host looks like:

<VirtualHost *:443>

    ServerName my.domain.com

    ProxyRequests Off

    ProxyPreserveHost on

    ProxyPass "/"  "http://127.0.0.1:8088/"

    ProxyPassReverse "/"  "http://127.0.0.1:8088/"

    Include certificate-stuff.conf

    AllowEncodedSlashes On

    SSLEngine On

    SSLProxyEngine On

    ###### Adding WebSockets ######

    ProxyPass "/" "ws://127.0.0.1:8088/"

    ProxyPassReverse  "/" "ws://127.0.0.1:8088/"

    ###### Finished WebSockets #######

</VirtualHost>

/etc/apache2/mods-enabled/proxy_wstunnel.load contains a LoadModule for proxy_wstunnel_module, so I should be set up to proxy websockets.  I think the problem is happening BEFORE that point, because the REQUEST to engage websockets is getting missed.

WebSockets *can* work on this host, because I have another site successfully using them via:

      ProxyPassMatch (.*)(\/websocket)$ "ws://localhost:81/$1$2"

      ProxyPass / "http://localhost:81/"

      ProxyPassReverse / "http://localhost:81/"

I observe that the successful site has Apache proxying both http and ws on the same port, so Bob ought to be able to work with 1 port for both protocols. I do see that the successful site has a way to distinguish between http and ws traffic -- their site puts 'websocket' in its WebSocket URLs, but as I said earlier, Bob's *request* to go to ws is getting ignored, not ws traffic itself.

I'm a little worried about these two directives conflicting:

    ProxyPass "/"  "http://127.0.0.1:8088/"

    ProxyPass "/" "ws://127.0.0.1:8088/"

I don't know enough to be sure, but I don't see a way for Apache to distinguish whether "/" should be proxied as http or https.

[Kevin Kleinfelter]

Aha!  Apache does need to distinguish which traffic to proxy as WebSocket.  Many examples on the web assume that the URI path can be used to recognize WebSocket traffic.  e.g. All WS traffic begins with "/ws".  But Bob issue the same URI for the WS connection as the Tiddlywiki page itself.  e.g. https://my.domain.com/.

I thought I'd fail to put Bob behind an Apache SSL-terminating proxy, until I realized I could use the WS header itself to recognize the traffic. The 'WebSockets' section in the httpd.conf below says, "If you get a request with the HTTP headers "Upgrade: websocket" and "Connection: Upgrade" rewrite it to a WebSocket request for the IP/port where Bob is running.

<VirtualHost *:443>

    ServerName my.domain.com

    ProxyRequests Off

    ProxyPreserveHost on

    ###### Adding WebSockets ######

    RewriteEngine On

    RewriteCond %{HTTP:Connection} Upgrade [NC]

    RewriteCond %{HTTP:Upgrade} websocket [NC]

    RewriteRule /(.*) ws://127.0.0.1:8088/$1 [P,L]

    ###### Finished WebSockets #######

    ProxyPass "/"  "http://127.0.0.1:8088/"

    ProxyPassReverse "/"  "http://127.0.0.1:8088/"

    Include cert-stuff.conf

    AllowEncodedSlashes On

    SSLEngine On

    SSLProxyEngine On

    <Location "/">

       AuthType Basic

       AuthName "Restricted test Content"

       AuthUserFile /path/to/passwords

       Require valid-user

    </Location>

</VirtualHost>

Keyword fodder for any future searcher (including myself): Use Apache as an SSL-terminating proxy in front of Tiddlywiki-with-Bob to provide user/password authentication so you can expose your wiki on the Internet.

[Jed Carty]

I will also copy this post and put it in the bob documentation unless you have some objection.

[Kevin Kleinfelter]

Oh gosh no objection.  If it makes a shred of sense, propagate it.