Setting up tiddlyweb/tiddlywebwiki on RH Openshift

[Fardil Pirani]

I am trying to install/configure/setup tiddlyweb on Openshift small gear cloud python 2.7, but I am having a difficult time in do so.
Any advice/guidance would be appreciated.

So far I have followed http://tiddlyweb.tiddlyspace.com/Using%20TiddlyWeb

I am stuck on the mount a server using cherrypy.
twanager server $OPENSHIFT_PYTHON_IP $OPENSHIFT_PYTHON_PORT

shows

ERROR: error: No socket could be created -- (('$OPENSHIFT_PYTHON_IP', 8080): [Errno 98] Address already in use)

is there a possible conflict with the apache (since httpd is installed on the default cartridge)?

If there is a cartridge somewhere that has the config already done, please send me link, so I can see what I am doing wrong.

Love the tiddlers.

[chris...@gmail.com]

On Fri, 6 Feb 2015, Fardil Pirani wrote:

> I am trying to install/configure/setup tiddlyweb on Openshift small gear
> cloud python 2.7, but I am having a difficult time in do so.
> Any advice/guidance would be appreciated.
>
> So far I have followed http://tiddlyweb.tiddlyspace.com/Using%20TiddlyWeb

For service like OpenShift, Heroku, etc. it is best to use the
provided web server rather than trying to use cherrypy. That way you
can leave out some of the complexity and interacting you need to do.

Just this morning I've made something go that seems to work okay.
However I haven't had a chance to annotate it or provide instructions.
I will do so later today (I'm on my way out right now). In the
meantime:

* http://tweb-cdent.rhcloud.com/default#cow is the running gear
* If you let me know if your openshift username I should be able to
add you to my gear. I'm not sure how to just link you to it (not my
area of expertise)
* I've pushed the code up to github:
https://github.com/cdent/twebwiki-openshift

My set up uses mod_wsgi, so the wsgi.py is replaced with wsgiapp.py
from tiddlyweb. It has a tiddlywebconfig.py and there is a deploy
action hook that is responsible for populating the store associated
with the instance.

I'm not clear on how persistence works over ups and downs of gears,
but I've used the OPENSHIFT_DATA_DIR, I hope that's right.

This install is setup without policies on the bags or recipes so
anyone can create stuff.

Note that this install tiddlyweb and tiddlywebwiki, so it is working
with TiddlyWiki Classic. If you want five this installation can be
used with it, but you'll need to build a custom five that talks to it.
One of the options is discussed here:
https://groups.google.com/d/msg/tiddlyweb/UcYrnvGwNUQ/UJRgSn4VjfYJ

> I am stuck on the mount a server using cherrypy.
> twanager server $OPENSHIFT_PYTHON_IP $OPENSHIFT_PYTHON_PORT

I ran into this problem as well. From what I can the latter is not
actualy the port your gear is listening on for HTTP but rather one
that is being used to communicate between the gear and some proxy
server. I had to not use it.

> If there is a cartridge somewhere that has the config already done, please
> send me link, so I can see what I am doing wrong.

How can I make my cartridge reusable/installable by other people?

And finally: Apparently I did this before (there are some remains of
past work in my openshift account) but this new version is far more
automated and clean.

More when I return later today.

--
Chris Dent http://burningchrome.com/
[...]

[chris...@gmail.com]

On Sat, 7 Feb 2015, chris...@gmail.com wrote:

> More when I return later today.

I've added a README and comments in some of the code to the repo. If
someone who is more familiar with OpenShift cares to help make it more
useful please shout out and I'll give you acess to tune things up.

https://github.com/cdent/twebwiki-openshift

[Fardil Pirani]

Your reply was indeed very helpful and highly appreciated. Thank you.

I followed your directions on the ReadMe and now I am able to save the wiki thru tiddlyweb!
I have a working tiddlyweb server on:

http://tiddly-neofalcon.rhcloud.com/recipes/default/tiddlers.wiki

I will be setting up with tw5 next, and I will be in touch with you.

Since I am just a beginner, I haven't learned enough on how to make your environment available as an OpenShift cartridge.
I did find docs on the OpenShift website on how to deploy a reusable cartridge using their API:

https://developers.openshift.com/en/get-involved-extend-openshift.html#_build_a_cartridge


All the best and God bless you.

[Tobias Beer]

Just wanted to say, but ;-)

Best wishes, Tobias. 

[chris...@gmail.com]

On Mon, 9 Feb 2015, Tobias Beer wrote:

> Just wanted to say, but

> <http://tiddly-neofalcon.rhcloud.com/recipes/default/tiddlers.wiki#BUT> ;-)

I'm sure you're aware that this is an easy problem to fix:

http://tiddlyweb.tiddlyspace.com/policy

But policies are a complex enough topic that there's no sense in
messing with them when first experimenting with getting things
going. Once they are, it's a SimpleMatterofJSON™ to get policies,
recipes, bags and users[1] all in place to make things go.

[1] Actually with a remote server like this an additional plugin
would be required for user handling:
https://pypi.python.org/pypi/tiddlywebplugins.socialusers

[Fardil Pirani]

Hello again,

I am having a minor problem with setting up TW5.
I set up tiddlyweb for tw5 from the link.
While I had gotten stuck running bin/tankbld.sh due to a older version of node that the gear has by default (v0.6.x). So I redirected thru the action hooks and used v0.10.25:
1. Create file called NODEJS_VERSION inside .openshift/markers and write 0.10.25
2. Make a lib directory in .openshift/ and create setup_custom_nodejs_env file and utils file

3. under .openshift/lib/setup_custom_nodejs_env file paste

#  Utility functions for bash session - sourced in via the user's
#  bash profile ($OPENSHIFT_DATA_DIR/.bash_profile).

#  Source utility functions.
source $OPENSHIFT_REPO_DIR/.openshift/lib/utils


#  Internal function to setup path and remove the wrappers.
function _setup_path_and_remove_wrappers() {
   #  First invocation of npm or node, so setup the custom path and
   #  unset the wrappers. Add the custom node binaries to the PATH.
   [ -z "$ZDEBUG" ]  ||  echo "Setting path to include custom Node version"
   setup_path_for_custom_node_version
   unset node
   unset npm
   unset _setup_path_and_remove_wrappers

}  #  End of function  _setup_path_and_remove_wrappers.


#  Temporary wrapper function to setup path before invoking npm.
function npm() {
   #  Setup path, remove wrappers and reinvoke npm.
   _setup_path_and_remove_wrappers
   npm "$@"

}  #  End of function  npm.


#  Temporary wrapper function to setup path before invoking node.
function node() {
   #  Setup path, remove wrappers and reinvoke node.
   _setup_path_and_remove_wrappers
   node "$@"

}  #  End of function  node.


#
# EOF

 4. Write the following in .openshift/lib/utils

#!/bin/bash
#
#  Utility functions.
#


#  Returns the configured Node version - defaults to 0.8.24.
function get_node_version() {
   marker="$OPENSHIFT_REPO_DIR/.openshift/markers/NODEJS_VERSION"
   nodejs_ver=$(egrep -v "^\s*#.*" "$marker" | egrep -v "^\s*$" | tail -1)
   echo "${nodejs_ver:-"0.8.24"}"

}  #  End of function  get_node_version.


#  Returns the directory where Node is to be installed/is installed.
function get_node_install_dir() {
   echo "$OPENSHIFT_DATA_DIR"

}  #  End of function  get_node_install_dir.


#  Returns the path to the npm binary.
function get_npm_bin_path() {
   ver=${1:-"$(get_node_version)"}
   echo "$(get_node_install_dir)/node-v$ver-linux-x64/bin"

}  #  End of function  get_npm_bin_path.


#  Returns the path to the node binary.
function get_node_bin_path() {
   echo "$(get_npm_bin_path $@)"

}  #  End of function  get_node_bin_path.


#  Returns the temporary directory we use for processing.
function get_node_tmp_dir() {
   ztmpdir="$OPENSHIFT_DATA_DIR/.nodejs.tmp"

   #  Ensure temp directory is created.
   [ -d "$ztmpdir" ]  ||  mkdir -p "$ztmpdir"

   echo "$ztmpdir"

}  #  End of function  get_node_tmp_dir.


#
#  Download and install the specified Node.js version.
#
function _install_nodejs() {
   ver=${1:-"$(get_node_version)"}

   # Sample download links:
   #    http://nodejs.org/dist/v0.8.24/node-v0.8.24-linux-x64.tar.gz
   #    http://nodejs.org/dist/v0.10.10/node-v0.10.10-linux-x64.tar.gz
   zfile="node-v${ver}-linux-x64.tar.gz"
   zlink="http://nodejs.org/dist/v${ver}/${zfile}"

   instdir="$(get_node_install_dir)"

   #  Download and extract the gzipped tarball.
   dldir="$OPENSHIFT_DATA_DIR/downloads"
   mkdir -p "$dldir"

   echo "  - Downloading and extracting $zlink ... "

   if ! curl -L -o "$dldir/$zfile" "$zlink"; then
      echo "  - ERROR  -- download failed for $zlink"
      echo "  - download uri = $dldir/$zfile"
      return 1
   fi

   (cd "$instdir"; tar -zxf "$dldir/$zfile")
   echo "  - Done installing Node.js version $ver"

}  #  End of function  _install_nodejs.


#  Ensure npm and node symlinks exist in ~/.node_modules/.bin/ directory.
function _ensure_symlinks_in_node_modules_bin_dir() {
   zdir="$HOME/.node_modules/.bin/"
   if [ -d "$zdir" ]; then
      ln -sf "$(get_npm_bin_path)/npm"   "$zdir"
      ln -sf "$(get_node_bin_path)/node" "$zdir"
   fi

}  #  End of function  _ensure_symlinks_in_node_modules_bin_dir.


#  Ensure context - this is a bit too hacky.
function _ensure_context_exists() {
   zenv="$OPENSHIFT_HOMEDIR/nodejs/configuration/node.env"
   if ! egrep '^\s*function\s+nodejs_context' "$zenv" > /dev/null 2>&1; then
      echo -e "function nodejs_context() { bash -c \"\$@\"; }\n" >> "$zenv" || :
   fi

}  #  End of function  _ensure_context_exists.


#  Ensure the shell env setup bits are added to user's .bash_profile.
function _ensure_bash_profile_setup() {
   dot_bash_profile=$OPENSHIFT_DATA_DIR/.bash_profile
   pattern='\s*source(.*)\.openshift/lib/setup_custom_nodejs_env\s*(.*)\s*'
   if ! egrep "$pattern" $dot_bash_profile > /dev/null 2>&1 ; then

      cat >> $dot_bash_profile  <<SRCEOF

#  Setup shell env for the custom Node[.js] version.
source "\$OPENSHIFT_REPO_DIR/.openshift/lib/setup_custom_nodejs_env"
SRCEOF

      echo "  - Added source setup_custom_nodejs_env to .bash_profile"
   fi

}  #  End of function  _ensure_bash_profile_setup.


#  Check and install custom Node[.js] version.
function ensure_node_is_installed() {
   ver=${1:-"$(get_node_version)"}

   echo "  - Checking to see if Node.js version $ver is installed ... "

   #
   #  To re-download and reinstall Node.js, uncomment these lines.
   #     rm -f downloads/node-v${ver}-linux-x64.tar.gz
   #     rm -rf $OPENSHIFT_DATA_DIR/node-v${ver}-linux-x64/
   #
   #  Note:  This function could be run multiple times on every git push,
   #         so use w/ caution (do once).
   #

   if [ -d  "$(get_node_bin_path)" ]; then
      echo "  - Node.js version $ver is already installed"
   else
      _install_nodejs "$ver"
   fi

   #  Ensure node and npm symlinks exist in ~/.node_modules/.bin directory.
   _ensure_symlinks_in_node_modules_bin_dir

   #  Ensure context exists.
   _ensure_context_exists

   #  Ensure .bash_profile sets up path for custom Node[.js] version.
   _ensure_bash_profile_setup

}  #  End of function  ensure_node_is_installed.


#  Sets up PATH to include the custom node version binaries.
function setup_path_for_custom_node_version() {
   #  Get version.
   ver=${1:-"$(get_node_version)"}

   #  Get node binary path.
   node_bin_path=$(get_node_bin_path "$ver")

   #  Add the node binary path to the PATH.
   export PATH="$node_bin_path:${PATH}"

   if [ -n "$_SHOW_SETUP_PATH_MESSAGES" ]; then
      echo "  - PATH set to include custom node version ($ver) from"
      echo "       $node_bin_path "
      echo "    PATH = $PATH"
   fi

}  #  End of function  setup_path_for_custom_node_version.



#
# EOF
#

 5. inside .openshift/action_hooks/build paste:

#!/bin/bash
# This is a simple build script and will be executed on your CI system if
# available.  Otherwise it will execute while your application is stopped
# before the deploy step.  This script gets executed directly, so it
# could be python, php, ruby, etc.


#  Source utility functions.
source "$OPENSHIFT_REPO_DIR/.openshift/lib/utils"

#  Setup path to include the custom Node[.js] version.
_SHOW_SETUP_PATH_MESSAGES="true" setup_path_for_custom_node_version

 6. Write the following into .openshift/action_hooks/pre_build

#!/bin/bash
# This is a simple script and will be executed on your CI system if
# available.  Otherwise it will execute while your application is stopped
# before the build step.  This script gets executed directly, so it
# could be python, php, ruby, etc.


#  Source utility functions.
source "$OPENSHIFT_REPO_DIR/.openshift/lib/utils"

#  Ensure custom node version if not installed.
echo ""
ensure_node_is_installed


#  We need to move the package.json file out of the way in pre_build, so
#  that the OpenShift git post-receive hook doesn't try and use the old
#  npm version to install the dependencies.

That redirects the node to a later version so the app.html file can be built from bin/tankbld.sh

Now I am trying to navigate to $OPENSHIFT_REPO_DIR/TiddlyWiki5/tmp/app.html
I did have some success by using a symlink from the dir above into the wsgi/static/app.html directory, but I get a

Sync error while processing '$:/StoryList': XMLHttpRequest error code: 0

Thanks again

[Fardil Pirani]

Hello again,

Just wanted to update as to why I was getting the sync error. I forgot to set config for the web server.

I found this:

<VirtualHost *>
    ServerName barney.example.com
    AllowEncodedSlashes On
    Alias /static /home/barney/public_html/barney.example.com/static
    <Directory /home/barney/public_html/barney.example.com/static>
        Order allow,deny
        Allow from all
    </Directory>
    ErrorLog /var/log/apache2/barney.example.com-error.log
    CustomLog /var/log/apache2/barney.example.com-access.log combined
    WSGIDaemonProcess barney.example.com user=barney processes=1 threads=10
    WSGIProcessGroup barney.example.com
    WSGIPassAuthorization On
    WSGIScriptAlias /wiki /home/barney/tiddlywebs/barney.example.com/wsgiapp.py
</VirtualHost>

 

However, I do not know whether .htaccess is the correct file to write this config into.

[chris...@gmail.com]

On Fri, 27 Feb 2015, Fardil Pirani wrote:

> Just wanted to update as to why I was getting the sync error. I forgot to
> set config for the web server.

Can you remind us which sync error you are getting and whether this
is with the TiddlyWeb python server or the TiddlyWiki5 node server?

Also, a reminder: If you're using the web server that is built into
the default OpenShift gear you do not need to configure the web
server, it is already configured to run properly with applications
in your setup. What you need to do is make sure that you have the
right code in the place where openshift wants to find it.