Error when Enabling Encryption/Uploading Recovery Key

[Neil Azzaro]

After a couple hurdles getting CV built and deployed, I was able to install it and run it on a test machine.  Encryption succeeds, but Key Escrow fails.  Any suggestions are extremely welcome.

Note:  Homebrew install of Python, npm, etc; running on OS X 10.11.3

When I say "Hurdles", I'm primarily referring to the create_gae_bundle.sh.  First it failed during the copy phase, which was because /server didn't exist.  Then it failed due to overwriting linked files, hence the -f flag.  Then sed wasn't cooperating, and after some research -i on a Mac requires a pair of single quotes after the -i flag. 

Below is my diff of create_gae_bundle.sh which completed successfully while running: make server_config:

diff --git a/create_gae_bundle.sh b/create_gae_bundle.sh

index 6938e95..c701707 100755
--- a/create_gae_bundle.sh
+++ b/create_gae_bundle.sh
@@ -30,7 +30,7 @@ VE_PYTHON=${VE_PYTHON:=VE/bin/python}


 # Create Google App Engine bundle directory.
 rm -rf $BUNDLE_ROOT
-mkdir -p $BUNDLE_ROOT/$SUBDIR
+mkdir -p $BUNDLE_ROOT/$SUBDIR/server
 touch $BUNDLE_ROOT/__init__.py
 touch $BUNDLE_ROOT/$SUBDIR/__init__.py


@@ -40,14 +40,14 @@ mkdir -p "${BUNDLE_ROOT}/${SUBDIR}/server/static/app_out"
 cp "${ROOT}/tmp/app.html" "${BUNDLE_ROOT}/${SUBDIR}/server/static/app_out"


 # Symlink the shared settings file inside the app directory.
-ln -s ../$SRC_REL_PATH/settings.py $BUNDLE_ROOT/$SUBDIR/settings.py
+ln -sf ../$SRC_REL_PATH/settings.py $BUNDLE_ROOT/$SUBDIR/settings.py


 # Symlink necessary files at the root of the bundle.
-ln -s $SERVER_REL_PATH/appengine_config.py $BUNDLE_ROOT/appengine_config.py
-ln -s $SERVER_REL_PATH/app.yaml $BUNDLE_ROOT/app.yaml
-ln -s $SERVER_REL_PATH/index.yaml $BUNDLE_ROOT/index.yaml
-ln -s $SERVER_REL_PATH/cron.yaml $BUNDLE_ROOT/cron.yaml
-ln -s $SERVER_REL_PATH/main.py $BUNDLE_ROOT/main.py
+ln -sf $SERVER_REL_PATH/appengine_config.py $BUNDLE_ROOT/appengine_config.py
+ln -sf $SERVER_REL_PATH/app.yaml $BUNDLE_ROOT/app.yaml
+ln -sf $SERVER_REL_PATH/index.yaml $BUNDLE_ROOT/index.yaml
+ln -sf $SERVER_REL_PATH/cron.yaml $BUNDLE_ROOT/cron.yaml
+ln -sf $SERVER_REL_PATH/main.py $BUNDLE_ROOT/main.py


 # Create symlinks to python egg files.
 if [ -d pyasn1-*.egg ]; then
@@ -60,4 +60,4 @@ fi
 cd ${BUNDLE_ROOT} && ln -f -s ${VE_PATH}/lib/python2.7/site-packages/keyczar keyczar


 # Update the app.yaml application value based on DOMAIN and SUBDOMAIN settings.
-cd ${ROOT} && sed -i "s/ENTER_APPID_HERE/$(PYTHONPATH=src/cauliflowervest/ $VE_PYTHON appid_generator.py)/" ${BUNDLE_ROOT}/app.yaml ${BUNDLE_ROOT}/cron.yaml
+cd ${ROOT} && sed -i '' "s/ENTER_APPID_HERE/$(PYTHONPATH=src/cauliflowervest/ $VE_PYTHON appid_generator.py)/" ${BUNDLE_ROOT}/app.yaml ${BUNDLE_ROOT}/cron.yaml

I was also running into issues with the oauth2client loading webbrowser, so I edited base_client.py as described here https://github.com/google/cauliflowervest/commit/00bbe110aa5a8e74c70b8da16e5f45d40e3ce9c6, below is my diff of that:

diff --git a/src/cauliflowervest/client/base_client.py b/src/cauliflowervest/client/base_client.py
index 06bb56b..fa9982d 100755
--- a/src/cauliflowervest/client/base_client.py
+++ b/src/cauliflowervest/client/base_client.py
@@ -23,13 +23,14 @@
 import json
 import logging
 import optparse
+import os
+import sys
 import time
 import urllib
 import urllib2
+import webbrowser




-import os
-import sys
 try:
   import fancy_urllib
 except ImportError:
@@ -282,7 +283,7 @@ def GetOauthCredentials():
       )
   authorize_url = flow.step1_get_authorize_url()


-  oauth2client.tools.webbrowser.open(authorize_url, new=1, autoraise=True)
+  webbrowser.open(authorize_url, new=1, autoraise=True)
   httpd.handle_request()


   if 'error' in httpd.query_params:

Now on to the logs/tracebacks from the current issue:

Traceback from Server:

Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
    handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
    handler, path, err = LoadObject(self._handler)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 96, in LoadObject
    __import__(cumulative_path)
ImportError: No module named server

Client Output:

tester-7038:~ tester$ /usr/local/bin/cauliflowervest 
WARNING:root:Fetching XSRF token failed with (HTTP Error 500: Internal Server Error). Retrying ...
WARNING:root:Fetching XSRF token failed with (HTTP Error 500: Internal Server Error). Retrying ...
WARNING:root:Fetching XSRF token failed with (HTTP Error 500: Internal Server Error). Retrying ...
WARNING:root:Fetching XSRF token failed with (HTTP Error 500: Internal Server Error). Retrying ...
ERROR:root:Fetching XSRF token failed permanently.
Traceback (most recent call last):
  File "/usr/local/cauliflowervest/cv/lib/python2.7/site-packages/cauliflowervest-0.10.2-py2.7.egg/cauliflowervest/client/base_client.py", line 183, in _RetryRequest
    return self.opener.open(request)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 500: Internal Server Error
ERROR:root:Encryption was enabled, but escrowing the recovery passphrase failed.

Please reboot, manually disable FileVault in System Preferences -> Security & Privacy, wait for decryption to complete, reboot again, and run CauliflowerVest again.
Traceback (most recent call last):
  File "/usr/local/cauliflowervest/cv/lib/python2.7/site-packages/cauliflowervest-0.10.2-py2.7.egg/cauliflowervest/client/mac/tkinter.py", line 178, in _PlainVolumeAction
    client_.UploadPassphrase(volume_uuid, recovery_token)
  File "/usr/local/cauliflowervest/cv/lib/python2.7/site-packages/cauliflowervest-0.10.2-py2.7.egg/cauliflowervest/client/base_client.py", line 229, in UploadPassphrase
    xsrf_token = self._FetchXsrfToken(base_settings.SET_PASSPHRASE_ACTION)
  File "/usr/local/cauliflowervest/cv/lib/python2.7/site-packages/cauliflowervest-0.10.2-py2.7.egg/cauliflowervest/client/base_client.py", line 174, in _FetchXsrfToken
    response = self._RetryRequest(request, 'Fetching XSRF token')
  File "/usr/local/cauliflowervest/cv/lib/python2.7/site-packages/cauliflowervest-0.10.2-py2.7.egg/cauliflowervest/client/base_client.py", line 192, in _RetryRequest
    '%s failed permanently: %%s' % description, str(e))
RequestError: ('Fetching XSRF token failed permanently: %s', 'HTTP Error 500: Internal Server Error')

[Justin McWilliams]

As for oauth2client/webbrowser, that fix was inadvertently reverted, and I've just now readded it here: https://github.com/google/cauliflowervest/commit/736d9f2a966d2bd3d2fd449fb85e2582fbb4e566

For the GAE bundle creation, I'll let Max (an engineer on our team) take over with debugging this, as he's recently made changes in this area.  At minimum, I can confirm the issue with the "sed" replace at the end, due to sed syntax differing on Mac vs Linux (...sigh!). Expect to hear from Max soon...

--
You received this message because you are subscribed to the Google Groups "cauliflowervest-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cauliflowervest-d...@googlegroups.com.
To post to this group, send email to cauliflower...@googlegroups.com.
Visit this group at https://groups.google.com/group/cauliflowervest-discuss.
For more options, visit https://groups.google.com/d/optout.

[Maxim Ermilov]

I pushed fix.

Please, try again with clean repository.

Thanks,

Maxim