Some lessons learned from using AppScale 3.1.0 on the Raspberry Pi 2 with Ubuntu Trusty
41 views
Skip to first unread message
Christian Baun
Sep 20, 2016, 9:28:48 AM9/20/16
to AppScale Community
Hi all,
I played a bit with appscale 3.1.0 on the Raspberry Pi 2 and maybe some of the things I learned are interesting for others.
* Running appscale on a single Raspberry Pi 2 is possible when using the Ubuntu Trusty and applying the Cassandra memory tweaks and reducing the GPU share of the main memory to 16 MB and (!) reducing the number of virtual consoles from 6 to just a single one to have a few more MB of main memory available. But: Even if the PaaS starts, it is not usable at all. Main memory consumption in idle mode was according to the web GUI between 98 and 103%. I did not even succeed in deploying this simple guestbook application
As result, the web GUI was not accessible any more and I had to restart appscale.
* A simple solution to avoid the memory problems is just using more Raspberry Pi. I copied the image two times, modified the hostnames and IPs and modified the AppScalefile in a way that I have one master (which also hosts appengine and zookeeper), one dedicated DB node and another node with appengine + zookeeper.
ips_layout:
master : 10.0.2.102
appengine:
- 10.0.2.102
- 10.0.2.104
database: 10.0.2.103
zookeeper:
- 10.0.2.102
- 10.0.2.104
As result, I had more free main memory on the nodes and it was no problem to deploy and use the simple guestbook application:
# appscale status
Status of node at 10.0.2.102:
Currently using 22.98 Percent CPU and 74.00 Percent Memory
Hard disk is 30 Percent full
Is currently: load_balancer, taskqueue_master, zookeeper, taskqueue, memcache, shadow, login, appengine
Database is at 10.0.2.103
Is in cloud: cloud1
Current State: Done starting up AppScale, now in heartbeat mode
Hosting the following apps: none, guestbook27
Information for application: guestbook27
Language : python27
Number of AppServers: 1 running
HTTP port : 8080
HTTPS port : 4380
Status of node at 10.0.2.104:
Currently using 0.00 Percent CPU and 30.00 Percent Memory
Hard disk is 30 Percent full
Is currently: memcache, zookeeper, appengine, taskqueue_slave
Database is at 10.0.2.103
Is in cloud: cloud1
Current State: Done starting up AppScale, now in heartbeat mode
Status of node at 10.0.2.103:
Currently using 31.45 Percent CPU and 66.00 Percent Memory
Hard disk is 30 Percent full
Is currently: db_master
Database is at 10.0.2.103
Is in cloud: cloud1
Current State: Done starting up AppScale, now in heartbeat mode
But as drawback, the platform service was not quite stable. After 1-3 hours, it gets unusable again.
"appscale status" gives back the status (and does not indicate any problems) but the web GUI is not accessible any more and the same applies for the deployed application.
In the controller-17443.log I found theses lines which may be related to the problem.
I, [2016-09-20T14:03:05.990834 #24372] INFO -- : --- Node at 10.0.2.102 has 347MB memory available and knows about these apps {"appscaledashboard"=>{"language"=>"python", "appservers"=>0, "pending_appservers"=>0, "http"=>1080, "https"=>1443, "total_reqs"=>0, "reqs_enqueued"=>0}, "guestbook27"=>{"language"=>"python", "appservers"=>1, "pending_appservers"=>0, "http"=>8080, "https"=>4380, "total_reqs"=>0, "reqs_enqueued"=>0}}.
...
W, [2016-09-20T14:03:16.199367 #24372] WARN -- : Couldn't send instance info to the AppDashboard because of a Errno::ECONNREFUSED exception.
W, [2016-09-20T14:03:16.911488 #24372] WARN -- : Didn't see any monitoring info - appscaledashboard may not be running.
W, [2016-09-20T14:03:16.911963 #24372] WARN -- : Didn't see any request data - not sure whether to scale up or down.
I, [2016-09-20T14:03:16.949750 #24372] INFO -- : Couldn't send request info for app guestbook27 to https://10.0.2.102:1443/apps/json/guestbook27
I, [2016-09-20T14:03:17.736096 #24372] INFO -- : Reloading nginx service.
...
W, [2016-09-20T14:06:44.987399 #24372] WARN -- : [start_app] SOAP call to 10.0.2.102 timed out
W, [2016-09-20T14:06:44.987938 #24372] WARN -- : FailedNodeException encountered while starting appscaledashboard with AppManager: Time out talking to 10.0.2.102:17445
W, [2016-09-20T14:06:44.988179 #24372] WARN -- : Something went wrong starting AppServer for appscaledashboard: check logs and running processes as duplicate ports may have been allocated.
W, [2016-09-20T14:06:48.022319 #24372] WARN -- : Couldn't send instance info to the AppDashboard because of a Errno::ECONNREFUSED exception.
W, [2016-09-20T14:06:48.734597 #24372] WARN -- : Didn't see any monitoring info - appscaledashboard may not be running.
W, [2016-09-20T14:06:48.735123 #24372] WARN -- : Didn't see any request data - not sure whether to scale up or down.
I, [2016-09-20T14:06:48.787537 #24372] INFO -- : Couldn't send request info for app guestbook27 to https://10.0.2.102:1443/apps/json/guestbook27
I, [2016-09-20T14:06:48.813366 #24372] INFO -- : Starting TaskQueue worker for app appscaledashboard: {"error"=>false}
I, [2016-09-20T14:06:48.813769 #24372] INFO -- : Done adding AppServer for appscaledashboard.
I, [2016-09-20T14:06:50.882260 #24372] INFO -- : Starting first AppServer for app: appscaledashboard.
I, [2016-09-20T14:06:50.882845 #24372] INFO -- : Received request to add an AppServer for appscaledashboard.
I, [2016-09-20T14:06:51.461997 #24372] INFO -- : Reloading nginx service.
I, [2016-09-20T14:06:51.539152 #24372] INFO -- : This node is now hosting appscaledashboard source (Found appscaledashboard in zookeeper.).
I, [2016-09-20T14:06:51.754138 #24372] INFO -- : Starting python27 app appscaledashboard on 10.0.2.102:20000
W, [2016-09-20T14:07:01.951445 #24372] WARN -- : Couldn't send instance info to the AppDashboard because of a Errno::ECONNREFUSED exception.
W, [2016-09-20T14:07:02.641011 #24372] WARN -- : Didn't see any monitoring info - appscaledashboard may not be running.
W, [2016-09-20T14:07:02.641460 #24372] WARN -- : Didn't see any request data - not sure whether to scale up or down.
I, [2016-09-20T14:07:02.675016 #24372] INFO -- : Couldn't send request info for app guestbook27 to https://10.0.2.102:1443/apps/json/guestbook27
I, [2016-09-20T14:07:03.437048 #24372] INFO -- : Reloading nginx service.
* I tried using the Ubuntu Trusy image ...
on a Raspberry Pi 3 because this could speed everything up a bit but this was a total fail. It does not even boot. A port of Ubuntu Mate exists ...
but it is based on Ubuntu 16 Xenial and this is not (yet?) supported by appscale.
And anyway, using the Raspberry Pi 3 would not resolve the problem of the quite small main memory.
* Summary:
Appscale is an impressive software, but I agree now that the Raspberry Pi 2 (especially when using Cassandra) is because of the tiny main memory not the best suited platform for running it. But I will continue to play around with it. Maybe it gets more stable when I do not use redundant services. At the moment I test a configuration where I have a dedicated master, a dedicated DB and one node with appengine + zookeeper.
ips_layout :
master : 10.0.2.102
appengine : 10.0.2.104
database : 10.0.2.103
zookeeper : 10.0.2.104
It runs stable since approx. 1 hour... ;-)
Best Regards
Christian
Reply all
Reply to author
Forward
0 new messages