Simian dashboard states No Clients were found

[Thomas Maerz]

So I'm deployed on AppScale for a year now. The certificate for the webserver expired and before I replaced it and for the period of time which it was expired, none of my clients could talk to the deployment. Once I updated the cert on my Appscale deployment everything started working as normal again EXCEPT the dashboard. I can push packages and do apple updates and all that, but when I go to the dashboard it says "No Clients were found". I can see all the clients if I look at the other screens like "Low Disk Space" etc.

I realize this is not a normal issue for most users since on GAE, the webserver certificate (not the simian certs) is handled by Google and never expires. 

Is there some kind of cache or key I can clear or something to make this dashboard start working again?

[Justin McWilliams]

The summary page should update every 4 hours (by default) to include any client which has checked in over the past 30 days: https://github.com/google/simian/blob/master/src/simian/mac/cron.yaml#L48

How long have you waited after fixing the certificate issue?  If you've already waited longer, are you positive clients are successfully connecting since the certs were fixed?

--
You received this message because you are subscribed to the Google Groups "Simian Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to simian-discuss+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thomas Maerz]

I have waited a few weeks, just getting back around to this issue. I am fairly certain clients are successfully connecting since the cert issue was fixed. I am seeing successful pre and postflights and able to install apps on the managed software update clients.

I am seeing this in the logs:

500 Traceback (most recent call last): File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__ rv = self.handle_exception(request, response, e) File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__ rv = self.router.dispatch(request, response) File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher return route.handler_adapter(request, response) File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__ return handler.dispatch() File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch return self.handle_exception(e, self.app.debug) File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch return method(*args, **kwargs) File "/var/apps/simian/app/simian/mac/cron/reports_cache.py", line 71, in get _GenerateTrendingInstallsCache(**kwargs) File "/var/apps/simian/app/simian/mac/cron/reports_cache.py", line 340, in _GenerateTrendingInstallsCache since_hours, query, None, 0, 0, trending) File "/var/apps/simian/app/simian/mac/cron/reports_cache.py", line 309, in _GenerateTrendingInstallsCacheDeferCallback query.cursor(), total_success, total_failure, trending) File "/root/appscale/AppServer/google/appengine/ext/deferred/deferred.py", line 268, in defer return task.add(queue, transactional=transactional) File "/root/appscale/AppServer/google/appengine/api/taskqueue/taskqueue.py", line 1132, in add return self.add_async(queue_name, transactional).get_result() File "/root/appscale/AppServer/google/appengine/api/apiproxy_stub_map.py", line 615, in get_result return self.__get_result_hook(self) File "/root/appscale/AppServer/google/appengine/api/taskqueue/taskqueue.py", line 1925, in ResultHook rpc.check_success() File "/root/appscale/AppServer/google/appengine/api/apiproxy_stub_map.py", line 581, in check_success self.__rpc.CheckSuccess() File "/root/appscale/AppServer/google/appengine/api/apiproxy_rpc.py", line 155, in _WaitImpl self.request, self.response) File "/root/appscale/AppServer/google/appengine/ext/remote_api/remote_api_stub.py", line 200, in MakeSyncCall self._MakeRealSyncCall(service, call, request, response) File "/root/appscale/AppServer/google/appengine/ext/remote_api/remote_api_stub.py", line 234, in _MakeRealSyncCall raise pickle.loads(response_pb.exception()) ProtocolBufferReturnError: 500

To unsubscribe from this group and stop receiving emails from it, send an email to simian-discus...@googlegroups.com.

[Justin McWilliams]

That's a different cron failing, /cron/reports_cache/trendinginstalls/*.

Filter logs to /cron/reports_cache/summary to see if anything shows up.  It seems possibly/likely that all crons using deferred are failing the same way though.  Seems like an AppScale problem.

To unsubscribe from this group and stop receiving emails from it, send an email to simian-discuss+unsubscribe@googlegroups.com.

[Thomas Maerz]

500
Traceback (most recent call last):
File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)

File "/var/apps/simian/app/simian/mac/cron/reports_cache.py", line 60, in get
_GenerateComputersSummaryCache()
File "/var/apps/simian/app/simian/mac/cron/reports_cache.py", line 382, in _GenerateComputersSummaryCache
deferred.defer(_GenerateComputersSummaryCache, query.cursor(), summary)

File "/root/appscale/AppServer/google/appengine/ext/deferred/deferred.py", line 268, in defer
return task.add(queue, transactional=transactional)
File "/root/appscale/AppServer/google/appengine/api/taskqueue/taskqueue.py", line 1132, in add
return self.add_async(queue_name, transactional).get_result()
File "/root/appscale/AppServer/google/appengine/api/apiproxy_stub_map.py", line 615, in get_result
return self.__get_result_hook(self)
File "/root/appscale/AppServer/google/appengine/api/taskqueue/taskqueue.py", line 1925, in ResultHook
rpc.check_success()
File "/root/appscale/AppServer/google/appengine/api/apiproxy_stub_map.py", line 581, in check_success
self.__rpc.CheckSuccess()
File "/root/appscale/AppServer/google/appengine/api/apiproxy_rpc.py", line 155, in _WaitImpl
self.request, self.response)
File "/root/appscale/AppServer/google/appengine/ext/remote_api/remote_api_stub.py", line 200, in MakeSyncCall
self._MakeRealSyncCall(service, call, request, response)
File "/root/appscale/AppServer/google/appengine/ext/remote_api/remote_api_stub.py", line 234, in _MakeRealSyncCall
raise pickle.loads(response_pb.exception())
ProtocolBufferReturnError: 500

Enter code here...

Thanks for the pointer. I'll see what the AppScale logs are saying about this code on the other side.

[Thomas Maerz]

It was AppScale related. Celery and amqp version mismatches due to apt-get errors during upgrade due to full /boot partition from too many old kernels sitting around. Not that anyone will probably ever have this problem, but that's what was wrong :) Cheers