Make sure you have the service account defined in
https://console.developers.google.com in your calendar API (which you probably already do) and the calendar you are attempting to access is WITHIN the domain of the service account.
And then in
https://admin.google.com -> more controls (at the bottom) -> Security -> Advanced Settings (show more) -> Manage API client access.
The setting there should read
**unique_string_from_credentials**.
apps.googleusercontent.com :
https://www.googleapis.com/auth/calendarwhere
Service account : scope
You may also need
https://admin.google.com -> Google Apps -> Calendar -> Sharing Settings -> Internal sharing should be set to “Share all information”
If you need more explanation please feel free to ask.
Not 100% sure but my service account was created by a user who is an admin of the app domain as well.