Since I didn't know the answer to the Animation Layer question immediately, I can tell you what my process was for finding the solution...
First, I will turn on Echo All Command for the Script Editor, and do the actual Select Objects for the Anim Layer. The print out tells me that it uses a MEL command: layerEditorSelectObjectAnimLayer
This takes a []string array of anim layers, and selects them. From here, you could use maya.mel.eval() and call it with an array format like "{AnimLayer1}".
But if you want to get a more programmatic way to get the objects, I would then type the mel command:
whatIs layerEditorSelectObjectAnimLayer
This shows me the mel file that contains this definition. When I open the mel script up and read the code, I see that it recursively calls that same function on the layers, looking for the root layer, to which is then uses the python equivalent:
cmds.animLayer("AnimLayer1", q=True, attribute=True)
This will dump out all the attributes of that Anim Layer, like:
# Result: [u'pSphere1.visibility',
u'pSphere1.translateX',
u'pSphere1.translateY',
u'pSphere1.translateZ',
u'pSphere1.rotateX',
u'pSphere1.rotateY',
u'pSphere1.rotateZ',
u'pSphere1.scaleX',
u'pSphere1.scaleY',
u'pSphere1.scaleZ'] #
But I would suppose what we want really is the object list. So I would loop over those results, and split out the object name into a set, so that the results are unique:
objs = set(attr.split('.')[0] for attr in cmds.animLayer("AnimLayer1", q=True, attribute=True))
# Result: set([u'pSphere1']) #
Hope that helps!