I have two observations:
1) Your assumption that the coordinates returned are erroneous is based on the assumption that you are clicking on the exact same point on the map. How are you verifying that you have clicked on the exact same point at the different zoom levels? I don't really know how the click event works, but my guess is that when you click on the map, they detect the pixel that you have clicked on and derive the coordinates from it's centroid relative to the map coordinate system. One thing to keep in mind is that when you are zoomed out, one pixel on the map could represent 100's or 1000's of square kilometers, whereas when you are zoomed in closer, a pixel could represent only a 10's to 100's of square meters.
2) The native coordinate system of OpenLayers is not LatLon (EPSG:4326), it is a form of web mercator (EPSG:3857). In your code, you are transforming the coordinates retrieved by the mouse click event from 3857 to 4326, which may introduce round off errors. Unfortunately, I don't have any idea what algorithm OpenLayers uses for transformation.