Hey Vincent,
The upper limit of 20 is intentional, so you will need to make numerous calls to actually retrieve all followers.
Regarding the 429 responses, we have lower rate limits set for requesting followers. You can see your current limit info by looking at the X-RateLimit-* headers. However, I would note that you should treat these only as informational, and not rely on their behavior or presence.
It's hard to say what your php.ini settings should be. You should be able to store the data for all those followers in a moderate amount of space, but may need additional execution time since you'll need to slow down your requests.
Regards,
-Mike Ceyko