remote_api, java, performance doing batch puts

[fredrossperry]

I've created a backup/restore capability for our customer data using
Java on the client side and remote_api on the server.

By default the SDK fetches in batches of 20. When I put data back to
the server, I am also doing batches of 20.

I worry about hitting per-request limits on data size and CPU time
doing it this way. Is there a way of estimating the "best" batch size
at run time?

-Fred

[Prateek Malhotra]

I think that's completely dependent on your models and any pre/post
processing you're doing. I'd play around with various sizes and
benchmark how well they perform. Possibly use transactions to
guarantee the writes and if you reach a deadline, have a standard
error code response and define a "next step" for your users to follow.
On reads you could have the user retry or retry with a smaller size.

Again, I think there is no "magic" number, you just need to bench what
fits your application.

[Tim Hoffman]

You logs will tell you your cpu_ms etc.. for each remote_api call.

You should be able to extrapolate a cpu cost per batch.

Rgds

T