Is it possible to throttle replication speed?

[Brendan Duddridge]

Hi,

I have a beta tester right now who is pulling his hair out of his head because when my app is replicating either between peers using peer-to-peer networking or via the cloud to IBM Cloudant, he says his network performance is suffering. He says that on his iOS devices he's getting kicked off of WiFi and for no apparent reason he's being bumped over to the 4G cellular network.

So I thought I'd ask to see if there was a way that I could enable some sort of request throttling so that replication doesn't use quite so much network resources.

Thanks,

Brendan

[Jens Alfke]

On May 20, 2016, at 1:54 PM, Brendan Duddridge <bren...@gmail.com> wrote:

Hi,

I have a beta tester right now who is pulling his hair out of his head because when my app is replicating either between peers using peer-to-peer networking or via the cloud to IBM Cloudant, he says his network performance is suffering. He says that on his iOS devices he's getting kicked off of WiFi and for no apparent reason he's being bumped over to the 4G cellular network.

That’s weird; I’ve never heard of a problem like that before*. All we do is regular HTTP requests (ok, and WebSockets for the changes feed, but that’s low-volume). That shouldn’t cause any more trouble than, say, browsing Instagram or syncing your Dropbox. Honestly I don’t think the replicator even has the throughput to saturate a modern WiFi connection.

My guess is there’s something screwy with his base station / router. Is this guy technical enough to do some troubleshooting?

So I thought I'd ask to see if there was a way that I could enable some sort of request throttling so that replication doesn't use quite so much network resources.

We don’t have a setting for that; honestly, we haven’t gotten a request for it, while we do get requests to make replication faster. :/

—Jens

* except with BitTorrent, which has the capacity to overwhelm a badly-designed router with traffic, but they switched to the µTP protocol a few years ago to alleviate that.

[Brendan Duddridge]

Unfortunately I don't know if he's technical enough to do proper debugging on this. He lives in Europe somewhere and there's a language barrier between us, so it can be challenging to understand him sometimes. He's gone as far as rebooting his router, but he just tells me that I should give up on developing my app until I can fix this problem. I don't really know what to do about it or why he's having so much trouble.

I can't make too much sense out of this, but this is what he has said:

Please believe me….

We are losing time!!!

There is something going realy wrong with Cloudant

Even nearby suffers

Losing internet every +- 1 hour

Going over to G4 without doing anything….

Losing even wifi…(in your house)

Reseting rooter…

Speedtest on Mac stays mostly OK

Speedtest on Devices is complytly zero

Sometimes on devices…

Less times on Mac….

It does not work for your future app!!!!!!!!!!!!!!!!

We are trying to make something nice & new but based on “Sync” ….. Wrong direction!

Please do not develop your nice program anymore before to be sure off the “basic”

So you can see how challenging it is to figure out the problem :-)

I'll ask him what kind of router he has and see if there's any chance of doing some network diagnostics.

Thanks,

Brendan 

[Brendan Duddridge]

So I found out that my beta tester has an ADSL Internet connection with about 20 mbit/s download speed and about 2 mbit/s upload speed. 

Because replication is doing a lot of uploading, it's saturating his available bandwidth. And because he has an ADSL connection I've read, the download performance can suffer greatly when you're using up a lot of the upload bandwidth.

He showed me some speedtest.net output and I can definitely see that when my app is replicating, his performance drops considerably during the speed test. I have a very fast cable connection to the Internet with approximately 120 mbit/s download and 10 mbit/s upload speed. I noticed that while I was replicating a new database with 117,000 documents in it, I was using up just over 2 mbit/s of my upload bandwidth according to speedtest.net. But I still had plenty of bandwidth left of course. In fact, while I was doing all this testing I also was watching a show on Netflix. I still had plenty left even with that.

I read an article about this ADLS problem and they said the only solution is to have the application throttle the upload speed so it doesn't consume so much of the upload bandwidth.

Anyway, just thought I'd update this thread with my findings.

Thanks,

Brendan

On Friday, May 20, 2016 at 6:05:13 PM UTC-6, Jens Alfke wrote:

[Jens Alfke]

On Jun 3, 2016, at 9:59 AM, Brendan Duddridge <bren...@gmail.com> wrote:

Because replication is doing a lot of uploading, it's saturating his available bandwidth. And because he has an ADSL connection I've read, the download performance can suffer greatly when you're using up a lot of the upload bandwidth.

I’ve had ADSL at home for all but a few of the last 16 years, and I don’t remember having this sort of problem. And for a long time my upload rate was well under 1mbit/sec. But I know there are different types of ADSL and maybe his is different.

The thing is, I don’t know what I could do about this. NSURLConnection and NSURLSession are the black boxes doing the networking in CBL, and they don’t offer any settings to limit bandwidth. In addition, any fixed limit isn’t going to work all the time since the total bandwidth depends on all processes on all devices. (BitTorrent is able to get around this, but by some clever heuristics that only work because they’ve implemented their network protocol on top of UDP so they have control over every packet they send.)

I’d suggest asking on the macnetworkprog mailing list at lists.apple.com. Various Apple networking gurus like Quinn and Josh Graessley hang out there and are very helpful.

—Jens

[Brendan Duddridge]

Ok cool. Thanks Jens.