MSAK download throughput estimation

[Ray Tseng]

Hi,

I have a questions about getting Estimated tput based on the payload when using MSAK download test. The following is my query. Currently, I group the same measurement ID which I think belong to the same test.

Then sum all the data acked and divide by ElapsedTime to get estimated tput.

It might not be a accurately estimation, but I would like to get some suggestion if this is a

possible way to guess what payload should be used to determine if network has enough bandwidth. Also, I have a question about ServerMeasurements.TCPInfo.BytesAcked vs ServerMeasurements.TCPInfo.BytesSent. Which one is really indicated the data that sent from Server to client? and should be used for estimation?

Thanks

Ray

WITH

test_time_range AS (

SELECT raw.MeasurementID as mid, min(raw.StartTime) as test_start_time, min(raw.EndTime) as test_end_time

FROM `measurement-lab.msak_raw.throughput1`

WHERE

date BETWEEN "2024-01-01" AND CURRENT_DATE

AND raw.Direction = "download"

GROUP BY raw.MeasurementID

-- Ignore tests longer than 30s.

HAVING TIMESTAMP_DIFF(test_end_time, test_start_time, SECOND) <= 30

),

stream_bytes_acked AS (

-- Get the last TCPInfo.BytesAcked for snapshots between the first stream that started

-- and the first stream that terminated.

SELECT raw.MeasurementId,

raw.UUID,

date,

ANY_VALUE(raw.CCAlgorithm) as cc,

r.test_start_time as StartTime,

r.test_end_time as EndTime,

max(sm.ElapsedTime) as elapsed,

max(sm.TCPInfo.BytesAcked) as max_bytes_acked

FROM `measurement-lab.msak_raw.throughput1` msak

JOIN UNNEST(raw.ServerMeasurements) sm

JOIN test_time_range r ON msak.raw.MeasurementID = r.mid

-- Verify that the test's start time + the measurement's elapsed time doesn't exceed end_time.

WHERE

UNIX_MICROS(msak.raw.StartTime) + sm.ElapsedTime <= UNIX_MICROS(r.test_end_time)

AND date BETWEEN "2024-01-01" AND CURRENT_DATE

AND raw.Direction = "download"

AND raw.CCAlgorithm = "bbr"

GROUP BY raw.MeasurementID, raw.UUID, date, r.test_start_time, r.test_end_time

)

SELECT

MeasurementID as id,

date,

STRUCT (

StartTime,

EndTime,

MAX(elapsed) as ElapsedTime,

SUM(max_bytes_acked) as Payload,

SUM(max_bytes_acked) / MAX(elapsed) * 8 as ThroughputMbps,

COUNT(*) as NumStreams,

ANY_VALUE(cc) as CongestionControl

) as details

FROM stream_bytes_acked

GROUP BY MeasurementID, date, StartTime, EndTime

Having

SUM(max_bytes_acked) <= 1048576 -- Total payload less than 1MB

AND SUM(max_bytes_acked) / MAX(elapsed) * 8 >= 25.0 -- TPut >= 25 Mbps

ORDER BY

details.ThroughputMbps,

details.Payload;

[Ray Tseng]

Btw, I modified the script I found under "throughput1_downloads" details section.
Also what is the unit of ServerMeasurements.ElapsedTime? is it second, NanoSec or MircoSec?

Thanks

[Roberto D'Auria]

Hi Ray,

I can answer some of your questions, but I'm not aware of any technique to estimate the minimum payload size that would be required to correctly represent the aggregate TCP throughput - if I understand that question.

>  Currently, I group the same measurement ID which I think belong to the same test. Then sum all the data acked and divide by ElapsedTime to get estimated tput.

That's correct, and also the intended way of grouping MSAK tests - MeasurementID is unique per test, and each row is a separate TCP stream.

> Also, I have a question about ServerMeasurements.TCPInfo.BytesAcked vs ServerMeasurements.TCPInfo.BytesSent.

> Which one is really indicated the data that sent from Server to client? and should be used for estimation?

BytesSent is the bytes that were sent, with no guarantee that they ever reached the client, BytesAcked is the bytes sent for which an ACK was also received, so bytes that reached the client (or a TCP-terminating middlebox, to be fair). In general, BytesSent > BytesAcked and BytesAcked is the right thing to use.

> Also what is the unit of ServerMeasurements.ElapsedTime?

Microseconds.

Hope this helps!

-Roberto

--
You received this message because you are subscribed to the Google Groups "discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@measurementlab.net.
To view this discussion visit https://groups.google.com/a/measurementlab.net/d/msgid/discuss/c636d6aa-26d0-47d8-aad5-7f71886e2212n%40measurementlab.net.

--

Roberto D'Auria

Platform Engineer

Measurement Lab

Code for Science & Society