base::ByteCount

[Avi Drissman]

I have a question about byte::BaseCount. We don't seem to have a base/ owners list, and this touches on C++, so this list is somewhat appropriate.

There's an ongoing switchover to base::ByteCount, which is a strongly-typed int64_t wrapper. Peter brought up concerns that a signed type was chosen rather than an unsigned one, and while I personally lean the other way (we need to handle negative byte amounts, we generally prefer signed types) his point about this decision being underdocumented is legitimate.

I have multiple refactoring CLs that I'd like to finish up and land, but I don't want to do so if there's murkiness about whether we're all on-board with this as-is.

Would we be able to come to consensus about this? Thanks.

Avi 

[K. Moon]

Do we actually need both a signed and unsigned version of this? You mentioned needing negative byte counts, but maybe that's situational?

Although I'm personally OK with just having a signed version if the negative byte counts are a hard requirement anyway.

--
You received this message because you are subscribed to the Google Groups "cxx" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cxx+uns...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/cxx/CACWgwAZ7UKKCkfX7Pqcqj7jsN%2BrzVkRhid-eZp7jFefjfy2ViA%40mail.gmail.com.

[Avi Drissman]

+François Doray who started base::ByteCount and who I missed when writing this thread.

[Alexei Svitkine]

My 2c is that it seems useful to be able to have negative ByteCounts, without resorting to different types or having to pass by just a number when a param can accept both negative and positive.

So IMHO, signed SGTM and we could just document the rationale in the header.

I guess the downside is code may need to validate that values passed are non-negative, but imho code should likely be doing validation anyway for values too large as well, so it's not much difference.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/cxx/CACWgwAYYyyVhGTncgDr4ZEYkfDrZ6m4cFsCOHT2Pq9VBXEWGUg%40mail.gmail.com.

[Marc Treib]

Do you have examples for situations where negative ByteCounts are needed? From a distance, having something called ByteCount be negative seems quite surprising to me.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/cxx/CAKFU2SC84EpV4FE70sZCawj%2BwB2fRmwb3D04XvuGZRewj09CFQ%40mail.gmail.com.

[Peter Kasting]

On Thu, Aug 28, 2025 at 7:38 AM Marc Treib <tr...@chromium.org> wrote:

Do you have examples for situations where negative ByteCounts are needed? From a distance, having something called ByteCount be negative seems quite surprising to me.

This was my thought too. The idea of subtracting a positive byte count (e.g. "how much of the input stream is left after we skip past the header") is valuable; but I don't know what a negative byte count means (if the input stream in said example is still in the header, then IMO the answer is not a negative byte count, but either "zero" or "error, that subtraction wasn't safe/sane right now" -- that is, either the ClampedNumeric or CheckedNumeric behaviors).

That said, I'm less worried about the semantic sanity of negative byte counts than I am about the practical potential for problems, given that our previous type for byte counts in most places was unsigned (size_t). Signedness conversions are scary, especially with sizes of objects in memory; the potential to paper over security holes by silencing warnings with static_casts (or actually introduce such holes) seems high.

PK

[Joe Mason]

Free space with a "reserved" watermark. If free space falls below the watermark, the API can report "negative space free" since it's now in the red zone and become critical to free resources.

I think an unsigned ByteCount would be useful too. The standard practice in C++ is to use a "u" or "unsigned" prefix for unsigned, and no prefix for signed (eg. int32_t vs uint32_t, IdType32 vs IdTypeU32 in base/types/id_type.h), so I'd suggest leaving ByteCount as signed and add UByteCount for unsigned (if there's enough demand).

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/cxx/CAOXvDB2BDSWZM6ToH_6OkZb6Fi44KKXZ_9CW1DmVONzvffuqmw%40mail.gmail.com.

[K. Moon]

I do worry that a negative byte "count" is maybe conflating concepts we don't want to conflate. Differences aren't really the same thing as a size, even if they're measured in the same units (bytes). We see this sort of dichotomy with time types, for example.

[Alexei Svitkine]

My preference is signed per my earlier reply, but imho the biggest pro for unsigned is:

unsigned

+ When taking this type as a param, can assume it's not negative.

On Thu, Aug 28, 2025 at 4:10 PM François Doray <fdo...@google.com> wrote:

The use of a signed integer for base::ByteCount was inspired from the use of a signed integer for base::TimeDelta. 

Comparing the 2 choices:

signed

+ Subtracting [large value] from [small value] works.

+ Can represent a possibly negative byte delta [example]

- Cannot represent values larger than 8EiB (my assumption is that this is rarely needed)

unsigned

- Subtracting [large value] from [small value] is an error.

- Cannot represent a possibly negative byte delta [example]

+ Can represent value larger than 8EiB

[Joe Mason]

Another +ve of unsigned is that the overflow behaviour of unsigned integers is well-defined (it wraps around) while signed integers work in mysterious ways on overflow.

Unsigned integers are preferred in security-sensitive code (like the size of buffers) because of this.

I don't think ByteCount is intended for buffer sizes, though. The main purpose is for measurements that are usually given in kiB, MiB, etc, to avoid confusion over which unit this particular variable holds. As a security reviewer, I'd look oddly at a ByteCount used to exactly measure a buffer size, since I'd expect that ByteCounts are used for conversions and might well be rounded. Maybe the name could be improved? (I don't want to get into too much byteshedding, though.)

Speaking of TimeDelta, maybe make ByteCount unsigned and name the signed equivalent ByteDelta?

On Thu, Aug 28, 2025 at 4:10 PM François Doray <fdo...@google.com> wrote:

The use of a signed integer for base::ByteCount was inspired from the use of a signed integer for base::TimeDelta. 

Comparing the 2 choices:

signed

+ Subtracting [large value] from [small value] works.

+ Can represent a possibly negative byte delta [example]

- Cannot represent values larger than 8EiB (my assumption is that this is rarely needed)

unsigned

- Subtracting [large value] from [small value] is an error.

- Cannot represent a possibly negative byte delta [example]

+ Can represent value larger than 8EiB

On Thu, Aug 28, 2025 at 3:15 PM K. Moon <km...@chromium.org> wrote:

[Peter Kasting]

On Thu, Aug 28, 2025 at 1:10 PM François Doray <fdo...@google.com> wrote:

The use of a signed integer for base::ByteCount was inspired from the use of a signed integer for base::TimeDelta.

As Kahmy said, though, this conflates differences and sizes, which are measured in the same units here but aren't the same concept. `base::TimeDelta` says in the name that it's a difference, so a negative value makes semantic sense. `ByteCount`, OTOH, is a size, and a negative size isn't meaningful.

As Joe alludes to, if you want to be able to mean both, you should probably have two separate types, ByteSize and ByteSizeDelta. The former is unsigned, the latter is signed. Subtracting one ByteSize from another gives a ByteSizeDelta, and so forth. This is precisely how the Time[Ticks] vs. TimeDelta types work. This also lets functions be clearer about which concept they mean, and makes it more obvious when sanity-checking for negatives is important.

PK

[Leszek Swirski]

Having signed types for diff and unsigned types for size has its own caveats, like how there would be cases of undefined subtraction between two valid sizes (like how ptrdiff_t can be UB for large enough pointer subtraction); unless ByteCount was constrained to the intersection of int64_t and uint64_t range, in which case it doesn't super matter which underlying representation it has: we either have dodgy code when it exits its valid range or it needs to anyway be runtime checked for exceeding its valid range.

- Leszek

--

You received this message because you are subscribed to the Google Groups "cxx" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cxx+uns...@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/cxx/CAO5B8Rs%2BRYZ45AU6cJcCdiJw6BtQFGXg4druD%3D%2BS%3D1anD_nFZw%40mail.gmail.com.

[François Doray]

The use of a signed integer for base::ByteCount was inspired from the use of a signed integer for base::TimeDelta. 

Comparing the 2 choices:

signed

+ Subtracting [large value] from [small value] works.

+ Can represent a possibly negative byte delta [example]

- Cannot represent values larger than 8EiB (my assumption is that this is rarely needed)

unsigned

- Subtracting [large value] from [small value] is an error.

- Cannot represent a possibly negative byte delta [example]

+ Can represent value larger than 8EiB

On Thu, Aug 28, 2025 at 3:15 PM K. Moon <km...@chromium.org> wrote:

[François Doray]

base::ByteCount uses CheckedNumeric to ensure that no overflow goes unnoticed.

I decided not to have separate types for "absolute quantity" and "delta" when I implemented the API, because we cannot deduce the right type to use for the result of an operation without extra context:

base::ByteCount total_ram;

base::ByteCount used_ram;

base::ByteCount available_ram = total_ram - used_ram; // we want the result of this subtraction to be a ByteSize

base::ByteCount previous_size;

base::ByteCount new_size;

base::ByteCount size_delta = new_size - previous_size; // we want the result of this subtraction to be a ByteSizeDelta

base::ByteCount bytes_transferred_at_beginning_of_interval;

base::ByteCount bytes_transferred_at_end_of_interval;

base::ByteCount bytes_transferred_during_interval; // depending on the larger context, we could argue that this is a ByteSize (absolute amount of bytes transferred during the interval) or a ByteSizeDelta (variation of the total amount of bytes transferred)

base::ByteCount with a signed integer ensures that a subtraction of normal quantities of bytes is always well-defined, and it's up to the calling code to decide how to handle negative values. asvitkine@ is right that a downside of this implementation choice is that code receiving a ByteCount cannot assume that it's a positive value.

On Fri, Aug 29, 2025 at 10:40 AM Peter Kasting <zer...@gmail.com> wrote:

On Fri, Aug 29, 2025 at 1:18 AM Leszek Swirski <les...@chromium.org> wrote:

Having signed types for diff and unsigned types for size has its own caveats, like how there would be cases of undefined subtraction between two valid sizes (like how ptrdiff_t can be UB for large enough pointer subtraction); unless ByteCount was constrained to the intersection of int64_t and uint64_t range, in which case it doesn't super matter which underlying representation it has: we either have dodgy code when it exits its valid range or it needs to anyway be runtime checked for exceeding its valid range.

Using CheckedNumeric under the hood would eliminate this problem, as it would automatically do the runtime check that the result is valid.

It would also constrain the set of problem cases to O(1): the implementation of these types themselves. This is preferable to potentially having problems at the usage sites.

PK

[Peter Kasting]

On Fri, Aug 29, 2025 at 1:18 AM Leszek Swirski <les...@chromium.org> wrote:

Having signed types for diff and unsigned types for size has its own caveats, like how there would be cases of undefined subtraction between two valid sizes (like how ptrdiff_t can be UB for large enough pointer subtraction); unless ByteCount was constrained to the intersection of int64_t and uint64_t range, in which case it doesn't super matter which underlying representation it has: we either have dodgy code when it exits its valid range or it needs to anyway be runtime checked for exceeding its valid range.

[Peter Kasting]

On Fri, Aug 29, 2025 at 9:39 AM François Doray <fdo...@google.com> wrote:

I decided not to have separate types for "absolute quantity" and "delta" when I implemented the API, because we cannot deduce the right type to use for the result of an operation without extra context:

base::ByteCount total_ram;

base::ByteCount used_ram;

base::ByteCount available_ram = total_ram - used_ram; // we want the result of this subtraction to be a ByteSize

base::ByteCount previous_size;

base::ByteCount new_size;

base::ByteCount size_delta = new_size - previous_size; // we want the result of this subtraction to be a ByteSizeDelta

base::ByteCount bytes_transferred_at_beginning_of_interval;

base::ByteCount bytes_transferred_at_end_of_interval;

base::ByteCount bytes_transferred_during_interval; // depending on the larger context, we could argue that this is a ByteSize (absolute amount of bytes transferred during the interval) or a ByteSizeDelta (variation of the total amount of bytes transferred)

Agree that these sorts of distinctions are important, and I appreciate your illustrative choices. I don't know that using a signed type unconditionally is the ideal answer, though.

One route (akin to some things base/time and ui/gfx/geometry do) would be to provide `ByteSize base::ByteSizeDelta::AsByteSize() const;`, which is effectively a `checked_cast`. So you'd have:

base::ByteSize total_ram;
base::ByteSize used_ram;
base::ByteSize available_ram = (total_ram - used_ram).AsByteSize();  // CHECKs if used_ram > total_ram

This way callers can make their intent clear. While it looks cumbersome at first, it's safer than leaving "are negative values possible/meaningful here" to call sites, and I suspect it wouldn't actually be used as frequently as people think.

(On its face this may appear to mirror size_t and ptrdiff_t, but there are important differences: subtracting two size_ts gives a size_t and may easily underflow, while subtracting two ByteSizes gives a ByteSizeDelta and will CHECK if the result is outside the representable range; and conversion between size_t and ptrdiff_t is two way and likely implicit (depending on compiler flags), while conversion between ByteSize and ByteSizeDelta is one-way and explicit. These distinctions reduce the footgun possibilities significantly.)

PK

[K. Moon]

I agree that you want to explicitly convert from a byte difference to a byte size. The available RAM example has me wondering how this is going to handle potentially negative available RAM. Using the right types eliminates that class of problems.

The conversions are only context-dependent because they're missing the explicit intent they should have.

[Avi Drissman]

Are we making progress on a decision? Should I land my pending ByteCount CLs while we discuss this further or should I continue to hold off?

Avi

[K. Moon]

I think using ByteCount in an unsigned way would be unobjectionable to everyone. I think using it for differences doesn't have consensus.

My vote would be for two types here, a count type and a difference type, or something along those lines.

[Peter Kasting]

By "using in an unsigned way" do you mean making the underlying type unsigned and using everywhere, making it unsigned and using only for non-differences, leaving it signed and using only for non-differences and inserting casts when necessary, ...?

I ask because my initial suggestion was basically "make unsigned and use everywhere" and it definitely didn't seem like that was unobjectionable to everyone. 

PK

[K. Moon]

My suggestion boils down to, "If this wouldn't change if the underlying type became unsigned, you can keep going. Otherwise, wait."

I agree that there isn't consensus on what these types should mean/be implemented, but treating a ByteCount as a count seems fine.

[Peter Kasting]

Sure, I think use that would literally require no changes (adding/removing casts) if we were to change the underlying type is uncontroversial. Knowing whether that's the case might be less obvious :)

PK

[Alexei Svitkine]

Re: 

base::ByteSize total_ram;
base::ByteSize used_ram;
base::ByteSize available_ram = (total_ram - used_ram).AsByteSize();  // CHECKs if used_ram > total_ram

That SGTM too. My vote for a signed type was in the scenario where we just have one type. But the ByteSizeDelta and above helper approach SGTM.

[Jeremy Roman]

If I might bikeshed, ByteSize (or ByteCount) vs ByteOffset would be an option (off_t is analogous in name even if ptrdiff_t isn't).

--

You received this message because you are subscribed to the Google Groups "cxx" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cxx+uns...@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/cxx/CAKFU2SD2x-%3D4eJGSe3%2BY46x99X1xzs%3DrSze6zn7SsKgbdtH9BQ%40mail.gmail.com.

[Marc Treib]

+1 to ByteSize + ByteSizeDelta (or ByteCount / ByteOffset - I'm not gonna participate in the bikeshed ;P)

Marc Treib

Software Engineer

tr...@google.com

Google Germany GmbH

Erika-Mann-Straße 33 80636 München

Geschäftsführer: Paul Manicle, Liana Sebastian

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich. Falls sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde. This e-mail is confidential. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.

[Peter Kasting]

Yes, ByteOffset seems mildly better than ByteSizeDelta IMO.

PK

[K. Moon]

I kinda like consistency with the similar time types, but I think the important thing is that it sounds like there's consensus around having 2 types.

Presumably can settle the bike shed color in the code review? 🙂

[Avi Drissman]

@François Doray you were the originator of the type; do you have an opinion here?

Right now, ByteCount seems to be in limbo, and either we should agree on a way forward so that work can continue, or we should unwind things.

Avi

[François Doray]

Hi everyone!

Thanks for sharing your thoughts. I'm now convinced that our best end state is an unsigned ByteSize and a signed ByteSizeDelta.

• Code consuming ByteSize can safely assume the value is non-negative and doesn't need to handle negative cases. This is the typical case.
• As pkasting@ demonstrated, we can have type conversion for cases where a difference conceptually yields a ByteSize, instead of a ByteSizeDelta:

• base::ByteSize available_ram = (total_ram - used_ram).AsByteSize();

However, my primary concern is the migration path. A reason why I initially went with the signed ByteCount, is because our existing codebase uses signed integers for "byte sizes" [examples: 1, 2, 3]. My assessment was that negative byte counts were more common than positive counts exceeding the max int64_t, so using int64_t would make the migration easier (no need to change logic as part of the migration). This assessment seems correct, as the migration to ByteCount has progressed without triggering CheckedNumeric failures [code]. 

Given this, I would be more comfortable reaching our end state through the following intermediate steps:

Step 1:

Introduce ByteSize and ByteSizeDelta, but keep ByteCount.

• ByteCount (signed, with a comment recommending ByteSize/ByteSizeDelta instead)
• ByteSize (unsigned)
• ByteSizeDelta (signed)
• ByteSize/Count - ByteSize/Count → ByteSizeDelta
• ByteSize + ByteCount → ByteCount
• ByteSizeDelta::AsByteSize() to allow conversion

Step 2:

• Disallow construction of ByteCount from unsigned variables or integer literals at compile-time. This forces all simple cases to migrate to ByteSize. We can still create ByteCount from a signed integer variable.
• Use runtime checks (DCHECKs, then DumpWithoutCrashing) to find instances of ByteCount holding a negative value, so we can make necessary adjustments.

Step 3:

• Once we have sufficient confidence that all remaining ByteCount instances represent positive values, we migrate all ByteCount usage to ByteSize and remove ByteCount.
• We probably want ByteSize::FromSignedBytes/FromSignedKiB/FromSignedMiB/etc. static methods to highlight cases where a ByteSize is created from a signed integer. A comment on this method would indicate that the calling code must ensure that the value is positive, and this is enforced with runtime checks.
  

What do you think of this phased approach?

François

[Daniel Cheng]

The staged rollout seems like a lot of work, but probably the safest approach. For step 3, one suggestion would be to have the "from signed" conversion helpers return an optional and have the caller decide if they want to handle it gracefully or not, but I don't feel strongly about that.

Naming-wise, I'd prefer the name ByteOffset just to make it a bit shorter, and it has precedence with `off_t` (which is admittedly not the best name) :)

80 columns is not a lot.

Alternatively we could do `ByteDelta` for consistency with `TimeDelta` but `ByteDelta` doesn't seem like the best name.

Daniel

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/cxx/CAGD3t5EmtL6e%2BKpkpbH8YADPKDTeTS1C9n2oQY70KgyfseaKUw%40mail.gmail.com.

[K. Moon]

Glad to see this is moving in a productive direction!

I think a gradual migration can make sense if there's a lot of existing usage that needs to be fixed up. (Note that this excludes existing usage that is already correct as unsigned.) If there isn't, then it's probably not worth it. I don't have data on which would be better in this case.

As far as construction goes, I'd try to emulate CheckedNumeric. It should be impossible to ever construct a "signed" ByteSize anyway, so I don't think we need a bunch of helpers to enforce this, necessarily, but no strong opinions if some would still be useful.

I think ByteDelta would only make sense if the fundamental type was Byte, which doesn't seem great. So +1 to sticking with ByteSize/ByteSizeDelta, or something similar.

Similarity in naming is great for APIs because of Principle of Least Astonishment: You see a thing, and another similarly-named thing behaves similarly. Which is why I like the Foo/FooDelta naming.