branch vs copy

[Nathan Tippy]

From Martins talk on InfoQ here is the slide showing the priority of design principles from Aeron

Design principles:

1. Garbage free in stead state running

2. Smart batching in the message path

3. Wait-free algos in the message path

4. Non-blocking IO in the message path

5. No exceptional cases in the message path

6. Apply the single writer principle

7. Prefer unshared state

8. Avoid unnecessary data copies

Given the above where does one balance between conditionals and extra data? 

In my messages I have a field that I add on some messages but not others.  The conditional

causes a branch failure at times so my thought is I should always add the field. In this way

I am assuming that the extra data copy is a better choice than a conditional that may stall.

Is this assumption right and is the "minimize of conditionals" something that belongs on the 

above list?  And if it belongs on the list is it above or below #8.  Or is this in fact #5?

[Vitaly Davidovich]

This is probably one of those things where you have to try both approaches to see what works best.  Generally speaking, it's best to avoid *unpredictable* branches; predicted branches in and of themselves aren't bad.  Determining whether a branch is predicted by the processor is somewhat difficult (modulo some pathological cases where there's truly no pattern present), and varies from processor to processor.

As for copying a member that's not always needed, that depends as well.  Is the data for that field likely to be present in the cache hierarchy of the writer? Is the extra field going to make the message sufficiently larger such that more frequent data is now pushed out to a different cache line? If the data being copied is small and the source and destination cache lines are likely to already be present, then the copy is virtually free.

A memory read that misses last level cache is going to be much more expensive than a single branch misprediction, so if anything, minimizing cache misses is of utmost importance in modern processors.

But there's lots of "assuming this and that" in the above, so you'd need to try it out and see if it actually makes any difference.

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Todd Montgomery]

Minimizing "random" or unpredictable conditionals is a great thing and fits in with #5 for Aeron.

But it depends. For Aeron, we eliminated all optional headers to avoid the "random" branching overhead, for example. In that case, while many things could have been done with header chaining, etc. all it really would have done is create more unpredictable branches in the main line of processing on the receiver. And add more complexity to the sender and more unpredictable branches. Also, we wanted fixed size data frame headers for ease of integration with log buffers.

-- Todd

[Martin Thompson]

As Vitaly has said measurement is the the most important to your decision making. 

As a general approach if data can be strided I prefer to have extra data over conditionals. Bandwidth is seldom an issue. If the memory access pattern is arbitrary then the cache misses will dominate over everything else, so best to minimize the cache misses with compact data.

Rather than take branches we can often do a bit of math so things stay constant. For example, we want to align frames on word boundaries. Rather than branch on this there are standard compiler tricks like the following:

https://github.com/real-logic/Agrona/blob/master/src/main/java/uk/co/real_logic/agrona/BitUtil.java#L113