Memory aligned to 64-bytes PooledByteBufAllocator

[Kiril Menshikov]

Hey,

How can use PooledByteBufAllocator to allocate 64-byte align memory.

Is it possible to know during memory request what memory address will be 64-byte aligned:

allocator.directBuffer(512, Integer.MAX_VALUE);

Do you have recommendations how to do it?

At the moment I can verify memorAddress and compute padding if necessary. But in this case I need to modify size as well (size+padding).

Thanks,

-Kiril

[Norman Maurer]

Why you need this ? 

There is currently not way to influence the alignment manual

[Norman Maurer]

I see…

yes its the right place to start (and also a few others). At some point I wanted to do something similar but could not prove any advantage here. That said I would love to review a PR :)

Here is my old code which may gives you some good hints:

https://github.com/netty/netty/pull/4726/

On 26 Jan 2017, at 15:30, Kirils Mensikovs <kmens...@gmail.com> wrote:

Apache Arrow (http://arrow.apache.org/) uses netty buffers. All contiguous memory buffers are aligned at 64-byte boundaries and padded to a multiple of 64 bytes. Padding to a multiple of 64 bytes allows using SIMD instructions consistently in loops without additional conditional check.

Is PoolArena.normalizeCapazity(int capacity) is right place to start with?

-Thanks

-- 
You received this message because you are subscribed to the Google Groups "Netty discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netty+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netty/0e6f759f-4b8d-493b-8397-f67faa933cfb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kirils Mensikovs]

Thanks. I already start to work on it. Hope to get positive results over the weekend.

On 26 January 2017 at 16:39, 'Norman Maurer' via Netty discussions <ne...@googlegroups.com> wrote:

I see…

yes its the right place to start (and also a few others). At some point I wanted to do something similar but could not prove any advantage here. That said I would love to review a PR :)

Here is my old code which may gives you some good hints:

https://github.com/netty/netty/pull/4726/

On 26 Jan 2017, at 15:30, Kirils Mensikovs <kmens...@gmail.com> wrote:

Apache Arrow (http://arrow.apache.org/) uses netty buffers. All contiguous memory buffers are aligned at 64-byte boundaries and padded to a multiple of 64 bytes. Padding to a multiple of 64 bytes allows using SIMD instructions consistently in loops without additional conditional check.

Is PoolArena.normalizeCapazity(int capacity) is right place to start with?

-Thanks

On Wednesday, January 25, 2017 at 8:19:26 PM UTC+2, Norman Maurer wrote:

Why you need this ? 

There is currently not way to influence the alignment manual

Am Dienstag, 24. Januar 2017 17:19:53 UTC+1 schrieb Kiril Menshikov:

Hey,

How can use PooledByteBufAllocator to allocate 64-byte align memory.

Is it possible to know during memory request what memory address will be 64-byte aligned:

allocator.directBuffer(512, Integer.MAX_VALUE);

Do you have recommendations how to do it?

At the moment I can verify memorAddress and compute padding if necessary. But in this case I need to modify size as well (size+padding).

Thanks,

-Kiril

-- 
You received this message because you are subscribed to the Google Groups "Netty discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to netty+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/netty/0e6f759f-4b8d-493b-8397-f67faa933cfb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Netty discussions" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/netty/EwULtAWMYfI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to netty+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netty/527EF78F-D3DC-4A87-92BE-57D8DB797B69%40googlemail.com.

For more options, visit https://groups.google.com/d/optout.

--

Thanks,

-Kiril
Phone +37126409291
Riga, Latvia
Skype perimetr122

[Norman Maurer]

Cool ... looking forward to it.

To unsubscribe from this group and stop receiving emails from it, send an email to netty+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netty/CABgfpaoKa1zZ6h3w9nE8NubgfyutH7A5iHzJ1X9_XBFEiVJ0jQ%40mail.gmail.com.

[Kiril Menshikov]

Here is my PR https://github.com/netty/netty/pull/6293

Actually it’s mostly your code from https://github.com/netty/netty/pull/4726/ with small modifications. Can you review it. 

I got good results for reads. Sometimes the result is not so obvious.

Benchmark                                       (cacheAlign)  (size)  Mode  Cnt      Score      Error  Units

PooledByteBufAllocatorAlignBenchmark.write                 0   00000  avgt   25    152.254 ±    7.125  us/op

PooledByteBufAllocatorAlignBenchmark.write                 0   00256  avgt   25    579.521 ±   87.000  us/op

PooledByteBufAllocatorAlignBenchmark.write                 0   01024  avgt   25   1110.064 ±  259.058  us/op

PooledByteBufAllocatorAlignBenchmark.write                 0   04096  avgt   25   3752.688 ± 1747.052  us/op

PooledByteBufAllocatorAlignBenchmark.write                 0   16384  avgt   25   7370.301 ± 1971.284  us/op

PooledByteBufAllocatorAlignBenchmark.write                 0   65536  avgt   25  24098.062 ± 1884.054  us/op

PooledByteBufAllocatorAlignBenchmark.write                64   00000  avgt   25    148.043 ±    4.063  us/op

PooledByteBufAllocatorAlignBenchmark.write                64   00256  avgt   25    855.125 ±  205.381  us/op

PooledByteBufAllocatorAlignBenchmark.write                64   01024  avgt   25    953.643 ±  202.361  us/op

PooledByteBufAllocatorAlignBenchmark.write                64   04096  avgt   25   3679.935 ± 1760.722  us/op

PooledByteBufAllocatorAlignBenchmark.write                64   16384  avgt   25   7452.614 ± 2142.256  us/op

PooledByteBufAllocatorAlignBenchmark.write                64   65536  avgt   25  24279.741 ± 1690.916  us/op

PooledByteBufAllocatorAlignBenchmark.writeRead             0   00000  avgt   25    181.093 ±    5.376  us/op

PooledByteBufAllocatorAlignBenchmark.writeRead             0   00256  avgt   25   3530.221 ±  763.284  us/op

PooledByteBufAllocatorAlignBenchmark.writeRead             0   01024  avgt   25   4700.575 ± 1399.451  us/op

PooledByteBufAllocatorAlignBenchmark.writeRead             0   04096  avgt   25   6415.920 ± 2009.234  us/op

PooledByteBufAllocatorAlignBenchmark.writeRead             0   16384  avgt   25  11260.743 ± 2043.894  us/op

PooledByteBufAllocatorAlignBenchmark.writeRead             0   65536  avgt   25  33286.627 ± 3410.495  us/op

PooledByteBufAllocatorAlignBenchmark.writeRead            64   00000  avgt   25    187.045 ±    9.996  us/op

PooledByteBufAllocatorAlignBenchmark.writeRead            64   00256  avgt   25   3111.288 ±  428.551  us/op

PooledByteBufAllocatorAlignBenchmark.writeRead            64   01024  avgt   25   4726.313 ± 1204.686  us/op

PooledByteBufAllocatorAlignBenchmark.writeRead            64   04096  avgt   25   5646.698 ± 1012.979  us/op

PooledByteBufAllocatorAlignBenchmark.writeRead            64   16384  avgt   25  10203.579 ± 1486.852  us/op

PooledByteBufAllocatorAlignBenchmark.writeRead            64   65536  avgt   25  31321.519 ± 2397.658  us/op

To unsubscribe from this group and all its topics, send an email to netty+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netty/C7CDD36D-1CA8-4E0F-9AE2-2D9AD53D8EFD%40googlemail.com.