How to calculate crc32 over a single byte data using clmul

[g terry]

Hi there,

I am trying to accelerate the crc calculation in my application code with clmul instruction. I know the basic idea of CRC and am able to get correct CRC value for data with length larger than 4 bytes using clmul instructions. However when the size of remaing data becomes less than 4 bytes, let's say just one byte, the clmul-based solution doesn't work as expected, it will produce CRC for {0x00, 0x00, 0x00, 0x1B}, not just for {0x1B}. Indeed those clmul instructions are designed to operate on 32bit registers. So is it possible to use them to get CRC32 for a single byte? Any advise are welcomed.

BTW: using lookup-table based solution, I do get CRC32 value for single byte data.

BR,

Terry

[Kristof Kiekens]

Hi Terry,

I think you indeed need padding.  I never heard about using lookup tables.

Did you try both?

0x0000001B

0x1B000000

Regards,

Kristof

Op dinsdag 16 april 2024 om 03:08:50 UTC+2 schreef te...@unifra.io:

[Tommy Murphy]

Anything useful here? 

https://github.com/WoWaster/riscv-crc32-clmul 

[g terry]

Hi Kristof,

I did tried both of them. The result I got is CRC value for four bytes "0x0000001B" or "0x1B000000", not just for the single byte "0x1B". There are some online crc calculator such as this one:   Online CRC-8 CRC-16 CRC-32 Calculator (crccalc.com), the single byte "0x1B" does has its own crc32 value which is different from "0x0000001B". I guess when use clmul to calculate crc32 for a single byte data, more actions/steps may need be taken to counteract the fact that clmul operates on 32bit register naturally.

BR,

Terry

[Kristof Kiekens]

Have a look at this: 

http://www.ross.net/crc/download/crc_v3.txt

From what I understood you can reverse cycle it. As you mentioned indeed counteract the side effect of padding.

https://stackoverflow.com/questions/54657385/is-it-possible-to-remove-padding-at-the-end-from-crc-checksum

Regards,

Kristof

Op dinsdag 16 april 2024 om 16:23:41 UTC+2 schreef te...@unifra.io:

[Tommy Murphy]

> it will produce CRC for {0x00, 0x00, 0x00, 0x1B}, not just for {0x1B}. 

Isn't that correct given that CRC-32 produces a 32-bit hash/checksum for the input data? E.g.

https://fuchsia.googlesource.com/third_party/wuffs/+/HEAD/std/crc32/README.md

"CRC-32 (Cyclic Redundancy Check 32) is a checksum algorithm that hashes byte sequences to 32 bit values."

[g terry]

Hi Tommy，

Thanks for your help. What you said is totally correct except that mine is a special or picky case where the "byte sequences" only include a single byte. The only question here is that the clmul instruction naturally operates on 4-bytes data, I am  trying to figure out how to use it to calculate a CRC32 for a single byte. In the WoWaster project, when the "byte sequence" contains less than 4-byte data, a lookup-table will be used rather than the clmul, this is correct as well, but in my case, I can't switch to use lookup-table and have to use clmul.

BR,

Terry

[g terry]

Hi Kristof,

Thanks very much. I will study those links.

BR,

Terry

[g terry]

Hi Kristof,

Many many thanks. The link you provide does work for me. I can get the expected CRC32 value for one single byte now.

BR,

Terry