libpmemlog supports log organization

[Gao Neal]

I'm new to the pmem library and trying to use libpmemlog for append only log implementations.
As I go through the API's, it seems to me that this library only supports writing variable sized logs but not reading variable sized logs from it. In other words,
for reading, it only supports fixed sized chunk read and full file read.

Enlighten me if not.
Thanks

[Krzysztof Czurylo]

Hi,

That's true, but keep in mind that there is actually no "read" operation. It's memory, so pmemlog_walk() literally "walks" through the log, invoking the 'process_chunk' callback for each fixed-size chunk of data or for the entire log if 'chunksize' is 0.

So, it's not like reading the entire log from a file into some in-memory buffer (which could be problematic), but processing the log in place.

If you need to process variable sized logs, you have to implement your own 'process_chunk' function that would iterate through the variable-sized chunks by advancing the 'buf' pointer.

K.

[Gao Neal]

To make use of my disk based logging module that takes care of log entry organization, I need the pmemlog read that works like fread but it can be in place read, instead,
My solution is adding a function pmemlog_read and a member called read_offset in the library.
Thanks for your answer.