Changing memory allocation functions after initialization/usage

[Bart Siwek]

I was wondering if Jansson can handle the change of memory allocator
after it has been initialized/used?

The scenario I'm looking at is that I have a couple of memory
allocators that I use in different places in code. So if some part of
code sets the memory allocator functions to A_malloc and A_free,
parses some JSON, then some other part of code will set memory
allocator functions to B_malloc and B_free what risk am I running
into? Is it possible that some memory allocated from A_malloc will be
passed to B_free for deallocation?

Best Regards,
Bart

[rogerz]

On Fri, Jun 1, 2012 at 11:41 PM, Bart Siwek <bartlomi...@gmail.com> wrote:

I was wondering if Jansson can handle the change of memory allocator
after it has been initialized/used?

From what I see in the code memory.c, you can call json_set_alloc_funcs() anytime you want to change the allocator. 

The scenario I'm looking at is that I have a couple of memory
allocators that I use in different places in code. So if some part of
code sets the memory allocator functions to A_malloc and A_free,
parses some JSON, then some other part of code will set memory
allocator functions to B_malloc and B_free what risk am I running
into? Is it possible that some memory allocated from A_malloc will be
passed to B_free for deallocation?

There's no guarantee for pairing the malloc and free in jansson. Actually it can only remember one pair. I'm afraid you will have to manage it by yourself carefully.

[Petri Lehtinen]

rogerz wrote:
> On Fri, Jun 1, 2012 at 11:41 PM, Bart Siwek <bartlomi...@gmail.com> wrote:
>
>> I was wondering if Jansson can handle the change of memory allocator
>> after it has been initialized/used?
>
> From what I see in the code memory.c, you can call json_set_alloc_funcs()
> anytime you want to change the allocator.

You can call it more than once but you shouldn't:

http://www.digip.org/jansson/doc/2.3/apiref.html#json_set_alloc_funcs

"This function has to be called before any other Jansson’s API
functions to ensure that all memory operations use the same
functions."

>> The scenario I'm looking at is that I have a couple of memory
>> allocators that I use in different places in code. So if some part of
>> code sets the memory allocator functions to A_malloc and A_free,
>> parses some JSON, then some other part of code will set memory
>> allocator functions to B_malloc and B_free what risk am I running
>> into? Is it possible that some memory allocated from A_malloc will be
>> passed to B_free for deallocation?
>
> There's no guarantee for pairing the malloc and free in jansson. Actually it
> can only remember one pair. I'm afraid you will have to manage it by yourself
> carefully.

Exactly. If you need to manage the memory with two different
allocators, I'm afraid you'll have to wrap them somehow as a single
alloc/free pair and do the bookkeeping yourself. This would be quite
complicated, though.

Petri

[Paul Harris]

The alternative is probably to further extend jansson to add an "environment" handle to every jansson call (at least, any call that does an alloc, eg jansson_integer(env, 42)

And in that environment handle, store the allocator/deallocator that you want jansson to use.

That also means the addref and decref functions will also need to be passed the environment/allocators.

Quite a lot of bloat for something only a few people may ever use.

As for doing things through a single alloc/free pair, it wouldn't be hard IF you were to run your A_ and B_ code in their own threads (and ONLY their own threads).

Then you can use thread-specific variables to tell the alloc/free functions how to do their jobs.

Add checks to ensure that you don't try to access B_'s stuff from anywhere outside of its own thread.  Or, if you do, ensure that the code won't result in an alloc/dealloc.

[Bart Siwek]

First of all - thank you everyone for the answers.

Secondly - I was actually considering augmenting the existing code
with the environment handles, but as Paul said that is a lot of work
and bloat. Also I like that idea of using different threads.

There might be another "good enough approach" that would for a single
thread in my situation - freeing all the memory that jansson allocated
during processing of one input, changing the allocator and then
processing another input. The only question that I have - how can I
make sure that everything was freed? Also - how can I check if there
are any references dangling?

My aim right now is a function like "int
jansson_free_all_the_things(void)" that:
- returns 0 and does nothing if there exist dangling references
- returns 1 and frees all the memory allocated if there are no
dangling references
Any tips how to start with this?

On Jun 3, 8:05 pm, Paul Harris <harris...@gmail.com> wrote:
> On Sunday, June 3, 2012 2:09:04 AM UTC+8, Petri Lehtinen wrote:
>
> > rogerz wrote:

> > > On Fri, Jun 1, 2012 at 11:41 PM, Bart Siwek <bartlomiej.si...@gmail.com>

[rogerz]

On Mon, Jun 4, 2012 at 6:49 AM, Bart Siwek <bartlomi...@gmail.com> wrote:

First of all - thank you everyone for the answers.

Secondly - I was actually considering augmenting the existing code
with the environment handles, but as Paul said that is a lot of work
and bloat. Also I like that idea of using different threads.

There might be another "good enough approach" that would for a single
thread in my situation - freeing all the memory that jansson allocated
during processing of one input, changing the allocator and then
processing another input. The only question that I have - how can I
make sure that everything was freed? Also - how can I check if there
are any references dangling?

I'm afraid there is no quick way to check dangling references. You have to manage all the reference you've created carefully and release them one by one. However, if you've already attached all the nodes to same root and no redundant reference created, it is as simple as json_decref(your_root).

 

My aim right now is a function like "int
jansson_free_all_the_things(void)" that:
- returns 0 and does nothing if there exist dangling references
- returns 1 and frees all the memory allocated if there are no
dangling references
Any tips how to start with this?

I think this is not easy, it is something like an auto garbage collection implementation in C. You needed to create your own malloc/free for this.

[Paul Harris]

You are creating your own malloc/free pairs, I would just add another custom function of your own:

* has_everything_been_freed()

In debug mode (or production if you like), keep a list of the memory that has been alloc'd and freed.

When you switch, call that function.  If there is anything that you haven't freed, then it indicates there is a bug.  Add extra logging during the malloc/free and you can figure out which call alloc'ed the memory that wasn't freed.

That said, jansson's tests use valgrind to check for memory leaks, so in theory you should not find any problems with jansson.  If you have leftover dangling memory allocs, then you may not be inc/dec references correctly.

If you follow this path, please share the code as it might be a handy thing for jansson users to "switch on" to check that their own code doesn't leave any memory unfreed.

cheers

Paul