Memory leak in parsing
New2JSON
Eric Haszlakiewicz
On Sep 10, 2012 5:53 AM, "New2JSON" <urpr...@gmail.com> wrote:
>
> Hi Friends
>
> I'm new to JSON and JSON-C. I'm seeing overall approx 4MB increase in memory usage. I mean its not constantly increasing, but after first use of JSON, memory usage is increasing by 4 MB. Is there there any solution to this ? I'm using the following code to release the memory, but of no use. Still the 4MB is lost.
>
> json_object* v = json_tokener_parse(json);
> m_pObject = json_object_object_get(m_pRoot, m_name);
Are v and m_pRoot the same thing? If you can provided an actual minimal test case that we can try compiling it could be helpful.
> json_object_object_del(m_pObject, elementName(ROWS));
> m_pRowArray = json_object_new_array();
> json_object_object_add(m_pObject, elementName(ROWS), m_pRowArray);
This looks ok, I don't see anything obviously wrong.
Are you doing a json_object_put on v (aka m_pRoot) at any point? If not, then your entire object tree is still allocated.
Eric
Eric Haszlakiewicz
On Sep 10, 2012 9:52 AM, "Eric Haszlakiewicz" <haw...@gmail.com> wrote:
> On Sep 10, 2012 5:53 AM, "New2JSON" <urpr...@gmail.com> wrote:
> > I'm using the following code to release the memory, but of no use. Still the 4MB is lost.
> >
> > json_object_object_del(m_pObject, elementName(ROWS));
> > m_pRowArray = json_object_new_array();
> > json_object_object_add(m_pObject, elementName(ROWS), m_pRowArray);
>
> This looks ok, I don't see anything obviously wrong.
Bleh, well there's nothing "obviously" wrong with your code, but json_object_object_del doesn't do quite what you think it would. It just removes the entry from the containing object, but it does *not* decrease the reference count on the object you're removing.
So to free the memory you actually need to do this:
json_object *toremove = json_object_object_get( m_pObject, elementName(ROWS));
json_object_object_del(m_pObject, elementName(ROWS));
Json_object_put(toremove);
Yes, this particular part of the api sucks. I'm going to add a new function that does the above steps (called json_object_object_delput, unless a better name presents itself). Of course, unless you're running a development version of json-c you'll need to do the put call explicitly.
Btw, what version are you using? 0.10?
Eric
New2JSON
ХХ
May be rename json_object_object_del to json_object_object_take, and add new function json_object_object_del - for "take and put" actions?
10.09.2012, 19:07, "Eric Haszlakiewicz" <haw...@gmail.com>:
>> On Sep 10, 2012 5:53 AM, "New2JSON" <urpr...@gmail.com> wrote:
>> > I'm using the following code to release the memory, but of no use. Still the 4MB is lost.
>> >
>> > json_object_object_del(m_pObject, elementName(ROWS));
>> > m_pRowArray = json_object_new_array();
>> > json_object_object_add(m_pObject, elementName(ROWS), m_pRowArray);
>>
>> This looks ok, I don't see anything obviously wrong.
>
>
> json_object *toremove = json_object_object_get( m_pObject, elementName(ROWS));
> json_object_object_del(m_pObject, elementName(ROWS));
> Json_object_put(toremove);
>
>
> Btw, what version are you using?О©╫ 0.10?
>
> Eric
Eric Haszlakiewicz
On Sep 10, 2012 11:53 PM, "New2JSON" <urpr...@gmail.com> wrote:
>
> Thank you Eric for your reply.
>
> I'm using the json-c v0.9.
You should consider upgrading, 0.9 is a very old version. It won't make any difference in the behavior of json_object_object_put, but there have been many other fixes.
> And in my wrapper class destructor , I'm using json_object_put(m_pRoot).
> m_pRoot and v are different objects.
Er... well, ok. The code snippet you posted didn't have any line that related v to m_pRoot, so I'm not quite clear why you included both. This is why I asked for a compilable test case. (Not relevant now, since I think we know where the problem is)
> Request for certain clarifications:
> 1. Lets say after calling json_object_put() the reference count becomes zero, and then when actually the memory gets released ? I mean as soon as the reference count becomes zero or after certain time delay.
It happens immediately. This is just plain C so there's no garbage collection.
> 2. For debug purpose can I know what is the current reference count for any particular json object ? Which will help me to find out the leak.
Yes, you can access that, but you'll need to use private structure members to do so, which means your code would need to include json_object_private.h, and use the same field that json_object_get() does:
https://github.com/json-c/json-c/blob/master/json_object.c#L126
This is discouraged, but if you need to do it I recommend putting this debug code in a separate source file so you only need to include the private header there, and the rest of your code is guaranteed to only use the public json-c api.
Eric
Eric Haszlakiewicz
On Sep 11, 2012 6:13 AM, "ХХ" <xla...@yandex.ru> wrote:
>
> Good day!
> May be rename json_object_object_del to json_object_object_take, and add new function json_object_object_del - for "take and put" actions?
>
No, sorry, I won't change the name of the existing function because that would break compatibility for people that are using existing code. For the same reason, I won't just change json_object_object_del to automatically do a put, because that behaviour change could easily lead to hard to discover bugs in existing code.
I might consider deprecating and eventually removing json_object_object_del, but that's not something to worry about until we're going to do a major release.
Eric
New2JSON
Eric Haszlakiewicz
On Sep 12, 2012 7:04 AM, "New2JSON" <urpr...@gmail.com> wrote:
>
> Thank you Eric again.
>
> You said you know, where the problem is !!!
> Are you referring to, some problem in reference counting or you got something else ?
> Please let me know the problem and the solution for that.
I'm referring to the fact the json_object_object_del does not call json_object_put, you need to do the put yourself. The del function just detaches the field from it's containing object without adjusting the reference count.
Obviously I can't be certain that that is the only memory leak in your program, but can you try adding the explicit put call and see if it helps? i.e.:
json_object *toremove = json_object_object_get( m_pObject, elementName(ROWS));
json_object_object_del(m_pObject, elementName(ROWS));
json_object_put(toremove);
Eric
Eric Haszlakiewicz
> On Sep 10, 2012 9:52 AM, "Eric Haszlakiewicz" <haw...@gmail.com> wrote:
>> On Sep 10, 2012 5:53 AM, "New2JSON" <urpr...@gmail.com> wrote:
>> > I'm using the following code to release the memory, but of no use. Still
>> > the 4MB is lost.
>> >
>> > json_object_object_del(m_pObject, elementName(ROWS));
>> > m_pRowArray = json_object_new_array();
>> > json_object_object_add(m_pObject, elementName(ROWS), m_pRowArray);
>>
>> This looks ok, I don't see anything obviously wrong.
>
> Bleh, well there's nothing "obviously" wrong with your code, but
> json_object_object_del doesn't do quite what you think it would. It just
> removes the entry from the containing object, but it does *not* decrease the
> reference count on the object you're removing.
look closely enough at what was happening in the code when I examined
it before.
I ran a test to make sure things were working properly, and realized
that json_object_object_del() *does* end up calling json_object_put()
(by way of lh_table_delete(), which calls json_object_lh_entry_free()
by way of a free_fn function pointer, which calls json_object_put()).
i.e. you don't need to, and should not explicitly call json_object_put
after calling json_object_object_del.
So, we're back to the beginning. You have some kind of test case that
leaks memory. Can you please provide a short working example? Then
maybe we'll be able to figure out what the actual problem is.
eric