What steps will reproduce the problem? 1. compiling using cl.exe version 9 2. using google perftools 3. call the crt function call _recalloc http://msdn.microsoft.com/en-us/library/ms235322.aspx 4. used the following code: int main(int argc, char *argv[]) { int *p = NULL, len = 0, i; p = _recalloc(p, sizeof(p) * (len + 2), 1); p[len] = len + 1; len++; p = _recalloc(p, sizeof(p) * (len + 2), 1); p[len] = len + 1; len++; p = _recalloc(p, sizeof(p) * (len + 2), 1); p[len] = len + 1; len++; for (i = 0; p && i < 5; i++); if (i > 3) { printf("error p[%d] isn't null and i=%d\n", len, i); return; } printf("success p[%d] is null i=%d\n", len, i);
}
OUTPUT: error p[3] isn't null and i=5
What is the expected output? What do you see instead? success p[3] is null i=3
What version of the product are you using? On what operating system? Windows XP SP2, version 1.4 of google-perftools
Please provide any additional information below. The problem is that when calling the crt function _recalloc, the function _msize on consecutive calls to _recalloc returns old_size differently from the size previously given by the user and so _recalloc doesn't use memset to set the new area of the block to zero (the old_size and the size_orig are the same at the last itiration into _recalloc).
/* ensure that (size * count) does not overflow */ if (count > 0) { _VALIDATE_RETURN((_HEAP_MAXREQ / count) >= size, ENOMEM, NULL); } size_orig = size * count; if (memblock != NULL) old_size = _msize(memblock); retp = _realloc_base(memblock, size_orig); if (retp != NULL && old_size < size_orig) { memset ((char*)retp + old_size, 0, size_orig - old_size); } return retp;
}
-- You received this message because you are listed in the owner or CC fields of this issue, or because you starred this issue. You may adjust your issue notification preferences at: http://code.google.com/hosting/settings
Blah, all the documentation I can find is imprecise about the semantics of _msize(). MSDN, which I guess is authoritative in these matters, says it's "the size" of memory allocated on the heap. But I don't know if that's the number of bytes requested by the malloc, or the number of bytes actually allocated by the memory system. We implement the latter, but it looks from your code snippet that windows -- or at least recalloc -- expects the former.
How are you using _recalloc? Are you calling it directly, or is it getting called by some internal windows routine?
The easiest way to fix this is to change windows calloc so it always zeros out the full block tcmalloc allocated, even if you requested less than the full block size. Then _recalloc_base won't do anything with the extra memory, but it's ok because it got zeroed out earlier. Of course, that's more expensive than it needs to be, but if this code is rarely used -- never used internally by windows for instance -- maybe that's ok.
A more difficult fix would be to store the size requested for every allocation call, and change _msize to always use that value. It's more difficult because we don't store that data currently, and there's no obvious place to put it. It would also increase the overhead of the allocator.
Hmmm, I'm not sure of the best thing to do.
Let's at least make sure the easy solution works. Look at src/tcmalloc.cc:do_calloc, and replace the line: memset(result, 0, size); with memset(result, 0, GetSizeWithCallback(result, &InvalidGetAllocatedSize));
Does this fix the problem for you?
Also let me know if you have any thoughts on the best way to resolve this issue.
-- You received this message because you are listed in the owner or CC fields of this issue, or because you starred this issue. You may adjust your issue notification preferences at: http://code.google.com/hosting/settings
I think there will still be a bug with your simple solution. Example: p = recalloc(p, 256, 1); memset(p, 0xff, 256); p = recalloc(p, 128, 1); p = recalloc(p, 256, 1); /* p[128] till p[255] should all be zero, but they aren't. Instead they are 0xff */
-- You received this message because you are listed in the owner or CC fields of this issue, or because you starred this issue. You may adjust your issue notification preferences at: http://code.google.com/hosting/settings
Yes, this is a good point. A pity. :-( I don't know of any good ways to solve this then. I don't suppose it's possible to get rid of _recalloc. :-} I'll think on it.
-- You received this message because you are listed in the owner or CC fields of this issue, or because you starred this issue. You may adjust your issue notification preferences at: http://code.google.com/hosting/settings
I've actually gotten ride of recalloc in my code, mainly because of issues of implementation in the win32 crt. BSD once had a recalloc but they also got ride of it, probably because of usage problems they had with it. Apparently it's a function with many problems. The main one being your not aloud to use it together with any other form of alloc (realloc for example). Maybe you should just put a discalmer stating that it doesn't work with _recalloc in win32 crt.
-- You received this message because you are listed in the owner or CC fields of this issue, or because you starred this issue. You may adjust your issue notification preferences at: http://code.google.com/hosting/settings
I like that solution! :-) I'll make the changes to the documentation for the next release.
-- You received this message because you are listed in the owner or CC fields of this issue, or because you starred this issue. You may adjust your issue notification preferences at: http://code.google.com/hosting/settings
As you point out, recalloc() is fundamentally broken; while it can ensure zero initialization when doing: recalloc(0, size)
it *cannot* ensure proper zero-initialization in any other case.
For example: ptr = recalloc(0, 19);
In this case, ptr will be alloc'd to 19 bytes of 0. But underneath, the allocator probably used a 32byte chunk. The allocator knows that 32 bytes were allocated here, NOT 19 bytes.
Now, when you do: ptr = recalloc(ptr, 66);
The allocator *does not know* where to start putting zeroes. It didn't know that only 19 of the 32 bytes were used previously. This is true on ALL allocators.
This is why recalloc has two meanings: a) If the ptr is zero, it behaves like calloc b) Otherwise, it behaves like realloc
Recalloc is an abomination.
-- You received this message because you are listed in the owner or CC fields of this issue, or because you starred this issue. You may adjust your issue notification preferences at: http://code.google.com/hosting/settings
|
