> Hello i am trying to understand c better and i am trying to re-implement
> the functions in malloc.c(.h) and what i can't find is documentation on
> how to link a pointer to an address in memomory that doesn't result in a
> segfault
Well, guess what: you can't. Nor can anybody else while writing fully
portable code, because the language doesn't support it. So you can't
implement your own malloc(). Which is exactly the reason why malloc()
is a standard library function that comes as part of the compiler.
I wouldn't recommend that. It's surprisingly hard, even when you've
figured out the machine-dependent bits. People have written PhD theses
on the subject of efficient malloc() implementations.
> and what i can't find is documentation on how to link a pointer to an
> address in memomory that doesn't result in a segfault
The answer varies greatly from system to system, and is completely
off-topic for this newsgroup.
Just to give you an idea:
http://cvsweb.freebsd.org/src/lib/libc/stdlib/malloc.c?rev=1.183
DES
--
Dag-Erling Smørgrav - d...@des.no
>Hello i am trying to understand c better and i am trying to re-implement
>the functions in malloc.c(.h) and what i can't find is documentation on
>how to link a pointer to an address in memomory that doesn't result in a
>segfault
Neither malloc.c nor malloc.h are standard. malloc and related
functions are declared in stdlib.h.
How you would request memory from the operating system without using
the standard malloc is very system dependent. You need to ask in a
newsgroup that discusses your system.
--
Remove del for email
There is no standard C function to do that, other than malloc().
There is no standard C header file <malloc.h>.
There is no requirement that an invalid pointer generate a segfault
(or smegmentation fault). Some systems consider all possible
pointers valid, in that they don't generate any kind of fault when
you dereference them, no matter what bit pattern they contain.
malloc() will take a chunk out of either (a) memory already allocated,
say, in a big static pre-allocated array, freed memory, or from a
previous request to the OS, or (b) memory requested from the OS,
and return its address to the caller.
Some implementations of malloc() use OS-dependent functions like
sbrk() or mmap(). Others use OS-dependent functions with different
names that work differently. If the OS uses virtual memory, it is
responsible for setting up mapping between real and virtual memory.
> Hello i am trying to understand c better and i am trying to
> re-implement the functions in malloc.c(.h) and what i can't find is
> documentation on how to link a pointer to an address in memomory that
> doesn't result in a segfault
You can't, while you stay within what C gives you. If you read the
source for your C library, you will find that the implementation of
malloc uses operating system calls (maybe inside a few layers of
wrappers) and is thus OS-dependent. That's a new and scarier level of
programming: stay out of it until you are proficient with the language.
Try this for a more useful learning task: construct a wrapper around
malloc() and free() that allows you to list the memory blocks allocated
and freed by your program.
--
John Dallman, j...@cix.co.uk, HTML mail is treated as probable spam.
review the documentation for your chosen target platform.
(or declare a large uninitialised array and allocate memory in it)
Sorry for my stepping in...
When you said the implementation of malloc is OS-dependent,IIRC C has
been there in 70s of the 20th century while MS Windows just came to
the world on 90s,so how?
Regards,
Sam
In anticipation of someone with more experience than me coming along to
explain more thoroughly, every time a new OS comes along, someone has to
write a compiler to run on it, with a new version of malloc for that OS,
on a given architecture. When MS Windows came along, someone had to
write a compiler that ran on it, and had to write a new version of
malloc for that OS and whatever architecture it ran on.
'Chops
> When you said the implementation of malloc is OS-dependent,IIRC C has
> been there in 70s of the 20th century while MS Windows just came to
> the world on 90s, so how?
The source code of a C run-time library for MS Windows is not the same
as the source for a run-time library for a UNIX-like OS. The system
calls used to obtain memory from the operating system are different,
along with many other things.
--
John Dallman, j...@cix.co.uk, HTML mail is treated as probable spam.
--
I don't follow your reasoning. Don't you think operating systems
existed before Windows?
DES
--
Dag-Erling Smørgrav - d...@des.no
>Sorry for my stepping in...
>When you said the implementation of malloc is OS-dependent,IIRC C has
>been there in 70s of the 20th century while MS Windows just came to
>the world on 90s,so how?
The longevity of the language is irrelevant. It is the various
implementations that provide the flexibility. Surely you would not
expect the run time library from my IBM mainframe to execute under
Windows. Even if the instruction set were compatible, which it
decidedly is not, the method of requesting services from the OS is
completely different. The same applies to almost all the I/O
routines.
--
Remove del for email
I don't understand your question. Why do you think there's a problem?
Although if he just wants to see if he can implement malloc as a
learning experience, and isn't interested in actually using it, the
most straightforward approach would be to use malloc itself to
allocate a very large block of memory, then use his own function to
partition that into smaller blocks as they're requested.
Ensuring that the pieces are always suitably aligned cannot be done
portably, though there are ways of substantially reducing the likelihood
that the pieces are misaligned.
What I confused is that how can malloc call O/S API say,MS's
GlobalAlloc(),while such O/S does not exist when C exists,for this
question I think 'chops has figured it out;on the other hand,another
question is if it is true malloc call O/S APIs ,then how does O/S
request memory in the source code,malloc?(of course NO),or pure
assembler?
Regards,
Sam
C does not specify how malloc() is implemented, only what should happen
when you call it. When Microsoft developed Windows, they wrote a C
library that implemented malloc() in terms of whatever low-level
primitives are available in Windows. The same happens whenever anybody
else writes a new operating system. It is not uncommon for the malloc()
implementation to change between versions of the same operating system,
either. FreeBSD has gone through at least three; the current one uses
mmap() to request memory from the operating systems, while previous
implementations used brk() / sbrk().
> for this question I think 'chops has figured it out;
Who or what is "chops"?
> on the other hand,another question is if it is true malloc call O/S
> APIs ,then how does O/S request memory in the source code,malloc?(of
> course NO),or pure assembler?
How the operating system manages memory and apportions it to
applications varies widely from operating system to operating system,
but typically, it allocates large contiguous chunks to the application,
which does with it as it sees fit.
Also, keep in mind that C is only one of many programming languages.
You should read one or more of these books:
http://www.amazon.com/dp/0136006639
http://www.amazon.com/dp/0123744938
http://www.amazon.com/dp/0201702452
DES
--
Dag-Erling Smørgrav - d...@des.no
--
True. I hadn't thought of that. But again, if it's just an
experiment, would it need to be portable? He could google the
rules used by his specific machine and follow them.
Intel's rules are below, and on a 32 bit machine most of them
wouldn't even come into play as long as he doesn't use uint64_t.
Assuming he's on an Intel, he could just align the block returned at
a multiple of 4 for anything except a char (and a char block
anywhere), and any element the block's corresponding array would also
be aligned properly. Segmentation would be a pain, but I'm pretty
sure it'll act as a flat memory space as long as he doesn't
allocate too large a block at first. Actually, I'm not even sure
if malloc can return a block that requires multiple segments to
address, but best to play it safe.
Align 8-bit data at any address.
Align 16-bit data to be contained within an aligned four-byte word.
Align 32-bit data so that its base address is a multiple of four.
Align 64-bit data so that its base address is a multiple of eight.
Align 80-bit data so that its base address is a multiple of sixteen.
Align 128-bit data so that its base address is a multiple of sixteen.
(http://software.intel.com/en-us/articles/
align-and-organize-data-for-better-performance)
On most modern systems, it is safe to assume that the correct alignment
for any scalar type T is sizeof(T) rounded up to the nearest power of
two.
DES
--
Dag-Erling Smørgrav - d...@des.no
True; I was only pointing out that the "learning experience" must
necessarily involve learning to write non-portable code; not exactly the
best thing to be learning, particularly at an elementary level.
> Intel's rules are below, and on a 32 bit machine most of them
> wouldn't even come into play as long as he doesn't use uint64_t.
Any block returned by malloc() must be correctly aligned for any object.
The rules you cite imply that 16 byte alignment is the smallest one that
is suitable for any data type. Therefore, every pointer returned by
malloc must be 16-byte aligned.
> Assuming he's on an Intel, he could just align the block returned at
> a multiple of 4 for anything except a char (and a char block
> anywhere), and any element the block's corresponding array would also
> be aligned properly.
He can't provide different alignment based upon the target data type,
because malloc() doesn't take an argument informing it of the data type.
> ... Segmentation would be a pain, but I'm pretty
> sure it'll act as a flat memory space as long as he doesn't
> allocate too large a block at first. Actually, I'm not even sure
> if malloc can return a block that requires multiple segments to
> address, but best to play it safe.
>
> Align 8-bit data at any address.
> Align 16-bit data to be contained within an aligned four-byte word.
> Align 32-bit data so that its base address is a multiple of four.
> Align 64-bit data so that its base address is a multiple of eight.
> Align 80-bit data so that its base address is a multiple of sixteen.
> Align 128-bit data so that its base address is a multiple of sixteen.
That tells you that 16 is the least common multiple of all of the
possible alignment requirements, which is the main thing that you need
to know for malloc(). It is otherwise a pretty useless summary; most
systems nowadays have alignment requirements that are powers of two. The
important question, not answered on that web page, is which data types
have each of those different alignment requirements - in other words,
which data types are "32-bit data"?
It's probably safe to assume that that's *a* correct alignment,
but not that it's *the* correct alignment. For example, on my
system (x86 Linux, gcc), types long, long long, double, and long
double all have 4-byte alignment, though their sizes are 4, 8, 8,
and 12, respectively.
Well, I suppose it's safe to assume in the sense that being wrong
won't hurt anything.
--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
I don't like making such implementation-specific assumptions. For those
who are so willing, keep in mind that this will overestimate the
requirements on many machines. However, even for those who are willing
to make such assumptions, which type T should they use? Remember that
malloc() must return a pointer suitably aligned to store ANY type.
As I said earlier; there are ways to substantially reduce the
possibility of misalignment. A union of, intmax_t, long double and
void*, and void(*)(void), is pretty likely to be maximally aligned.
However, there's no portable way to eliminate that possibility.
That has always been the case -- the alignment requirement for type T
cannot, by definition, be greater than sizeof(T).
However, that's a rather pessimistic method. How would malloc() know what
type T the memory is going to be used for? Your method would require, for
example, that when I malloc() a 256K buffer, it use 256K alignment.
Portable, yes. Practical, no.
--
Kenneth Brody
I can easily imagine a system where intmax_t is implemented in
sofware and might have a less stringent alignment than some smaller
integer type. Likewise for long double.
It might be a good idea (and it can't hurt) to add int, long,
long long, float, and double to the union.
--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
A 256K buffer is unlikely to have a scalar type, which is what
Dag-Erling specified.
In fact it's very likely to be an array of unsigned char, which means
its required alignment is 1.
--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
True. The more different types you add to the union, the more likely it
is that it will be maximally aligned; but you cannot be portably certain
that it is so aligned. There is always the possibility that the only
type(s) which is(are) maximally aligned are extended integer types that
are not used in any of the standard typedefs or type macros.
For that to be the case, there would have to be some extended integer
type that's more strictly aligned than either long long or intmax_t.
It's certainly possible, but I wouldn't seriously expect to to happen
on anything other than the DS9K.
--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Which is precisely my point...
DES
--
Dag-Erling Smørgrav - d...@des.no
In C sizeof(T) will give a valid alignment for T
(it must else array size computations will be wrong)
rounding up to a power of 2 could be problematic on systems where the
bus width is not a power of two chars-widths.
on other systems it probably safe to use the largest power of two that
is a factor of (T)
but if you need to know define a packed struct with a char and a T and
compare the size difference, that will tell you the hardware alignment
requrement, a non-packed struct will get you the optimum alignment for
speed.
That's a good point.
On most modern systems, the sizes of most scalar types are already
powers of two (long double is the only common exception). Rounding up
to a power of two either does nothing or runs some small risk of
giving you an incorrect alignment.
> on other systems it probably safe to use the largest power of two that
> is a factor of (T)
Right. If sizeof (long double) is 12, this gives you 4.
> but if you need to know define a packed struct with a char and a T and
> compare the size difference, that will tell you the hardware alignment
> requrement, a non-packed struct will get you the optimum alignment for
> speed.
Standard C doesn't support "packed structs".
--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"