Attributes of a memory request or address space request
James Harris
might be needed in a memory request. (Really it's an address space
request as the actual memory can be allocated by a page fault routine
as needed.)
In the general case are all of these needed? Are any other attributes
needed or advisable so that an address space allocator can be most
effective?
- Basic type - 2 bits
00 => read-write data
01 => read-only data
10 => stack
11 => executable code
- Where to allocate - 2 bits
00 => unimportant
10 => low addresses
11 => high addresses
- Guard pages should exist - 2 bits
below the region - no or yes
above the region - no or yes
- Privileged memory? - 1 bit
0 => user memory
1 => system memory
- Must be physically adjacent? 1 bit
0 => doesn't matter
1 => yes (for DMA or similar)
- Fixed position? - 1 bit
0 => no - may be moved after allocation
1 => yes - do not move once allocated
- Pinned? - 1 bit
0 => no - may be paged out
1 => yes - memory resident
- Specific location important? - 2 bits
00 => no
01 => preferred
10 => allocate near
11 => required to be exact
- Can be supplied not zeroed? - 1 bit
Only privileged code can request this
- Requested address or zero
- Requested size of the space
- Anticipated max size or zero
- Hard max size or zero
- Number of bytes at the bottom to preallocate
- Number of bytes at the top to preallocate
- Creation code to execute on allocation
- Destruction code to execute on deallocation
James
Alexei A. Frounze
wrote:
What are the 4 above?
> - Creation code to execute on allocation
> - Destruction code to execute on deallocation
+ Virtual range, physical range, executable yet unreadable code.
+ God forbid, NUMA node. :)
Alex
Marven Lee
James Harris wrote...
> Below is an attempt to come up with a maximal set of attributes that
> might be needed in a memory request. (Really it's an address space
> request as the actual memory can be allocated by a page fault routine
> as needed.)
>
> In the general case are all of these needed? Are any other attributes
> needed or advisable so that an address space allocator can be most
> effective?
You could check what attributes are used by posix mmap().
To add to those you mentioned, you could have an alignment field for
the virtual address. I found it useful to be able to mmap() on a 64k
alignment for the slab allocator I use so as to be able to find the header/
beginning of a slab.
--
Marv
James Harris
Good question. The first two are to allow the allocator to make better
initial placement or relocation decisions as follows.
Anticipated max size
- a hint how large the space is expected to grow or zero if
unspecified
- can be adjusted either up or down
Hard max size
- absolute size limit or zero if unspecified
- if not zero can be adjusted but only downwards
The preallocate attributes are intended to allow the app to suggest
ranges to preallocate (rather than waiting for the page fault handler
to allocate them which would be slower overall). Ignored where paging
not supported.
>
> > - Creation code to execute on allocation
> > - Destruction code to execute on deallocation
>
> + Virtual range, physical range, executable yet unreadable code.
Virtual range and physical range - you mean fixed addresses for one or
both?
Unreadability code cannot be enforced but it could be requested. Maybe
the basic type field should be three bits instead of two.
> + God forbid, NUMA node. :)
In what sense? You mean processor affinity or something like that? Or
is it memory speeds? (A single CPU can have more than one memory
speed. For example, some chipsets allow partially paired DIMMs.)
Perhaps an app should have the opportunity to indicate memory speed
not in absolute terms but relative to its other requests. I expect the
affinity would be governed by the requesting process.
James
BGB / cr88192
"James Harris" <james.h...@googlemail.com> wrote in message
news:eba3ccd6-1a20-4d95...@m38g2000yqd.googlegroups.com...
one possibility could be to use the PROT flag scheme from POSIX, but borrow
most of the rest from VirtualAlloc...
this is similar to what I did (for my interpreter), although it also accepts
the VirtualAlloc protection flags scheme (PAGE_*, but this depends on other
flags).
so, for example:
PROT_NOACCESS 0
PROT_READ 1
PROT_WRITE 2
PROT_EXEC 4
PROT_WRITECOPY 8
PROT_GUARD
...
MEM_RESERVE
MEM_COMMIT
MEM_TOP_DOWN
...
many of the MAP flags were placed between the PROT flags and the MEM flags
(I didn't move MS's flags), several flags were aliased (in a few cases when
both had essentially the same flag), and a few were overloaded.
there is a slight oddity is that the semantics don't mesh exactly between
POSIX flag usage and VirtualAlloc flag usage, but some are common (just be
careful with RESERVE and COMMIT, as these indicate VA behavior, vs
PRIVATE/SHARED/... which use POSIX behavior...).
'PHYSICAL' makes memory which is mapped linearly in both the process and the
host space.
...
or such...
> James
Alexei A. Frounze
wrote:
OK.
> The preallocate attributes are intended to allow the app to suggest
> ranges to preallocate (rather than waiting for the page fault handler
> to allocate them which would be slower overall). Ignored where paging
> not supported.
Um, OK.
> > > - Creation code to execute on allocation
> > > - Destruction code to execute on deallocation
>
> > + Virtual range, physical range, executable yet unreadable code.
>
> Virtual range and physical range - you mean fixed addresses for one or
> both?
Both. Physical is useful for the kernel, virtual may be useful
everywhere.
> Unreadability code cannot be enforced but it could be requested. Maybe
> the basic type field should be three bits instead of two.
Yeah, you're right. It can only be enforced at the segment level but
as soon as the code segment overlaps with any data segment, it can be
read through non-CS. There's no read protection at the page level.
> > + God forbid, NUMA node. :)
>
> In what sense? You mean processor affinity or something like that? Or
> is it memory speeds? (A single CPU can have more than one memory
> speed. For example, some chipsets allow partially paired DIMMs.)
Yeah.
> Perhaps an app should have the opportunity to indicate memory speed
> not in absolute terms but relative to its other requests. I expect the
> affinity would be governed by the requesting process.
Maybe. Can be hard to take into account everything, though.
Alex
James Harris
> "James Harris" <james.harri...@googlemail.com> wrote in message
It's an understatement to say that it is a challenge to accomodate all
requirements. I've included suggestions you guys have made: mmap
attributes, alignment, memory speeds, physical addressing, etc.
I've changed from a type code to use attribute bits - execute, read
and write - and added hints for where to allocate addresses and
physical memory.
Apart from the attribute bits and the size required all the other
values can default to zero. This is to allow a simple requests despite
the optional complexities. The intention is that the requestor be able
to zero the structure then just update the fields required.
How does the following idea sound?
Since the request block is of significant size I've assumed it will be
persistent. Included are fields the allocator can update and return
and a request block version number. The idea is that the requestor
specifies the requirements in the block then uses that block as
needed. The same request block would be used for allocate, resize and
free requests. The allocator would update fields in the block before
returning (but also keep its own data as needed).
Anyway here's the updated address space request block in the form of a
C struct. You'll see that 32-bit and 64-bit options are combined. The
first eight bytes of the block are of fixed form (a 32-bit header and
32 bits of flags). Subsequent fields are of machine word size whether
that is 32 or 64 bits.
typedef struct Asrb {
/* An address space request block */
uint8_t rb_size = 60; /* Size of the request block: 60 or 112 */
uint8_t reserved1 = 0;
uint8_t rb_log_word_size = 5; /* 5 (for 32-bit) or 6 (for 64-bit) */
uint8_t rb_ver = 1; /* 0x81 while developing */
uint32_t flags;
/* 1 bit: writeable required */
/* 1 bit: readable required */
/* 1 bit: executable required */
/* 3 bits reserved */
/* 2 bits sharing
* 00 => not shared
* 01 => may be shared
* 10 => all shared
* 11 => copy on write
*/
/* 4 bits: where to allocate addresses:
* anywhere, lowest, low, bottom half, just below middle
* just above middle, upper half, high, highest
* fixed location preferred, suggested, required
*/
/* 2 bits: where to allocate physical
* 00 => anywhere
* 01 => preferred
* 10 => near
* 11 => exact
*/
/* 2 bits reserved */
/* 1 bit: guard page below */
/* 1 bit: guard page above */
/* 1 bit: automatic extension below */
/* 1 bit: automatic extension above */
/* 1 bit: physical adjacency */
/* 1 bit: do not move once allocated */
/* 1 bit: do not swap out any pages once allocated */
/* 1 bit reserved */
/* 2 bits: memory speed hint
* 00 => default (fast) or faster
* 01 => very fast
* 10 => slow or faster
* 11 => slowest or faster
*/
/* 1 bit: can allocate non-local memory */
/* 1 bit: can allocate own dirty memory */
/* 1 bit: can allocate any dirty memory (only the system can
request) */
/* 1 bit: system protection (only the system can request this) */
uint32/64_t address; /* Address requested or zero */
uint32/64_t physical_address; /* Physical address requested or zero
*/
uint32/64_t size; /* Size requested or returned */
uint32/64_t max; /* maximum size or zero (can be changed) */
uint32/64_t abs_max; /* absolute maximum size (can be decreased) or
zero */
uint32/64_t preallocate_low; /* Bytes at the bottom to preallocate
*/
uint32/64_t preallocate_high; /* Bytes at the top to preallocate */
uint32/64_t granularity; /* Must be a power of 2 */
uint32/64_t exec_creation; /* Code to execute on creation */
uint32/64_t exec_destruction; /* Routine to execute on destruction
*/
uint32/64_t garbage; /* A word used in garbage collection */
uint32/64_t handle; /* Index or handle of the block (for allocator
use) */
uint32/64_t reserved;
} Asrb;
Given that this is intended to be a maximal description is anything
missing?
Any other comments?
James
James Harris
wrote:
...
Some suggested changes to the above - for my own records even if no
one else is interested.
1. Remove both preallocate entries. If preallocation (i.e. allocate
physical pages without waiting for the pager to allocate on faults) is
really needed it can be specified in a subsequent call. Although this
requires an extra syscall it is more flexible and saves space in the
above request block.
2. Remove the exec_creation entry. I don't think it is needed.
3. Redefine the exec_destruction entry to be called prior to
reclamation of the space. The routine invoked should maybe have an
option to request reclamation not go ahead.
4. Add an entry for the expected number of such requests. Zero means
unspecified. One means this is the only space of its size and
attributes expected to be requested - i.e. it is not part of an array.
More than one means the total number of such requests which are
expected. This is a hint, not something to be enforced.
5. Change the physical where-to-allocate bits' meanings to
00 - anywhere
01 - the exact location
10 - below the specified location
11 - above or at the specified location
Total size of request block including reserved word, garbage and
handle entries.
Eleven 64-bit words (88 bytes), or
Twelve 32-bit words (48 bytes)
As before, all zeroed fields take sensible defaults. I think the only
required field is the size requested.
It's not there yet but I think these are improvements.
James
James Harris
...
I've updated the list of potential attributes (mainly to allow further
expansion) and put it on the wiki at
http://aodfaq.wikispaces.com/asrb
I know it's not perfect and will need revision but it's in a form that
anyone can correct. If you see something wrong with either the
attributes or the comments fault feel free to either change it or
point out the fault here.
James
Rod Pemberton
> On 14 Jan, 20:23, James Harris <james.harri...@googlemail.com> wrote:
>
> ...
>
> I've updated the list of potential attributes (mainly to allow further
> expansion) and put it on the wiki at
>
> http://aodfaq.wikispaces.com/asrb
>
"There are at least three distinct places in an operating system where
requests for address space may be made."
1. 2. 3. ..., um, ok, you're not including stack, heap, arrays, etc. in
you're definition of "... address space ..." FYI, my first thoughts were
those. You're talking about allocations via an API.
I'm wondering about you using uint64_t. I'd make a strong guess that
"uint64_t" specifically is probably not present on a quite a few systems. I
also know of a few 32-bit compilers where an "unsigned long long" isn't
available either and so uint64_t can't be defined. Although it complicates
things, if needed, you might change to an array of a smaller type. E.g.,
uint64_t flags;
Becomes:
uint8_t flags[4];
Or:
uint32_t flags[2];
This is assuming that the structure sizing is fixed and not determined by
your "first 32 bits ...[as] identification value", or rb_ver (version?), or
rb_log_word_size variables. I.e., is the structure from uint64_t flags on
down different for 32-bits and 64-bits? E.g., 32-bit data types on 32-bit
architecture, but 64-bit data types on 64-bit architecture? If so, it might
be better to end the struct with a single placeholder variable (e.g., the
C90 "struct hack") and use it's location to overlay an architecture
appropriate structure, or end the structure with a "pointer" (as an array of
say 8 chars, e.g., uint8_t pointer[8]) whose "address" you fill in later
(via casts) to an architecture appropriate typedef'd structure (no space
allocated for typedef's), i.e., with an appropriate 32-bit or 64-bit layout.
A personal preference is to use multiples of 8 bits (a byte), or 4 bits (a
nybble), or bitfields via an binary-and and #define's. I'm not fond of the
3-bit data since 3-bits doesn't fit well with hexadecimal. On x86, 8-bit
sizes are directly accessable. For 16-bit and 32-bit modes, 4-bit nybbles
are also accessable via the "AAM 16" instruction (64-bit obsolete). Also,
bitfields larger than a single bit in size, will likely require shifts to
use their value. I'm also not fond of enums or bitfields in structures,
since old C compiler's didn't always implement these correctly. Also, I'd
think you'd want to design the structure to work well or easily with both C
and assembly. That may mean extra or less efficient use of space.
RP
s_dub...@yahoo.com
wrote:
>
> It's not there yet but I think these are improvements.
>
> James- Hide quoted text -
>
> - Show quoted text -
RE:
Asrb;
Given that this is intended to be a maximal description is anything
missing?
Any other comments?
James
( google won't let me properly address your previous msg., to old?)
-comment/question-
Data drives the Methods, but the Methods are framed in context of a
Paradigm.
What is the paradigm in mind?
Is it the usercode making the allocation request? Should the usercode
be allowed to dynamically allocate memory for itself from the system
pool? (if the linker provides static size of program .BSS, block
storage segment the usercode neednot/should not go outside this for
dynamic use? -if it needs more than .BSS, something is wrong and
suspect.)
Is it systemcode, a Loader, who is the requester? If so, is the
attribute list relevant in its list of attributes? -say you have a
handler and another instance of it is required dynamically, in that
case it is systemcode Scheduler thru the Loader, making the request,
and things like guard pages and alignment are superfluous (likely, as
the allocator paradigm has simplified these issues). -no?
Steve
James Harris
> "James Harris" <james.harri...@googlemail.com> wrote in message
>
> news:1dfc362b-9220-4cab...@l19g2000yqb.googlegroups.com...
>
> > On 14 Jan, 20:23, James Harris <james.harri...@googlemail.com> wrote:
>
> > ...
>
> > I've updated the list of potential attributes (mainly to allow further
> > expansion) and put it on the wiki at
>
> > http://aodfaq.wikispaces.com/asrb
>
> "There are at least three distinct places in an operating system where
> requests for address space may be made."
>
> 1. 2. 3. ..., um, ok, you're not including stack, heap, arrays, etc. in
> you're definition of "... address space ..."
My first idea had codes for types of memory like the stack but that
got changed that to a lower-level approach. Now, instead of requesting
a stack and expecting it to be at higher addresses the idea is that
the requestor asks for space at or below a specific high address. The
"at or below" bits and the "address" fields would be set. How does
that sound? I'm not sure what you have in mind to do with by heaps and
arrays.
> FYI, my first thoughts were
> those. You're talking about allocations via an API.
An API as opposed to what? I'm actually thinking about a single system
call for reallocation where
1) New memory is allocated by calling realloc with an unused request
block
2) Memory is reallocated by calling realloc with an existing request
block
3) Memory is freed by requesting a new size of zero
So there's no malloc and no free call, only realloc or the equivalent.
Again, how does that sound?
>
> I'm wondering about you using uint64_t. I'd make a strong guess that
> "uint64_t" specifically is probably not present on a quite a few systems.
The types are only meant to be illustrative of the data widths
required. I'll bear your comments in mind if I make it concrete.
> I also know of a few 32-bit compilers where an "unsigned long long" isn't
> available either and so uint64_t can't be defined. Although it complicates
> things, if needed, you might change to an array of a smaller type. E.g.,
>
> uint64_t flags;
>
> Becomes:
>
> uint8_t flags[4];
>
> Or:
>
> uint32_t flags[2];
>
> This is assuming that the structure sizing is fixed and not determined by
> your "first 32 bits ...[as] identification value", or rb_ver (version?), or
> rb_log_word_size variables. I.e., is the structure from uint64_t flags on
> down different for 32-bits and 64-bits?
Yes. The whole thing is just a suggestion at the moment but if it
became a real structure the idea is that the first four bytes would
define the layout of the rest. If the request block was pointed to by
EBX the routine receiving the call could do
mov eax, [ebx]
and it would either recognise the value in EAX or it would not. Yes,
rb_ver is one of those four bytes and is intended to be a version
number. (Strictly, it only needs to change when an *incompatible*
change is made to the request block.)
> E.g., 32-bit data types on 32-bit
> architecture, but 64-bit data types on 64-bit architecture?
Yes. If this became a concrete structure the first 16 bytes would
always be the same regardless of word size and the address would
always be at base + 16. The address and the remaining ten words would
be 32-bit on a 32-bit machine and 64-bit on a 64-bit machine.
> If so, it might
> be better to end the struct with a single placeholder variable (e.g., the
> C90 "struct hack") and use it's location to overlay an architecture
> appropriate structure, or end the structure with a "pointer" (as an array of
> say 8 chars, e.g., uint8_t pointer[8]) whose "address" you fill in later
> (via casts) to an architecture appropriate typedef'd structure (no space
> allocated for typedef's), i.e., with an appropriate 32-bit or 64-bit layout.
I'm not familiar with the term C90 "struct hack" though I've maybe
come across the idea. Is the address in your suggestion so that an
arbitrary number of extra parameters can be added by pointing to an
additional structure? I left a reserved four or eight bytes at the end
just in case but there's plenty of space and the idea of the length
and version numbers is to allow data to be added to the structure as
it stands.
>
> A personal preference is to use multiples of 8 bits (a byte), or 4 bits (a
> nybble), or bitfields via an binary-and and #define's. I'm not fond of the
> 3-bit data since 3-bits doesn't fit well with hexadecimal.
Agreed. In the file I had on my PC I had pad bits between 3-bit fields
so they would be nibbles in the range 0 to 7. I took them out when
placing on the web as the structure is supposed to be a concept.
> On x86, 8-bit
> sizes are directly accessable. For 16-bit and 32-bit modes, 4-bit nybbles
> are also accessable via the "AAM 16" instruction (64-bit obsolete). Also,
> bitfields larger than a single bit in size, will likely require shifts to
> use their value.
AAM used to be quite slow. I don't know if it still is.
> I'm also not fond of enums or bitfields in structures,
> since old C compiler's didn't always implement these correctly. Also, I'd
> think you'd want to design the structure to work well or easily with both C
> and assembly. That may mean extra or less efficient use of space.
C's bitfields (with the colon syntax) are, AIUI, not portable. Shifted
and masked unsigned integers, though, should be OK shouldn't they?
They can do everything C colon-specified bitfields do and very
quickly. There is a set of such operations at the following link which
shows that for any bitfield only two defining parameters are needed: a
mask and a shift amount.
http://codewiki.wikispaces.com/bitfield_operations.c
As well as the usual extract and set the link shows less familiar
operations on bitfields. They will work as long as the bitfields are
in one value even if the bitfields use bits which are not adjacent.
James
James Harris
...
> Also, I'd
> think you'd want to design the structure to work well or easily with both C
> and assembly. That may mean extra or less efficient use of space.
Yes, it should be the same structure whether expressed in C or asm.
Insofar as it is or becomes a real structure it would need to define
the same memory layout regardless of language. I thought of putting up
an asm form or just putting the fields up as a table. I opted for a C
struct as I thought that would be familiar to most people.
In asm it would be
align N (preferably N is cache-line size)
rb_size: resb 1 #Bytes in the request block
resb 1 #Reserved (zero)
rb_ver: resb 1 #Version number
rb_log_word_size: resb 1 #Log of machine word size (5 or 6)
resd 1 #Reserved (zero)
flags: resq 1
address: resd/q 1
physical_address: resd/q 1
size: resd/q 1
max: resd/q 1
abs_max: resd/q 1
number_expected: resd/q 1
granularity: resd/q 1
exec_destruction: resd/q 1
garbage: resd/q 1
handle: resd/q 1
resd/q 1 #Reserved (zero)
James
James Harris
...
> RE:
> Asrb;
http://aodfaq.wikispaces.com/asrb
>
> Given that this is intended to be a maximal description is anything
> missing?
...
> ( google won't let me properly address your previous msg., to old?)
> -comment/question-
> Data drives the Methods, but the Methods are framed in context of a
> Paradigm.
>
> What is the paradigm in mind?
The paradign I have in mind is any program running while making
dynamic address-space changes. Importantly, the program could be
running in user mode or system (privileged) mode. The two types of
request seem to me to have more in common than their differences and
this is borne out in the flag bits and other fields shown on the web
page. Notably, on x86 there is one address space and, by default, one
set of page tables shared between system and user modes. Hence I'm
looking to define a common set of attributes for either type of
request.
As mentioned on the web page there seems to me to be one other
important memory request point: that where the application
(unprivileged) address-space manager makes a call to the OS to obtain
more space. I.e. the app has made a request of the AS manager (such as
via malloc) and the AS manager does not have enough to satisfy the
request and asks for more from the OS (such as via brk or mmap or
VirtualAlloc).
Speaking of malloc, there's no need for all allocation requests to
specify or require the detail shown in the web page. The C struct is
intended to identify the biggest set of attributes that could be
needed. There's nothing to stop a C program just using malloc and its
friends when the only thing it specifies is the required size. Or,
indeed, just using mmap and its friends. (If designed this way there
could be a user mmap that does what it can in user mode and calls a
privileged routine only when necessary.)
>
> Is it the usercode making the allocation request? Should the usercode
> be allowed to dynamically allocate memory for itself from the system
> pool?
To my way of thinking there could be some memory still available in
the userspace allocator - such as if it made its requests in chunks.
In which case it can satisfy a request for more space. If not it has
to get more from the kernel.
> (if the linker provides static size of program .BSS, block
> storage segment the usercode neednot/should not go outside this for
> dynamic use? -if it needs more than .BSS, something is wrong and
> suspect.)
Some programs may have a simple static memory footprint but others may
need to request and release memory as they run.
>
> Is it systemcode, a Loader, who is the requester?
The idea is that any runing program can make a request. The loader is
one such. It has unusual requirements but it is still a program.
> If so, is the
> attribute list relevant in its list of attributes?
It is intended to be.
> -say you have a
> handler and another instance of it is required dynamically, in that
> case it is systemcode Scheduler thru the Loader, making the request,
> and things like guard pages and alignment are superfluous (likely, as
> the allocator paradigm has simplified these issues). -no?
Not sure. Can you explain a bit more about what you mean?
Your mention of the linker and loader has prompted me to write a
separate post (qv) about siting the system loader which I've been
meaning to ask for some time.
James
Rod Pemberton
On 24 Feb, 19:54, "Rod Pemberton" <do_not_h...@havenone.cmm> wrote:
> "James Harris" <james.harri...@googlemail.com> wrote in message
>
> news:1dfc362b-9220-4cab...@l19g2000yqb.googlegroups.com...
>
> > On 14 Jan, 20:23, James Harris <james.harri...@googlemail.com> wrote:
>
> > ...
>
> > I've updated the list of potential attributes (mainly to allow further
> > expansion) and put it on the wiki at
>
> > > http://aodfaq.wikispaces.com/asrb
>
> > "There are at least three distinct places in an operating system where
> > requests for address space may be made."
> >
> > 1. 2. 3. ..., um, ok, you're not including stack, heap, arrays, etc. in
> > you're definition of "... address space ..."
>
>
> > FYI, my first thoughts were
> > those. You're talking about allocations via an API.
>
> An API as opposed to what?
Ok... You're passing a data structure to some type of system, function, or
API call - which is called by a user or process, etc. That call then
allocates memory. Yes? Whereas, the allocation of arrays, stack, and heap
are typically hidden or implemented behind the scenes. I.e., "... where
reequests for address space may be made ..." isn't for these "hidden"
activities, is it? (I think you may have answered "No" to this in your
reply to Steve...)
> I'm actually thinking about a single system
> call for reallocation where
>
> 1) New memory is allocated by calling realloc with an unused request
> block
> 2) Memory is reallocated by calling realloc with an existing request
> block
> 3) Memory is freed by requesting a new size of zero
>
When reading your description, it seems reasonable...
> So there's no malloc and no free call, only realloc or the equivalent.
First, I have seen simpler implementations of memory allocation than C uses.
IIRC, they eliminate free-ing but not allocation, while others use garbage
collection.
Second, this realloc won't be C spec. compliant, if that's important, since
the C spec. requires the memory to be allocated previously for realloc().
Third, implementing a C spec. compliant realloc is a real PIA. Your list
does all that and then some. The size of the old block is "unknown", i.e.,
system dependent as to how the function determines the size of the old
block, e.g., typically hidden in header behind the scenes by the compiler...
You can have a new block both larger and smaller than the old block. You
can pass in and return NULL's. You must preserve contents of old block.
Etc... There's lots of work hidden behind this "simple" function.
> I'm not familiar with the term C90 "struct hack" though I've maybe
> come across the idea.
"Flexible array members" section here:
http://www.david.tribble.com/text/cdiffs.htm
"28. Structs: ..." section here:
http://home.datacomm.ch/t_wolf/tw/c/c9x_changes.html
> Is the address in your suggestion so that an
> arbitrary number of extra parameters can be added by pointing to an
> additional structure?
Yes.
> Shifted
> and masked unsigned integers, though, should be OK shouldn't they?
I believe so...
Rod Pemberton
s_dub...@yahoo.com
wrote:
I guess with the mention of malloc & C, we're talking about a POSIX
framework.
>
> > Is it the usercode making the allocation request? Should the usercode
> > be allowed to dynamically allocate memory for itself from the system
> > pool?
>
> To my way of thinking there could be some memory still available in
> the userspace allocator - such as if it made its requests in chunks.
> In which case it can satisfy a request for more space. If not it has
> to get more from the kernel.
>
To my way of thinking, there are two storage classes of dynamic
memory.
One is incorporated thru the .BSS of the process which the process
self manages without going to the kernel. This is a static block of
blocksize determined by the programmer for the process when the
programmer builds it. The process self-manages it for its needs of a
local heap - dynamic data.
The second class of dynamic memory intimately involves the kernel.
But here it is for the larger purpose of memory management, on a
process scale, to allocate for dynamic loading of threads or forking a
process image.
Conceivably, my user process could saturate its BSS and need to clone
to meet processing demand for a dynamic thing such as meeting an
increase in demand for tcp port handling. But such a need falls into
the second class, and may mean for my user process little more than a
deperate call to the kernel; help_me_clone(), and on confirmation;
Load_Balance();
> > (if the linker provides static size of program .BSS, block
> > storage segment the usercode neednot/should not go outside this for
> > dynamic use? -if it needs more than .BSS, something is wrong and
> > suspect.)
>
> Some programs may have a simple static memory footprint but others may
> need to request and release memory as they run.
>
Why? Why load(demand) the OS with this request as a matter of habit?
Why not self manage 95% of it off a local heap spanned by .BSS? -Here
the process data is of a specific nature and generic fluff
functionality isn't required, is distained.
>
>
> > Is it systemcode, a Loader, who is the requester?
>
> The idea is that any runing program can make a request. The loader is
> one such. It has unusual requirements but it is still a program.
>
I don't think that is very safe.
<humor on> As ranking member of The Free Loader With Respect Society,
I don't think you are giving Mr. Loader his due! <humor off>
The crossroads of the OS go thru the Loader, the Loader is the
gardian of the integrity of the system (well could be, maybe even
should be). It is there that the paradigm rubber meets the road.
> > If so, is the
> > attribute list relevant in its list of attributes?
>
> It is intended to be.
>
> > -say you have a
> > handler and another instance of it is required dynamically, in that
> > case it is systemcode Scheduler thru the Loader, making the request,
> > and things like guard pages and alignment are superfluous (likely, as
> > the allocator paradigm has simplified these issues). -no?
>
> Not sure. Can you explain a bit more about what you mean?
>
Well I wrote that, more to get you to say more about the paradigm you
have in the back of your head, but roughly;
http://en.wikipedia.org/wiki/Process_(computing)
-And under 'Process States' the diagram sums up the states, but not
how they are implemented.
> Your mention of the linker and loader has prompted me to write a
> separate post (qv) about siting the system loader which I've been
> meaning to ask for some time.
>
Some of my comment belong there, probably.
Steve
> James