Using graphics for a book and teaching materials

[Alexandru Radovici]

Hello,

I am a professor at the Politehnica University of Bucharest in Romania where my research focuses on IoT. My teaching includes several operating systems and IoT classes.

I am currently writing a book for Apress regarding new ways of developing microcontroller applications that will focus on using Tock and Rust. At the same time, I am using Tock for my IoT Architectures and Services class [1].

I would like to ask for permission to use the graphics (png files) from the Tock's github documentation [2]. I am referring to the stack, architecture and memory layout. Usage will imply providing the image source and copyright.

The class materials will be provided for free online.

Thank you,

Best regards,

Alexandru

[1] https://ocw.cs.pub.ro/courses/saiot

[2] https://github.com/tock/tock/tree/master/doc

[Brad Campbell]

Hi Alexandru,

No problem. Permission granted!

It's cool to see that Tock has been useful in your classes!

- Brad

--
You received this message because you are subscribed to the Google Groups "Tock Embedded OS Development Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tock-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tock-dev/e09c1116-2289-4a20-8959-ff3760de46e6n%40googlegroups.com.

[Alexandru Radovici]

Hello Brad,

Thank you so much. I saw that you are planning a possible workshop, please feel also free to use any of my materials.

Best regards,

Alexandru

--

Alexandru RADOVICI
Universitatea "Politehnica" din Bucuresti

e-mail: al...@ipworkshop.ro
telefon: 0742061223
www.ipworkshop.ro

[Alexandru Radovici]

Hi guys,

While writing a chapter about the Tock user space, the following question popped up regarding user space memory layout. There seems to be 3 parameters related to it: STACK_SIZE, HEAP_SIZE and KERNEL_HEAP_SIZE. So far my understanding is that KERNEL_HEAP_SIZE is the minimum size for the grants space (not sure of this includes the PCB and the pointer list). What I cannot figure out is why the application should provide this, as the kernel is perfectly capable of computing this value (and it actually does this). Any feedback would be great.

Thank you,

Alexandru

[Hudson Randal Ayers]

Hi Alexandru,

I am not an expert on how the kernel handles this, but I'll take a shot at answering your question.

I believe there are two reasons that `KERNEL_HEAP_SIZE` can be set by the app. First, Tock provides the `DynamicGrant` mechanism which allows for capsules to *dynamically* allocate memory from a process' grant region. At compile-time it is generally impossible to know how much memory capsules might dynamically allocate in response to application requests, so it is left up to the application to reserve enough in-kernel heap space for any requests it expects underlying capsules to make. Second, even "normal" `Grant`s are lazily allocated the first time they are accessed. This is important because most applications will use only a small fraction of the available capsules on a board, and thus a small fraction of the total grants. Reserving memory for every process to use every grant would probably exhaust all RAM on most Tock chips if more than a few processes were loaded. But only an application can know at loading time how many capsules it will interface with (at least unless we change tbfheader to require all apps to specify a list of driver numbers), so the application is responsible for requesting enough memory for these grants as well.

I do not think it is true that the kernel computes the minimum size for the grants space -- I think it just computes the minimum size for the pointer list. (Please correct me if I missed something).

Tracing through elf2tab and the tock kernel, KERNEL_HEAP_SIZE is expected to include any space used for "dynamic" and "non-dynamic" allocations in a process' grant region, or for the Process struct (what you call the PCB). 

elf2tab calculates the minimum_app_ram_size stored in the tbfheader as {sum of size of all segments in elf destined for RAM} + STACK_SIZE + HEAP_SIZE + KERNEL_HEAP_SIZE.

process.rs calculates initial_kernel_memory_size as the size of grant pointer list + the size of the callbacks buffer for that process + the size of the Process struct (aka PCB).

process.rs then computes min_total_memory_size as minimum_app_ram_size + initial_kernel_memory_size.

One thing to note is that no capsules in upstream Tock today actually use the `DynamicGrant` mechanism, so in practice apps today only have to worry about the amount of memory that will be used by "normal" `Grant`s. 

I hope this is helpful, please let me know if any of that does not make sense.

Best,

Hudson

---

From: tock...@googlegroups.com <tock...@googlegroups.com> on behalf of Alexandru Radovici <msg4...@gmail.com>
Sent: Thursday, January 7, 2021 5:28 AM
To: Brad Campbell <bra...@gmail.com>
Cc: Tock Embedded OS Development Discussion <tock...@googlegroups.com>
Subject: Re: [tock-dev] Using graphics for a book and teaching materials

 

To view this discussion on the web visit https://groups.google.com/d/msgid/tock-dev/CAHGAQoz3Mtf%3DDTu0La31E0Pev0qqDUMG2qsxiVOiX_1BthbkXA%40mail.gmail.com.

[Brad Campbell]

Hudson is spot on.

Second, even "normal" `Grant`s are lazily allocated the first time they are accessed. This is important because most applications will use only a small fraction of the available capsules on a board, and thus a small fraction of the total grants. Reserving memory for every process to use every grant would probably exhaust all RAM on most Tock chips if more than a few processes were loaded. But only an application can know at loading time how many capsules it will interface with (at least unless we change tbfheader to require all apps to specify a list of driver numbers), so the application is responsible for requesting enough memory for these grants as well.

This is the main reason why we expect apps to know how much grant space they need. Every app on the board has the same number of grant pointers (the number is based on how the kernel is compiled). However, different apps will use different numbers of capsules, and therefore require different amounts of grant space.

- Brad

[Alexandru Radovici]

On Thu, Jan 7, 2021 at 9:49 PM Hudson Randal Ayers <hay...@stanford.edu> wrote:

I think it is correct that the kernel app break may be lowered until it overlaps with the app break. So it is perhaps correct to consider KERNEL_HEAP_SIZE as only a hint -- the only guarantee is that the app will get at least APP_MEMORY + KERNEL_HEAP_SIZE bytes of RAM for combined use by the application and kernel. But if an app allocates more than APP_MEMORY bytes, it is possible that the kernel would then be unable to allocate KERNEL_HEAP_SIZE bytes afterward.

Is this the desired behaviour or should the kernel actually enforce the minimum KERNEL_HEAP_SIZE? I don't think it enforces it now.

Ah, DynamicGrant is the name for the fixed version which has not yet been merged. But even without that PR, the way dynamic allocation works is that `Grant::enter()` takes a parameter of type `F: FnOnce(&mut Borrowed<T>, &mut Allocator) -> R,`, and then supplies an allocator to that function. If a capsule wants to dynamically allocate memory based on a process' request, it can use the allocator struct passed to that function to do so. No capsules actually do this today so I can't point you to an example.

I can see how this is a small problem, as the Owned memory is actually invalid if the process stops. The PR does make sense now.

Hudson

---

From: Alexandru Radovici <msg4...@gmail.com>
Sent: Thursday, January 7, 2021 10:36 AM
To: Brad Campbell <bra...@gmail.com>
Cc: Hudson Randal Ayers <hay...@stanford.edu>; Tock Embedded OS Development Discussion <tock...@googlegroups.com>

Subject: Re: [tock-dev] Using graphics for a book and teaching materials

 

Hi Hudson, Brad,

Thank you for your answers, it is clear now. Correct me if I am wrong, but the KERNEL_HEAP_SIZE is still only a hint, as I don't see any limit for grant allocation within the kernel. As long as it does not overlap with the app break, the kernel app break may be lowered.

I have another question regarding the Dynamic Grants. I was the two PRs with it, but I cannot find anything in the kernel. There does not seem to be this option in grant.rs. Am I looking in the wrong place?

Thank you,

Alexandru

[Hudson Randal Ayers]

The choices on that behavior predate my involvement with Tock, but that design makes sense to me as it minimizes the chances of an OOM error. Enforcing the KERNEL_HEAP_SIZE bytes would just increase the likelihood of OOM crashes for apps. Today, even if an app uses more than APP_MEMORY bytes for use as its heap in userspace, it will not necessarily crash, as there is often extra memory leftover from the kernel not using all of KERNEL_HEAP_SIZE bytes. And there is effectively no difference between a userspace OOM error and the kernel attempting to allocate too much in a grant -- in either case. the process is faulted, but the kernel remains alive.

---

From: Alexandru Radovici <msg4...@gmail.com>
Sent: Thursday, January 7, 2021 11:57 AM
To: Hudson Randal Ayers <hay...@stanford.edu>
Cc: Brad Campbell <bra...@gmail.com>; Tock Embedded OS Development Discussion <tock...@googlegroups.com>

[Brad Campbell]

Right, the app actually only asks the kernel for memory in a single value (https://github.com/tock/tock/blame/master/doc/TockBinaryFormat.md#L226). Separating the categories only exists in libtock-c; by the time it is compiled into a TBF binary there is only one value for requesting memory. The kernel tries to assume almost nothing about how the app wants to use its memory, that is left (almost) entirely up to the app.

[Hudson Randal Ayers]

I think it is correct that the kernel app break may be lowered until it overlaps with the app break. So it is perhaps correct to consider KERNEL_HEAP_SIZE as only a hint -- the only guarantee is that the app will get at least APP_MEMORY + KERNEL_HEAP_SIZE bytes of RAM for combined use by the application and kernel. But if an app allocates more than APP_MEMORY bytes, it is possible that the kernel would then be unable to allocate KERNEL_HEAP_SIZE bytes afterward.

Ah, DynamicGrant is the name for the fixed version which has not yet been merged. But even without that PR, the way dynamic allocation works is that `Grant::enter()` takes a parameter of type `F: FnOnce(&mut Borrowed<T>, &mut Allocator) -> R,`, and then supplies an allocator to that function. If a capsule wants to dynamically allocate memory based on a process' request, it can use the allocator struct passed to that function to do so. No capsules actually do this today so I can't point you to an example.

Hudson

---

From: Alexandru Radovici <msg4...@gmail.com>
Sent: Thursday, January 7, 2021 10:36 AM
To: Brad Campbell <bra...@gmail.com>
Cc: Hudson Randal Ayers <hay...@stanford.edu>; Tock Embedded OS Development Discussion <tock...@googlegroups.com>

Subject: Re: [tock-dev] Using graphics for a book and teaching materials

Hi Hudson, Brad,

Thank you for your answers, it is clear now. Correct me if I am wrong, but the KERNEL_HEAP_SIZE is still only a hint, as I don't see any limit for grant allocation within the kernel. As long as it does not overlap with the app break, the kernel app break may be lowered.

I have another question regarding the Dynamic Grants. I was the two PRs with it, but I cannot find anything in the kernel. There does not seem to be this option in grant.rs. Am I looking in the wrong place?

Thank you,

Alexandru

[Alexandru Radovici]

Hi Hudson, Brad,

Thank you for your answers, it is clear now. Correct me if I am wrong, but the KERNEL_HEAP_SIZE is still only a hint, as I don't see any limit for grant allocation within the kernel. As long as it does not overlap with the app break, the kernel app break may be lowered.

I have another question regarding the Dynamic Grants. I was the two PRs with it, but I cannot find anything in the kernel. There does not seem to be this option in grant.rs. Am I looking in the wrong place?

Thank you,

Alexandru

[Alexandru Radovici]

Hello,

Thank you for all the information. I finished one chapter of the book that is describing the Tock architecture. I wrote it to the best of my knowledge and understanding of the Tock code. I was wondering if one of you, more experienced authors of Tock, would like and have the time to take a look at it, just to be sure that I did not miss anything important. Your feedback would be very valuable.

Thank you,

Alexandru

[Amit Levy]

Alexandru,

I'd love to take a look and offer feedback.

-Amit

To view this discussion on the web visit https://groups.google.com/d/msgid/tock-dev/CAHGAQozDzu385-%2BcJ0z7CXKqiWxT6gnFoOPTE1Ndt1-gCL7C3g%40mail.gmail.com.

[Alexandru Radovici]

Hello Amit,

Thank you very much, I appreciate it. I have shared with you a google drive document. Please excuse some of my English expressions as it is not my first language :) .

Alexandru

To view this discussion on the web visit https://groups.google.com/d/msgid/tock-dev/f6875d31-1372-a8c9-3428-f2ec508b31e3%40amitlevy.com.