Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Re: _SC_SIGQUEUE_MAX
4 views
Skip to first unread message
Christos Zoulas
Jun 9, 2016, 11:00:40 PM6/9/16
to
On Jun 9, 5:35pm, charles...@gmail.com (Charles Cui) wrote:
-- Subject: _SC_SIGQUEUE_MAX
| Hi guys,
|
| I am trying to add the support of SIGQUEUE_MAX, which is the maximal
| number of real time signals to the target process. There are several ways
| to
| send a signal to a process like, kill, raise, etc. I am wondering whether
| all real
| time signals should be only sent via sigqueue? (in which case, establishing
| a queue in the sigqueue code path and manage the queue should be enough to
| support _SC_SIGQUEUE_MAX).
You don't have to queue all signals; specially the ones that come from
traps. But if you look at the FreeBSD implementation they allow all of
them (at least this is from a cursory reading).
christos
-- Subject: _SC_SIGQUEUE_MAX
| Hi guys,
|
| I am trying to add the support of SIGQUEUE_MAX, which is the maximal
| number of real time signals to the target process. There are several ways
| to
| send a signal to a process like, kill, raise, etc. I am wondering whether
| all real
| time signals should be only sent via sigqueue? (in which case, establishing
| a queue in the sigqueue code path and manage the queue should be enough to
| support _SC_SIGQUEUE_MAX).
You don't have to queue all signals; specially the ones that come from
traps. But if you look at the FreeBSD implementation they allow all of
them (at least this is from a cursory reading).
christos
Charles Cui
Jun 10, 2016, 12:55:47 AM6/10/16
to
Hi guys,
I am trying to add the support of SIGQUEUE_MAX, which is the maximal
number of real time signals to the target process. There are several ways
to
send a signal to a process like, kill, raise, etc. I am wondering whether
all real
time signals should be only sent via sigqueue? (in which case, establishing
a queue in the sigqueue code path and manage the queue should be enough to
support _SC_SIGQUEUE_MAX).
Thanks, Charles
I am trying to add the support of SIGQUEUE_MAX, which is the maximal
number of real time signals to the target process. There are several ways
to
send a signal to a process like, kill, raise, etc. I am wondering whether
all real
time signals should be only sent via sigqueue? (in which case, establishing
a queue in the sigqueue code path and manage the queue should be enough to
support _SC_SIGQUEUE_MAX).
Christos Zoulas
Jun 10, 2016, 7:29:49 PM6/10/16
to
On Jun 10, 2:37pm, charles...@gmail.com (Charles Cui) wrote:
-- Subject: Re: _SC_SIGQUEUE_MAX
| Thanks, Christos, I will read FreeBSD related code carefully.
In the meantime also:
- there are no man pages or tests for PRIOPROTECT_AND_GETCLOCK
- there are no man pages or tests for PTHREAD (barrierattr calls)
- how did the SIGPOLL patch work?
I committed:
- SEM_NSEMS_MAX
- TIMERS
Best,
christos
-- Subject: Re: _SC_SIGQUEUE_MAX
| Thanks, Christos, I will read FreeBSD related code carefully.
In the meantime also:
- there are no man pages or tests for PRIOPROTECT_AND_GETCLOCK
- there are no man pages or tests for PTHREAD (barrierattr calls)
- how did the SIGPOLL patch work?
I committed:
- SEM_NSEMS_MAX
- TIMERS
Best,
christos
Christos Zoulas
Jun 10, 2016, 9:00:57 PM6/10/16
to
| I will add them. They are missing now.
Good. Thanks. Unit-tests would be good too, and they will show you how
to write atf tests...
| It is finished and worked well.
| Here is the patch
Great, I committed it.
The big job is the realtime signals...
christos
Good. Thanks. Unit-tests would be good too, and they will show you how
to write atf tests...
| It is finished and worked well.
| Here is the patch
Great, I committed it.
The big job is the realtime signals...
christos
Charles Cui
Jun 11, 2016, 4:47:51 PM6/11/16
to
Please see my comments inline.
2016-06-10 16:29 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
Here is the effectiveness
https://github.com/ycui1984/posixtestsuite/commit/e429db930832b83d575702813f15344a2042bcca
2016-06-10 16:29 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
> | Thanks, Christos, I will read FreeBSD related code carefully.
>
> In the meantime also:
>
> - there are no man pages or tests for PRIOPROTECT_AND_GETCLOCK
> - there are no man pages or tests for PTHREAD (barrierattr calls)
>
>
> In the meantime also:
>
> - there are no man pages or tests for PRIOPROTECT_AND_GETCLOCK
> - there are no man pages or tests for PTHREAD (barrierattr calls)
>
I will add them. They are missing now.
> - how did the SIGPOLL patch work?
>
>
It is finished and worked well.
Here is the patch
https://github.com/ycui1984/posixtestsuite/tree/master/patches/SIGNAL
Here is the patch
Here is the effectiveness
https://github.com/ycui1984/posixtestsuite/commit/e429db930832b83d575702813f15344a2042bcca
Charles Cui
Jun 14, 2016, 1:39:58 AM6/14/16
to
Hi Christos,
I have completed the man pages for PRIOPROTECT_AND_GETCLOCK and PTHREAD.
Basically, I have organized them into 2 patches.
One patch provides man pages for pthread functions that I have added.
Please find it here (
https://github.com/ycui1984/posixtestsuite/commit/a5a3390253995934b20e9af6932377a0d118cd3d
)
The other patch provides man pages for_SC_THREAD_PRIO_PROTECT. Please find
it here
(
https://github.com/ycui1984/posixtestsuite/commit/8edccb8a7ec18dc7af990bae9d002a219741f0e0
)
Let me know if there are problems. In the next several days, I will try to
add some unit tests.
After adding the unit tests, I will continue the work of real time signals.
Thanks, Charles.
2016-06-10 16:29 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
> On Jun 10, 2:37pm, charles...@gmail.com (Charles Cui) wrote:
> -- Subject: Re: _SC_SIGQUEUE_MAX
>
> | Thanks, Christos, I will read FreeBSD related code carefully.
>
> In the meantime also:
>
> - there are no man pages or tests for PRIOPROTECT_AND_GETCLOCK
> - there are no man pages or tests for PTHREAD (barrierattr calls)
I have completed the man pages for PRIOPROTECT_AND_GETCLOCK and PTHREAD.
Basically, I have organized them into 2 patches.
One patch provides man pages for pthread functions that I have added.
Please find it here (
https://github.com/ycui1984/posixtestsuite/commit/a5a3390253995934b20e9af6932377a0d118cd3d
)
The other patch provides man pages for_SC_THREAD_PRIO_PROTECT. Please find
it here
(
https://github.com/ycui1984/posixtestsuite/commit/8edccb8a7ec18dc7af990bae9d002a219741f0e0
)
Let me know if there are problems. In the next several days, I will try to
add some unit tests.
After adding the unit tests, I will continue the work of real time signals.
Thanks, Charles.
2016-06-10 16:29 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
> On Jun 10, 2:37pm, charles...@gmail.com (Charles Cui) wrote:
> -- Subject: Re: _SC_SIGQUEUE_MAX
>
> | Thanks, Christos, I will read FreeBSD related code carefully.
>
> In the meantime also:
>
> - there are no man pages or tests for PRIOPROTECT_AND_GETCLOCK
> - there are no man pages or tests for PTHREAD (barrierattr calls)
> - how did the SIGPOLL patch work?
>
>
Christos Zoulas
Jun 14, 2016, 12:58:32 PM6/14/16
to
In article <CA+SXE9s4bY1KLXgXUWWgdv+0...@mail.gmail.com>,
this is why we need to add unit tests. I am not going to commit
them before we make sure that they work properly, and for that we
need real unit-tests. For example, I had asked you to change the
hard-coded masks to macros... One of the reasons I did that is
because the code looks wrong, for example:
pthread_mutexattr_gettype(const pthread_mutexattr_t *attr, int *typep)
{
+ uintptr_t val;
+
pthread__error(EINVAL, "Invalid mutex attribute",
attr->ptma_magic == _PT_MUTEXATTR_MAGIC);
- *typep = (int)(intptr_t)attr->ptma_private;
+ val = (uintptr_t)attr->ptma_private & ~0x00ffL;
+ *typep = (int)val;
return 0;
}
That should be & 0x00ff probably. Don't bother fixing it now, I have
created macros for all of them and fixed some other minor indentation
and whitespace nits. I will post the diffs once everything builds later.
There are a few questions:
1. Should we commit the pshared stuff now (when we don't really support
process shared mutexes), or will that confuse 3rd party software?
2. Should there be more technical review for ad@'s priority inversion work?
3. Is the PTHREAD_STACK_MIN patch useful, or can we do better?
Best,
christos
Charles Cui <charles...@gmail.com> wrote:
>
>Let me know if there are problems. In the next several days, I will try to
>add some unit tests.
>After adding the unit tests, I will continue the work of real time signals.
They mostly look fine. I am not sure that stuff has ever been tested
>
>Let me know if there are problems. In the next several days, I will try to
>add some unit tests.
>After adding the unit tests, I will continue the work of real time signals.
this is why we need to add unit tests. I am not going to commit
them before we make sure that they work properly, and for that we
need real unit-tests. For example, I had asked you to change the
hard-coded masks to macros... One of the reasons I did that is
because the code looks wrong, for example:
pthread_mutexattr_gettype(const pthread_mutexattr_t *attr, int *typep)
{
+ uintptr_t val;
+
pthread__error(EINVAL, "Invalid mutex attribute",
attr->ptma_magic == _PT_MUTEXATTR_MAGIC);
- *typep = (int)(intptr_t)attr->ptma_private;
+ val = (uintptr_t)attr->ptma_private & ~0x00ffL;
+ *typep = (int)val;
return 0;
}
That should be & 0x00ff probably. Don't bother fixing it now, I have
created macros for all of them and fixed some other minor indentation
and whitespace nits. I will post the diffs once everything builds later.
There are a few questions:
1. Should we commit the pshared stuff now (when we don't really support
process shared mutexes), or will that confuse 3rd party software?
2. Should there be more technical review for ad@'s priority inversion work?
3. Is the PTHREAD_STACK_MIN patch useful, or can we do better?
Best,
christos
Christos Zoulas
Jun 14, 2016, 5:14:14 PM6/14/16
to
| 1. add unit tests for ad's work to make sure it works well.
| 2. may add some changes according to your comments on PTHREAD_STACK_MIN.
Yes, and for me is to post my diffs of your changes.
I think we should hold both the code and the data changes for the pshared
stuff until we have them. Perhaps I will ifdef it out.
christos
| 2. may add some changes according to your comments on PTHREAD_STACK_MIN.
Yes, and for me is to post my diffs of your changes.
I think we should hold both the code and the data changes for the pshared
stuff until we have them. Perhaps I will ifdef it out.
christos
Charles Cui
Jun 14, 2016, 5:56:32 PM6/14/16
to
man pages.
Actually it does not related with ad's patch but is included in the ad's
working porting patch (I did the modification of
this function from code review and ad's work simultaneously.)
The other thing is about PTHREAD_STACK_MIN, do you think we should go to
arch dependent way?
>
> christos
>
Christos Zoulas
Jun 14, 2016, 6:44:04 PM6/14/16
to
| One thing I guess you can commit is pthread_condattr_getclock and it's
| man pages.
I will wait on that; let me get you my proposed patch back.
| man pages.
| Actually it does not related with ad's patch but is included in the ad's
| working porting patch (I did the modification of
| this function from code review and ad's work simultaneously.)
| The other thing is about PTHREAD_STACK_MIN, do you think we should go to
| arch dependent way?
christos
Christos Zoulas
Jun 14, 2016, 7:02:56 PM6/14/16
to
| I got it. Thanks Christos.
And here it is:
http://www.netbsd.org/~christos/pthread20160614.diff
christos
And here it is:
http://www.netbsd.org/~christos/pthread20160614.diff
christos
Charles Cui
Jun 14, 2016, 11:18:04 PM6/14/16
to
Charles Cui
Jun 15, 2016, 2:36:10 AM6/15/16
to
Hi Christos,
Please find my comments below.
I am sorry to miss your instruction of using macro instead of direct bit
mask.
and Thanks for your work!
>
> There are a few questions:
>
> 1. Should we commit the pshared stuff now (when we don't really support
> process shared mutexes), or will that confuse 3rd party software?
I think we can hold the pshared stuff until we have cross process
synchronization.
You can commit other man pages.
>
2. Should there be more technical review for ad@'s priority inversion work?
>
Yes, we should. We need more persons with deep understanding of NetBSD
(like you and Martin) to review
the patch before we commit because it is not a small feature.
> 3. Is the PTHREAD_STACK_MIN patch useful, or can we do better?
>
This patch actually makes use of MINSIGSTKSZ, which is a fixed value in
NetBSD and is arch dependent
in FreeBSD (the accurate name is __MINSIGSTKSZ). So, we may go to the
FreeBSD's way to make
the macro arch dependent if there is no netbsd specific reason to keep it
as a const.
So to conclude, the action items for me is to
>
> christos
>
>
>
Please find my comments below.
mask.
and Thanks for your work!
>
> There are a few questions:
>
> 1. Should we commit the pshared stuff now (when we don't really support
> process shared mutexes), or will that confuse 3rd party software?
synchronization.
You can commit other man pages.
>
2. Should there be more technical review for ad@'s priority inversion work?
>
(like you and Martin) to review
the patch before we commit because it is not a small feature.
> 3. Is the PTHREAD_STACK_MIN patch useful, or can we do better?
>
NetBSD and is arch dependent
in FreeBSD (the accurate name is __MINSIGSTKSZ). So, we may go to the
FreeBSD's way to make
the macro arch dependent if there is no netbsd specific reason to keep it
as a const.
So to conclude, the action items for me is to
1. add unit tests for ad's work to make sure it works well.
2. may add some changes according to your comments on PTHREAD_STACK_MIN.
>
>
> Best,
2. may add some changes according to your comments on PTHREAD_STACK_MIN.
>
>
>
> christos
>
>
>
Charles Cui
Jun 17, 2016, 10:09:37 AM6/17/16
to
Thanks, I will complete my test cases as soon as possible.
2016-06-14 16:02 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
2016-06-14 16:02 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
> | I got it. Thanks Christos.
>
>
Charles Cui
Jun 17, 2016, 9:55:46 PM6/17/16
to
Yeah, that makes sense. I will see how to construct such a unit test(or
call it benchmark)
Thanks, Charles.
2016-06-17 10:44 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
> | Hi Christos,
> |
> | I have completed some unit tests for ad's work and run these unit
> tests
> | on two versions (my version and your version).
> | Your initial thinking is correct. There are bugs when getting and setting
> | different fields and these bugs can be
> | verified via my unit tests (it does not pass in my version, but pass in
> | your version).
> | Here are these patches.
> |
> https://github.com/ycui1984/posixtestsuite/commit/1a5bafe2691daf1755f7156d246f297568f7768b
> |
> https://github.com/ycui1984/posixtestsuite/commit/927e1ea5a3c9aa072ef7a18316903f6cf1a20056
>
> Thanks; these look great but only test the basic functionality of
> getting and setting fields. They don't test any of the actual
> functionality of the feature being added. What I would like you to
> do is to think how to construct test cases which involve multiple
> threads that create the conditions for the priority inheritance
> (and priority inversion if searching for that helps you understand
> what is going on) that the new system call is attempting to fix.
>
> christos
>
call it benchmark)
Thanks, Charles.
2016-06-17 10:44 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
> | Hi Christos,
> |
> | I have completed some unit tests for ad's work and run these unit
> tests
> | on two versions (my version and your version).
> | Your initial thinking is correct. There are bugs when getting and setting
> | different fields and these bugs can be
> | verified via my unit tests (it does not pass in my version, but pass in
> | your version).
> | Here are these patches.
> |
> https://github.com/ycui1984/posixtestsuite/commit/1a5bafe2691daf1755f7156d246f297568f7768b
> |
> https://github.com/ycui1984/posixtestsuite/commit/927e1ea5a3c9aa072ef7a18316903f6cf1a20056
>
> Thanks; these look great but only test the basic functionality of
> getting and setting fields. They don't test any of the actual
> functionality of the feature being added. What I would like you to
> do is to think how to construct test cases which involve multiple
> threads that create the conditions for the priority inheritance
> (and priority inversion if searching for that helps you understand
> what is going on) that the new system call is attempting to fix.
>
> christos
>
Charles Cui
Jun 19, 2016, 9:01:59 PM6/19/16
to
Hi Christos,
I have completed some unit tests for ad's work and run these unit tests
on two versions (my version and your version).
Your initial thinking is correct. There are bugs when getting and setting
different fields and these bugs can be
verified via my unit tests (it does not pass in my version, but pass in
your version).
Here are these patches.
https://github.com/ycui1984/posixtestsuite/commit/1a5bafe2691daf1755f7156d246f297568f7768b
https://github.com/ycui1984/posixtestsuite/commit/927e1ea5a3c9aa072ef7a18316903f6cf1a20056
Thanks Charles.
I have completed some unit tests for ad's work and run these unit tests
on two versions (my version and your version).
Your initial thinking is correct. There are bugs when getting and setting
different fields and these bugs can be
verified via my unit tests (it does not pass in my version, but pass in
your version).
Here are these patches.
https://github.com/ycui1984/posixtestsuite/commit/1a5bafe2691daf1755f7156d246f297568f7768b
https://github.com/ycui1984/posixtestsuite/commit/927e1ea5a3c9aa072ef7a18316903f6cf1a20056
2016-06-14 16:19 GMT-07:00 Charles Cui <charles...@gmail.com>:
> Thanks, I will complete my test cases as soon as possible.
>
>
>> On Jun 14, 3:50pm, charles...@gmail.com (Charles Cui) wrote:
>> -- Subject: Re: _SC_SIGQUEUE_MAX
>>
>> On Jun 14, 3:50pm, charles...@gmail.com (Charles Cui) wrote:
>> -- Subject: Re: _SC_SIGQUEUE_MAX
>>
Christos Zoulas
Jun 20, 2016, 1:34:36 PM6/20/16
to
| Yeah, that makes sense. I will see how to construct such a unit test(or
| call it benchmark)
How's that coming along? Also can you please prepare a short summary of
| call it benchmark)
what you've done so far and what you are planning to do for the rest of
the summer, since it is midsummer review time?
Thanks,
christos
Christos Zoulas
Jun 20, 2016, 9:59:11 PM6/20/16
to
| I have write a draft of the benchmark, but have not verified successfully
| on the patched system.
| The idea is to
| 1. create a mutexattr data structure and set pthread priority protect
| protocol.
| 2. set the real time scheduling policy.
| 3. use the mutexattr to init a mutex.
| 4. create 2 threads to contend for the mutex
| 5. verify that the thread who is waiting for the mutex is executed on a
| higher priority(ceiling priority of mutex)
| I will notify you the new results once I have made progress.
|
| I will send you the summary and plan in a separate email later.
Sounds like a good plan, thanks!
christos
| on the patched system.
| The idea is to
| 1. create a mutexattr data structure and set pthread priority protect
| protocol.
| 2. set the real time scheduling policy.
| 3. use the mutexattr to init a mutex.
| 4. create 2 threads to contend for the mutex
| 5. verify that the thread who is waiting for the mutex is executed on a
| higher priority(ceiling priority of mutex)
| I will notify you the new results once I have made progress.
|
| I will send you the summary and plan in a separate email later.
Sounds like a good plan, thanks!
christos
Charles Cui
Jun 21, 2016, 6:10:03 PM6/21/16
to
Hi Christos,
Some updates on the tests of priority protect.
You need to apply my new patch here,
https://github.com/ycui1984/posixtestsuite/blob/master/patches/PRIOPROTECT_AND_GETCLOCK/0005-extend-sched_param.patch
and run the test,
https://github.com/ycui1984/posixtestsuite/blob/master/patches/PRIOPROTECT_AND_GETCLOCK/0006-unit-tests-for-priority-protect.patch
You can see thread who is blocked on mutex has higher protect priority,
which is considered by kernel when scheduling.
Thanks, Charles
Some updates on the tests of priority protect.
You need to apply my new patch here,
https://github.com/ycui1984/posixtestsuite/blob/master/patches/PRIOPROTECT_AND_GETCLOCK/0005-extend-sched_param.patch
and run the test,
https://github.com/ycui1984/posixtestsuite/blob/master/patches/PRIOPROTECT_AND_GETCLOCK/0006-unit-tests-for-priority-protect.patch
You can see thread who is blocked on mutex has higher protect priority,
which is considered by kernel when scheduling.
Thanks, Charles
Charles Cui
Jun 22, 2016, 6:08:02 AM6/22/16
to
I have write a draft of the benchmark, but have not verified successfully
on the patched system.
The idea is to
1. create a mutexattr data structure and set pthread priority protect
protocol.
2. set the real time scheduling policy.
3. use the mutexattr to init a mutex.
4. create 2 threads to contend for the mutex
5. verify that the thread who is waiting for the mutex is executed on a
higher priority(ceiling priority of mutex)
I will notify you the new results once I have made progress.
I will send you the summary and plan in a separate email later.
Thanks, Charles.
on the patched system.
The idea is to
1. create a mutexattr data structure and set pthread priority protect
protocol.
2. set the real time scheduling policy.
3. use the mutexattr to init a mutex.
4. create 2 threads to contend for the mutex
5. verify that the thread who is waiting for the mutex is executed on a
higher priority(ceiling priority of mutex)
I will notify you the new results once I have made progress.
I will send you the summary and plan in a separate email later.
2016-06-20 10:34 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
> On Jun 17, 10:45am, charles...@gmail.com (Charles Cui) wrote:
> -- Subject: Re: _SC_SIGQUEUE_MAX
>
Christos Zoulas
Jun 22, 2016, 5:53:30 PM6/22/16
to
| Any comments on the tests?
Yes, I am not finished yet testing and looking. Here are some preliminary
ones:
1. You've added more fields to sched_params. This should not be done lightly
because the system call interface is an ABI. If we want to keep them we
should version the syscall. There are more questions about the change.
Is this the right set of fields? How about exposing others like depth?
Should those fields be settable? Do they belong in the sched_params if
they are not? Even if we accept all of that they changes should be
documented...
2. On the unit test itself. It is fine what you did to check if the priority
of the thread was changed after it took the mutex lock. I would have
also written the test like:
create thread 1
make thread 1 low priority
print message with priority info
grab mutex.
print message with priority info
release mutex
print message with priority info
create thread 2
make thread 2 high priority
print message with priority info
attempt to grab same mutex as thread 1
print message with priority info
see that thread 1 runs before thread 2
instead of using thread 1 and main to make things more obvious, but it
is fine.
I am running a system with the patches and testing it... Let's discuss
the sched_params syscall modification... We should also look what other
fields other OS's expose. Once we decide that you can add documentation
and I can start committing the changes.
christos
Yes, I am not finished yet testing and looking. Here are some preliminary
ones:
1. You've added more fields to sched_params. This should not be done lightly
because the system call interface is an ABI. If we want to keep them we
should version the syscall. There are more questions about the change.
Is this the right set of fields? How about exposing others like depth?
Should those fields be settable? Do they belong in the sched_params if
they are not? Even if we accept all of that they changes should be
documented...
2. On the unit test itself. It is fine what you did to check if the priority
of the thread was changed after it took the mutex lock. I would have
also written the test like:
create thread 1
make thread 1 low priority
print message with priority info
grab mutex.
print message with priority info
release mutex
print message with priority info
create thread 2
make thread 2 high priority
print message with priority info
attempt to grab same mutex as thread 1
print message with priority info
see that thread 1 runs before thread 2
instead of using thread 1 and main to make things more obvious, but it
is fine.
I am running a system with the patches and testing it... Let's discuss
the sched_params syscall modification... We should also look what other
fields other OS's expose. Once we decide that you can add documentation
and I can start committing the changes.
christos
Christos Zoulas
Jun 22, 2016, 6:28:42 PM6/22/16
to
| Thanks Christos for these comments!
| In terms of the new fields exposed by sched_param, those are only for
| testing convenience, if we decide to expose those to users, we need to
| consider carefully.
Does not matter, if we expose them or not. If the structure size changes,
it breaks the ABI... We just need to decide what to do. I don't think it
is that useful to expose them... Anyway if we change it we should decide
if we want to include the sporadic scheduler stuff so we don't have to
keep burning syscalls (or version the structure).
| In terms of the testing method proposed by you, did you already implement
| and test
| or want me to provide the test?
Why don't you give it a try...
Thanks,
christos
| In terms of the new fields exposed by sched_param, those are only for
| testing convenience, if we decide to expose those to users, we need to
| consider carefully.
Does not matter, if we expose them or not. If the structure size changes,
it breaks the ABI... We just need to decide what to do. I don't think it
is that useful to expose them... Anyway if we change it we should decide
if we want to include the sporadic scheduler stuff so we don't have to
keep burning syscalls (or version the structure).
| In terms of the testing method proposed by you, did you already implement
| and test
| or want me to provide the test?
Why don't you give it a try...
Thanks,
christos
Charles Cui
Jun 22, 2016, 6:46:45 PM6/22/16
to
Thanks Christos for these comments!
In terms of the new fields exposed by sched_param, those are only for
testing convenience, if we decide to expose those to users, we need to
consider carefully.
In terms of the new fields exposed by sched_param, those are only for
testing convenience, if we decide to expose those to users, we need to
consider carefully.
In terms of the testing method proposed by you, did you already implement
and test
or want me to provide the test?
Thanks Charles.
and test
or want me to provide the test?
2016-06-22 14:53 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
Charles Cui
Jun 22, 2016, 6:51:58 PM6/22/16
to
Any comments on the tests?
2016-06-21 14:36 GMT-07:00 Charles Cui <charles...@gmail.com>:
> Hi Christos,
>
> Some updates on the tests of priority protect.
> You need to apply my new patch here,
> https://github.com/ycui1984/posixtestsuite/blob/master/patches/PRIOPROTECT_AND_GETCLOCK/0005-extend-sched_param.patch
>
> and run the test,
>
> https://github.com/ycui1984/posixtestsuite/blob/master/patches/PRIOPROTECT_AND_GETCLOCK/0006-unit-tests-for-priority-protect.patch
> You can see thread who is blocked on mutex has higher protect priority,
> which is considered by kernel when scheduling.
>
> Thanks, Charles
>
> 2016-06-20 18:58 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
> Hi Christos,
>
> Some updates on the tests of priority protect.
> You need to apply my new patch here,
> https://github.com/ycui1984/posixtestsuite/blob/master/patches/PRIOPROTECT_AND_GETCLOCK/0005-extend-sched_param.patch
>
> and run the test,
>
> https://github.com/ycui1984/posixtestsuite/blob/master/patches/PRIOPROTECT_AND_GETCLOCK/0006-unit-tests-for-priority-protect.patch
> You can see thread who is blocked on mutex has higher protect priority,
> which is considered by kernel when scheduling.
>
> Thanks, Charles
>
>
>> On Jun 20, 1:07pm, charles...@gmail.com (Charles Cui) wrote:
>> -- Subject: Re: _SC_SIGQUEUE_MAX
>>
>> On Jun 20, 1:07pm, charles...@gmail.com (Charles Cui) wrote:
>> -- Subject: Re: _SC_SIGQUEUE_MAX
>>
>> | I have write a draft of the benchmark, but have not verified
>> successfully
>> | on the patched system.
>> | The idea is to
>> | 1. create a mutexattr data structure and set pthread priority protect
>> | protocol.
>> | 2. set the real time scheduling policy.
>> | 3. use the mutexattr to init a mutex.
>> | 4. create 2 threads to contend for the mutex
>> | 5. verify that the thread who is waiting for the mutex is executed on a
>> | higher priority(ceiling priority of mutex)
>> | I will notify you the new results once I have made progress.
>> |
>> | I will send you the summary and plan in a separate email later.
>>
>> successfully
>> | on the patched system.
>> | The idea is to
>> | 1. create a mutexattr data structure and set pthread priority protect
>> | protocol.
>> | 2. set the real time scheduling policy.
>> | 3. use the mutexattr to init a mutex.
>> | 4. create 2 threads to contend for the mutex
>> | 5. verify that the thread who is waiting for the mutex is executed on a
>> | higher priority(ceiling priority of mutex)
>> | I will notify you the new results once I have made progress.
>> |
>> | I will send you the summary and plan in a separate email later.
>>
Charles Cui
Jun 24, 2016, 5:58:05 AM6/24/16
to
I tried to implement the unit test proposed by Christos, but needs more
clarifications.
Let me first describe my understanding of your test.
First created a thread with low priority, and let it grab the mutex. After
this step, you expect the thread with low priority will increase. Then,
created a thread with higher priority. When these two threads grab mutex at
the same time, scheduler will select
the thread with higher priority. If thread with lower priority runs first,
it indicates that the priority actually increased when grabbing the mutex.
I have 2 concerns about this unit test.
1. When there is no thread competing for a mutex, or in other words, if the
mutex is available, the thread will acquire the mutex, but never increases
priority. The priority will only increases when one thread tries to acquire
a mutex but is blocked (in which case will call mutex_lock_slow).
2. The timing of this test is hard to manage. We require these two threads
compete for the same mutex at the same time.
Any events may interrupt this, and may let us see incorrect results.
Let me know if I misunderstand your idea.
Thanks, Charles.
2016-06-22 14:53 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
clarifications.
Let me first describe my understanding of your test.
First created a thread with low priority, and let it grab the mutex. After
this step, you expect the thread with low priority will increase. Then,
created a thread with higher priority. When these two threads grab mutex at
the same time, scheduler will select
the thread with higher priority. If thread with lower priority runs first,
it indicates that the priority actually increased when grabbing the mutex.
I have 2 concerns about this unit test.
1. When there is no thread competing for a mutex, or in other words, if the
mutex is available, the thread will acquire the mutex, but never increases
priority. The priority will only increases when one thread tries to acquire
a mutex but is blocked (in which case will call mutex_lock_slow).
2. The timing of this test is hard to manage. We require these two threads
compete for the same mutex at the same time.
Any events may interrupt this, and may let us see incorrect results.
Let me know if I misunderstand your idea.
Thanks, Charles.
2016-06-22 14:53 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
> | Any comments on the tests?
>
>
Christos Zoulas
Jun 24, 2016, 9:06:23 AM6/24/16
to
In article <CA+SXE9tP1u8G7D0rVz3A7Vd75v=fDEOK8ceq0gbsPWf=7dW...@mail.gmail.com>,
Charles Cui <charles...@gmail.com> wrote:
>-=-=-=-=-=-
t1 runs, grabs m1, sleeps, attempts to grab m2
t2 runs, grabs m2, attempts to grab m1
christos
Charles Cui <charles...@gmail.com> wrote:
>-=-=-=-=-=-
>
>I tried to implement the unit test proposed by Christos, but needs more
>clarifications.
>Let me first describe my understanding of your test.
>First created a thread with low priority, and let it grab the mutex. After
>this step, you expect the thread with low priority will increase. Then,
>created a thread with higher priority. When these two threads grab mutex at
>the same time, scheduler will select
>the thread with higher priority. If thread with lower priority runs first,
>it indicates that the priority actually increased when grabbing the mutex.
>I have 2 concerns about this unit test.
>1. When there is no thread competing for a mutex, or in other words, if the
>mutex is available, the thread will acquire the mutex, but never increases
>priority. The priority will only increases when one thread tries to acquire
>a mutex but is blocked (in which case will call mutex_lock_slow).
>2. The timing of this test is hard to manage. We require these two threads
>compete for the same mutex at the same time.
>Any events may interrupt this, and may let us see incorrect results.
>Let me know if I misunderstand your idea.
Perhaps this works:
>I tried to implement the unit test proposed by Christos, but needs more
>clarifications.
>Let me first describe my understanding of your test.
>First created a thread with low priority, and let it grab the mutex. After
>this step, you expect the thread with low priority will increase. Then,
>created a thread with higher priority. When these two threads grab mutex at
>the same time, scheduler will select
>the thread with higher priority. If thread with lower priority runs first,
>it indicates that the priority actually increased when grabbing the mutex.
>I have 2 concerns about this unit test.
>1. When there is no thread competing for a mutex, or in other words, if the
>mutex is available, the thread will acquire the mutex, but never increases
>priority. The priority will only increases when one thread tries to acquire
>a mutex but is blocked (in which case will call mutex_lock_slow).
>2. The timing of this test is hard to manage. We require these two threads
>compete for the same mutex at the same time.
>Any events may interrupt this, and may let us see incorrect results.
>Let me know if I misunderstand your idea.
t1 runs, grabs m1, sleeps, attempts to grab m2
t2 runs, grabs m2, attempts to grab m1
christos
Charles Cui
Jun 25, 2016, 8:16:18 AM6/25/16
to
Hi Christos,
This test generates a deadlock. t1 holds resource m1 and waits for m2,
t2 holds resource m2 and waits for m1.
Is this the result that you want to generate?
2016-06-24 6:05 GMT-07:00 Christos Zoulas <chri...@astron.com>:
> In article <CA+SXE9tP1u8G7D0rVz3A7Vd75v=fDEOK8ceq0gbsPWf=
This test generates a deadlock. t1 holds resource m1 and waits for m2,
t2 holds resource m2 and waits for m1.
Is this the result that you want to generate?
2016-06-24 6:05 GMT-07:00 Christos Zoulas <chri...@astron.com>:
> In article <CA+SXE9tP1u8G7D0rVz3A7Vd75v=fDEOK8ceq0gbsPWf=
Christos Zoulas
Jun 25, 2016, 9:33:07 AM6/25/16
to
| Hi Christos,
|
| This test generates a deadlock. t1 holds resource m1 and waits for m2,
| t2 holds resource m2 and waits for m1.
| Is this the result that you want to generate?
That is correct. It should get a deadlock. And it is the wrong test.
|
| This test generates a deadlock. t1 holds resource m1 and waits for m2,
| t2 holds resource m2 and waits for m1.
| Is this the result that you want to generate?
Perhaps we can do it with a 3rd thread like it is described here:
https://en.wikipedia.org/wiki/Priority_inversion
christos
Charles Cui
Jun 26, 2016, 8:30:22 AM6/26/16
to
Yeah, I will try to construct a new test.
Thanks, Charles.
2016-06-22 15:28 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
> keep burning syscalls (or version the structure).
Thanks, Charles.
2016-06-22 15:28 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
> | Thanks Christos for these comments!
> | In terms of the new fields exposed by sched_param, those are only for
> | testing convenience, if we decide to expose those to users, we need to
> | consider carefully.
>
> | In terms of the new fields exposed by sched_param, those are only for
> | testing convenience, if we decide to expose those to users, we need to
> | consider carefully.
>
> Does not matter, if we expose them or not. If the structure size changes,
> it breaks the ABI... We just need to decide what to do. I don't think it
> is that useful to expose them... Anyway if we change it we should decide
> if we want to include the sporadic scheduler stuff so we don't have to
> it breaks the ABI... We just need to decide what to do. I don't think it
> is that useful to expose them... Anyway if we change it we should decide
> keep burning syscalls (or version the structure).
>
> | In terms of the testing method proposed by you, did you already implement
> | and test
> | or want me to provide the test?
>
> | In terms of the testing method proposed by you, did you already implement
> | and test
> | or want me to provide the test?
>
Charles Cui
Jul 1, 2016, 5:21:00 AM7/1/16
to
Hi Christos,
I have spend several days of thinking how to design a unit test to prove
the usefulness of
the priority protect patch and here are some updates of the new unit tests.
After investigating several ways of designing the tests, I realized that if
we
can prove low priority thread runs first after raising priority due to
mutex blocking,
then it indicates scheduler takes the new priority into considerations.
Here are some steps,
1. main thread sets itself to be a realtime task and launched two tasks,
one has higher priority and
the other has lower priority.
2. each child thread(low and high priority thread) sets its scheduler and
priority.
3. each child thread did several rounds of computation, after each round it
sleep 1 second.
4. the child thread with low priority will call _sched_protect to increase
its protect priority.
5. We verify the thread with low priority runs first.
Why does it work?
For main thread, we launched thread with high priority first, this actually
gives this thread
some benefits of starting first. And if the thread with low priority did
not call _sched_protect,
the high priority thread should finish task first. After each round of
computation, we call
sleep, this will put the task into the sleep queue, and wake up again after
the timer times out.
This gives the scheduler the changes to decide which task to run.
So, thread with real high priority will always block thread with real low
priority.
Here is the code,
https://github.com/ycui1984/posixtestsuite/blob/master/patches/PRIOPROTECT_AND_GETCLOCK/0007-unit-test-for-scheduling-proof-of-priority-ceiling.patch
Thanks, Charles
2016-06-27 12:45 GMT+08:00 Charles Cui <charles...@gmail.com>:
> I will think about how to do it.
>
>
> Thanks Charles
>
> 2016-06-25 6:32 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
I have spend several days of thinking how to design a unit test to prove
the usefulness of
the priority protect patch and here are some updates of the new unit tests.
After investigating several ways of designing the tests, I realized that if
we
can prove low priority thread runs first after raising priority due to
mutex blocking,
then it indicates scheduler takes the new priority into considerations.
Here are some steps,
1. main thread sets itself to be a realtime task and launched two tasks,
one has higher priority and
the other has lower priority.
2. each child thread(low and high priority thread) sets its scheduler and
priority.
3. each child thread did several rounds of computation, after each round it
sleep 1 second.
4. the child thread with low priority will call _sched_protect to increase
its protect priority.
5. We verify the thread with low priority runs first.
Why does it work?
For main thread, we launched thread with high priority first, this actually
gives this thread
some benefits of starting first. And if the thread with low priority did
not call _sched_protect,
the high priority thread should finish task first. After each round of
computation, we call
sleep, this will put the task into the sleep queue, and wake up again after
the timer times out.
This gives the scheduler the changes to decide which task to run.
So, thread with real high priority will always block thread with real low
priority.
Here is the code,
https://github.com/ycui1984/posixtestsuite/blob/master/patches/PRIOPROTECT_AND_GETCLOCK/0007-unit-test-for-scheduling-proof-of-priority-ceiling.patch
Thanks, Charles
2016-06-27 12:45 GMT+08:00 Charles Cui <charles...@gmail.com>:
> I will think about how to do it.
>
>
> Thanks Charles
>
> 2016-06-25 6:32 GMT-07:00 Christos Zoulas <chri...@zoulas.com>:
>
>> On Jun 24, 1:56pm, charles...@gmail.com (Charles Cui) wrote:
>> -- Subject: Re: _SC_SIGQUEUE_MAX
>>
>> On Jun 24, 1:56pm, charles...@gmail.com (Charles Cui) wrote:
>> -- Subject: Re: _SC_SIGQUEUE_MAX
>>
Christos Zoulas
Jul 3, 2016, 10:10:31 AM7/3/16
to
On Jul 1, 3:32pm, charles...@gmail.com (Charles Cui) wrote:
-- Subject: Re: _SC_SIGQUEUE_MAX
| Hi Christos,
|
-- Subject: Re: _SC_SIGQUEUE_MAX
| Hi Christos,
|
christos
Christos Zoulas
Jul 3, 2016, 10:01:12 PM7/3/16
to
Can you take a stab at a man page for _sched_protect first? I think that realtime signal support is a bit too much for GSoC, what part of mmap are you suggesting to do?
thanks!
christos
> On Jul 3, 2016, at 6:33 PM, Charles Cui <charles...@gmail.com> wrote:
>
> thanks! What's the next part do you want me to focus? signal, mmap, SCHED_SPORADIC scheduling class or cross process synchronization?
> <sanitizer.log>
thanks!
christos
> On Jul 3, 2016, at 6:33 PM, Charles Cui <charles...@gmail.com> wrote:
>
> thanks! What's the next part do you want me to focus? signal, mmap, SCHED_SPORADIC scheduling class or cross process synchronization?
Charles Cui
Jul 3, 2016, 11:37:10 PM7/3/16
to
Yes, I can add the man pages for _sched_protect, mmap support is the missing
features at the bottom of this doc,
https://github.com/ycui1984/posixtestsuite/blob/master/errors.txt
features at the bottom of this doc,
https://github.com/ycui1984/posixtestsuite/blob/master/errors.txt
Christos Zoulas
Jul 4, 2016, 10:28:06 AM7/4/16
to
| Yes, I can add the man pages for _sched_protect, mmap support is the missing
| features at the bottom of this doc,
| https://github.com/ycui1984/posixtestsuite/blob/master/errors.txt
Nobody has the mmap functions yet:
| features at the bottom of this doc,
| https://github.com/ycui1984/posixtestsuite/blob/master/errors.txt
https://www.gnu.org/software/gnulib/manual/html_node/posix_005ftyped_005fmem_005fopen.html
SCHED_SPORADIC is also a feature that not many things use. So let's either
try to make the PSHARED semaphores work for real (easier) or go for the
realtime signal implementation (harder). Let's see what others think soo.
christos
Martin Husemann
Jul 4, 2016, 10:34:26 AM7/4/16
to
On Mon, Jul 04, 2016 at 10:27:52AM -0400, Christos Zoulas wrote:
> SCHED_SPORADIC is also a feature that not many things use. So let's either
> try to make the PSHARED semaphores work for real (easier) or go for the
> realtime signal implementation (harder). Let's see what others think soo.
I agree, but for PSHARED we would need discussion + consensus first,
> SCHED_SPORADIC is also a feature that not many things use. So let's either
> try to make the PSHARED semaphores work for real (easier) or go for the
> realtime signal implementation (harder). Let's see what others think soo.
which might be harder than implementing the realtime signals ;-}
Martin
Christos Zoulas
Jul 4, 2016, 11:24:11 AM7/4/16
to
be controversial too once you look at the implementation details.
christos
Charles Cui
Jul 4, 2016, 9:01:36 PM7/4/16
to
Well, in this case, what I suggest is that I try both things at the same
time and
see which part is more easily for me to make a breakthrough. Sounds good?
I will let you guys know my progress.
Thanks, Charles.
time and
see which part is more easily for me to make a breakthrough. Sounds good?
I will let you guys know my progress.
Thanks, Charles.
Charles Cui
Jul 5, 2016, 12:43:15 PM7/5/16
to
man pages for sched_protect
https://github.com/ycui1984/posixtestsuite/commit/50450adf1a890965440d6e1bf9c6f113a247867e
Thanks Charles
https://github.com/ycui1984/posixtestsuite/commit/50450adf1a890965440d6e1bf9c6f113a247867e
Thanks Charles
Christos Zoulas
Jul 5, 2016, 5:28:30 PM7/5/16
to
In article <CA+SXE9s_m0K-VDkd9j8WYeHG...@mail.gmail.com>,
christos
>man pages for sched_protect
>https://github.com/ycui1984/posixtestsuite/commit/50450adf1a890965440d6e1bf9c6f113a247867e
Thanks, fixed and committed...
>https://github.com/ycui1984/posixtestsuite/commit/50450adf1a890965440d6e1bf9c6f113a247867e
christos
0 new messages