Importable use of SYS_tid in Core_extended

[Anil Madhavapeddy]

As some context, we've almost got all of Core_extended compiling under
OpenBSD, so it can be used to work through Real World OCaml. There's one
minor stickling point left, which is this snippet of core in the C stubs
get the current thread id.

--- extended_unix_stubs.c.orig Thu Aug 7 15:22:05 2014
+++ extended_unix_stubs.c Thu Aug 7 15:23:59 2014

#ifndef __USE_ISOC99
@@ -75,7 +76,7 @@ CAMLprim value statvfs_stub (value v_path)
/* copy of the ocaml's stdlib wrapper for getpid */
CAMLprim value extended_ml_gettid(value v_unit __unused)
{
- return Val_int(syscall(SYS_gettid));
+ return Val_int(syscall(SYS_getpid));
}

Discussion continues below with the list CCed:

On 27 Aug 2014, at 05:32, Philip Guenther <pgue...@proofpoint.com> wrote:

> On Tue, 26 Aug 2014, Kenneth R Westerback wrote:
>> It is now in-tree and core_extended, with the diff below, seems to work
>> to the extent RWO's github_org_info example requires. Using SYS_getpid
>> instead of SYS_gettid is likely wrong on several levels, but I don't
>> think adding SYS_gettid to OpenBSD is likely in the near future. Unless
>> guenther@ has more info. I note we do support SYS_gettid in
>> compat/linux.
>
> "What problem is it trying to solve?" That is, what does it want to use
> this for? gettid() implies that it can use native threads; I'm presuming
> it implements this on top of pthreads? If so, is there some reason it
> doesn't use the opaque pthread_t value returned by pthread_self()?
>
> Note that on many platforms, including amd64, i386, and sparc64 on
> OpenBSD, pthread_self() doesn't require a system call, getting its value
> (semi-)directly from the arch's thread register.

OCaml does indeed link to pthread, so it should be able to use pthread_t
rather than the thread id. Can one of the Core authors comment on what
the direct SYS_tid value is used for -- could it be replaced with pthread_t
references instead?

thanks,
Anil

[Stephen Weeks]

I suspect someone was just wrapping system calls and put this one in
for completeness. We have often written code for Linux without
consideration of portability, and this is likely one such case. Our
usual approach to handling portability is to wrap a function in an
[Or_error.t]. In fact, we've already done this for
[Core.Linux_ext.gettid]:

(* Get the thread ID of the current thread (see gettid(2)). *)
val gettid : (unit -> int) Or_error.t

Given the existence of that, I think it would be OK for us to delete
[Core_extended.Unix.gettid].

> --
> You received this message because you are subscribed to the Google Groups "ocaml-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ocaml-core+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Yaron Minsky]

Looping in Mark.

Can we go one step better and make gettid use the pthread call? Seems
better if we can increase the portability so that more of these
optional bits are available on more UNIXs...

y

[Yaron Minsky]

OK, so it seems like making it a platform-dependent value (i.e., it's
an Or_error that might or might not be there) and put our Linux
implementation in there with an IFDEF, seems about right. Others can
presumably contribute other implementations if that has value.

y

On Fri, Aug 29, 2014 at 10:05 AM, Kenneth Westerback
<kwest...@gmail.com> wrote:
> On 28 August 2014 07:52, Mark Shinwell <mshi...@janestreet.com> wrote:
>> [gettid] doesn't return the same thing as [pthread_self], though. The
>> latter returns an opaque handle used by the pthreads library; the
>> former returns the numeric thread ID. The purpose of the
>> [Core_extended] function seems almost certainly to be for retrieval of
>> the numeric ID, e.g. for correlation with the output of "top".
>>
>> How is the numeric thread ID obtained on a BSD system?
>>
>> Mark
>
> IANAE, but google says there is no "BSD" mechanism, just a bunch of
> different implementations. e.g. WINE has or had
>
> 1303 /***********************************************************************
> 1304 * get_unix_tid
> 1305 *
> 1306 * Retrieve the Unix tid to use on the server side for the current thread.
> 1307 */
> 1308 static int get_unix_tid(void)
> 1309 {
> 1310 int ret = -1;
> 1311 #ifdef HAVE_PTHREAD_GETTHREADID_NP
> 1312 ret = pthread_getthreadid_np();
> 1313 #elif defined(linux)
> 1314 ret = syscall( __NR_gettid );
> 1315 #elif defined(__sun)
> 1316 ret = pthread_self();
> 1317 #elif defined(__APPLE__)
> 1318 ret = mach_thread_self();
> 1319 mach_port_deallocate(mach_task_self(), ret);
> 1320 #elif defined(__NetBSD__)
> 1321 ret = _lwp_self();
> 1322 #elif defined(__FreeBSD__)
> 1323 long lwpid;
> 1324 thr_self( &lwpid );
> 1325 ret = lwpid;
> 1326 #elif defined(__DragonFly__)
> 1327 ret = lwp_gettid();
> 1328 #endif
> 1329 return ret;
> 1330 }
> 1331
>
> OpenBSD does not have pthread_getthreadid_np() as far as I can grep.
> For linux compat OpenBSD uses
>
> linux_pid_t
> linux_sys_gettid(struct proc *p, void *v, register_t *retval)
> {
> *retval = p->p_pid + THREAD_PID_OFFSET;
> return (0);
> }
>
> where THREAD_PID_OFFSET is currently defined in sys/proc.h as
>
> #define THREAD_PID_OFFSET 1000000
>
> Not sure how helpful all that is. :-). I guess partly it depends on
> what one expects to be able to do with the tid. Just display it? use
> it for debugging info? somehow influence the process/thread?
>
> .... Ken

[Mark Shinwell]

Ken, if you display threads in "top" on your BSD system, do you get
numeric IDs for the threads? Those are the values we should aim to
get, I think.

Mark

[Mark Shinwell]

[gettid] doesn't return the same thing as [pthread_self], though. The
latter returns an opaque handle used by the pthreads library; the
former returns the numeric thread ID. The purpose of the
[Core_extended] function seems almost certainly to be for retrieval of
the numeric ID, e.g. for correlation with the output of "top".

How is the numeric thread ID obtained on a BSD system?

Mark

On 28 August 2014 11:53, Yaron Minsky <ymi...@janestreet.com> wrote:

[Anil Madhavapeddy]

Yes, that sounds good to me. I'll be rotating OpenBSD into the OPAM
acceptance tests for new releases so we'll be aware of breakage as it
comes in. Just need to get a single working release and then it can
stay working quite easily, and we're pretty close now.

-anil

[Stephen Weeks]

I think that [Core.Linux_ext.gettid] is misnamed, given that it is an
[Or_error] for portability yet has "Linux" in its name. I think we
should rename [Core.Linux_ext.gettid] as [Core.Unix.gettid], and then
delete the gettid code in [Core_extended].

[Philip Guenther]

On Thu, 28 Aug 2014, Mark Shinwell wrote:
> [gettid] doesn't return the same thing as [pthread_self], though. The
> latter returns an opaque handle used by the pthreads library; the
> former returns the numeric thread ID. The purpose of the
> [Core_extended] function seems almost certainly to be for retrieval of
> the numeric ID, e.g. for correlation with the output of "top".

This is what I was looking for: this function is not for internal use by
the implementation itself but rather for diagnostic displays to the user
for correlation with outside tools.

> How is the numeric thread ID obtained on a BSD system?

For that purpose matching up with the ps -H output, you should use
getthrid() on OpenBSD. It's not currently even prototyped or documented.
I'll look at fixing those sooner, but for the immediate moment you can
declare it locally as:

pid_t getthrid(void);


(Why my hesitancy in mentioning this? I've seen some *horrid* misuses of
tids in other software...)


Philip

[Mark Shinwell]

Philip, could you just confirm for me what the correct preprocessor
conditional for OpenBSD compilation should be?

Mark

[Philip Guenther]

On Sun, 31 Aug 2014, Mark Shinwell wrote:
> Philip, could you just confirm for me what the correct preprocessor
> conditional for OpenBSD compilation should be?

Test whether __OpenBSD__ is defined. #ifdef vs #if defined() according to
your project's style...


Philip Guenther