openpty() and forkpty() Go equivalents
Roger Pau Monné
Giles Lean
Roger Pau Monné <roy...@gmail.com> wrote:
> Hello,I would like to program a remote shell server in Go (kind of
> telnetd), I've took a look at the syscall library, but I
> couldn't find anyway to open a new pseudo terminal (like the C
> functions openpty/forkpty/login_tty), how can I do it in Go?
That's fairly operating system specific behaviour, and at least on OS X
openpty() is in libc and refers to errno.
I'd recommend you start looking at your operating system and determining
what system calls (rather than library calls) or other facilities it has
(perhaps /dev/ptmx?), and also look into the cgo command if you do need
to call C code and not just Go code.
Re errno: be prepared for lots of pain. Other than when making a system
call via Go code which returns errno, it is not possible to examine
errno reliably (if it is possible at all). So I would avoid the libc
pty/tty interfaces.
On the subject of errno, while there are libc functions that use (or
abuse) errno:
a) errno is per-thread
b) goroutines are layered over operating system threads
c) goroutines migrate between operating system threads
'b' and 'c' imply that the errno you examine (if you can examine it at
all) may be the errno value from a different thread to the one you used
to make your system call, or the errno value from the same thread but
with its value changed by another goroutine sharing your thread.
I've been wondering if the Go runtime shouldn't track errno to allow
linking with libc when necessary, but I suspect the Go designers won't
want to, as:
i. they don't want to link with libc at all (as best I can tell)
ii. there would be a (small?) speed penalty for each goroutine switch,
and switching goroutines is supposed to be fast
If you have trouble understanding something I've written (it's difficult
subject matter, and I have not managed the simplest English either)
please feel free to ask more questions either to the list or to me
privately and I'll try to explain my response more clearly. But
please allow for a few hours delay; it's time for me to sleep!
Best regards,
Giles
James Aguilar
> Hello,
>
> I would like to program a remote shell server in Go (kind of telnetd), I've
> took a look at the syscall library, but I couldn't find anyway to open a new
> pseudo terminal (like the C functions openpty/forkpty/login_tty), how can I
> do it in Go?
The easiest way is to wrap the functions using the cgo foreign
function interface. I myself have successfully wrapped and used
openpty, tcgetattr and tcsetattr, along with a substantial subset of
the ncurses library. (I'm not sure if this is the "right" thing based
on Giles' response, but it worked for me.)
One side note: the ability to do openpty in OSX is predicated on a
changelist that's fallen by the wayside. If you patch
http://codereview.appspot.com/472041 into your goroot and rebuild cgo,
it should be trivial to wrap the functions you need. Now, if struct
termios is typedef'd to termios on your system, then this patch may
not even be necessary.
-- James
Giles Lean
James Aguilar <jagu...@google.com> wrote:
> The easiest way is to wrap the functions using the cgo foreign
> function interface. I myself have successfully wrapped and used
> openpty, tcgetattr and tcsetattr, along with a substantial subset of
> the ncurses library. (I'm not sure if this is the "right" thing based
> on Giles' response, but it worked for me.)
If you don't mind not being able to test errno when/if those
fail, then no harm, no foul. :-)
OTOH in a previous day job, I'd reject your code during a
review. It depends what standard of reliability and what
quality of error reporting that you need.
In a different day job, it was policy *not* to check memory
allocations but to assume that they always succeeded. That
went against my upbringing, I tell you, but not checking errno
"because the language won't let me" would have been a free
pass. Different company, different standards, and a different
quality of product.
I am now back to thinking about forcing the go runtime to
maintain errno. Yeah, doing so may be as ugly as sin. But it
will cover over a number of poor C library design choices that
cgo has no other way (that I can figure out, anyway) of
papering over or working around.
If nobody from the Go team pipes up soon I'll start a new
thread specifically to ask about errno. Without looking at
the runtime (which I'm happy enough to do, other than not
having enough time right now) I suspect it's going to be a tad
on the ugly side, but maybe not as ugly as in a MxN threads
implementation in C.
So far I think the goroutine scheduler will have to ensure
that:
a) before a system call or call to a foreign function the
goroutine must be locked to a particular OS thread such
that it can't be migrated
b) the goroutine's errno value will have to be copied into he
OS thread's errno value (e.g. for cases where it's required
to zero errno before making some library call, and on an
error return looking at errno to determine if a real error
occurred. getpwent() is a nasty example of what I'm talking
about, at least on OS X:
"Note that programs must explicitly set errno to zero
before calling any of these functions if they need to
distinguish between a non-existent entry and an error."
Blah.
c) on return from the system call or foreign function call,
the OS thread's errno value needs to be copied into the
goroutine's errno
d) the goroutine can be unlocked for migration to other
threads etc
All of (a) to (d) assumes that signals are already blocked in
all but one OS thread, which I suspect is already being done
from the semantics signal.go offers. If not, then a' is to
set the thread signal mask to block all signals, and e' is to
reset the thread signal mask to its previous value.
The presence of threads (with or without goroutines) greatly
complicates the traditional Unix errno and signal behaviour.
Nobody thinks what we have is pretty; it's just the best
people could come up with.
Now, all of that has to impose /some/ overhead, and the amount
of state a goroutine has will increase. Neither of which do
good things for speed, but the impact might be small: it would
take testing to know.
On the plus side (whatever the performance impact) we'd get
some additional flexibility and I'd like cgo a whole lot
more. :-)
> One side note: the ability to do openpty in OSX is
> predicated on a changelist that's fallen by the wayside. If
> you patch http://codereview.appspot.com/472041 into your
> goroot and rebuild cgo, it should be trivial to wrap the
> functions you need. Now, if struct termios is typedef'd to
> termios on your system, then this patch may not even be
> necessary.
Unfortunately, that issue looks more like a problem report and
question about what the proper way to fix it is than something
that can be signed off with a "LGTM" from Those In Charge.
(No offense to James for opening the issue; this is one of the
weakest areas of cgo, and I've butted my head against it too.)
I hesitate to ask who in the Go team "owns" cgo. I fear the
answer might be "whomever touched it last". But there are so
many modern OS APIs only defined in terms of libc and not
system calls that the non-typedef'd struct issue and the errno
problem really bite for systems programming work.
Giles
Ian Lance Taylor
> c) on return from the system call or foreign function call,
> the OS thread's errno value needs to be copied into the
> goroutine's errno
I think a better approach would revolve around inventing a way to have
cgo wrap a C function so that it returns two values when called from
Go, the usual return value and errno. Of course that would not be
appropriate for all C functions, so there would have to be some way to
tell cgo when to do it.
Ian
Giles Lean
That sounds more idiomatic, reduces the CPU overhead in the
normal ("we don't care about errno") case, and avoids adding
more state to goroutines.
Indeed, why didn't _I_ think of that? Too much C think, not
enough Go think. Thanks Ian.
Ian's better approach still leaves some issues open:
a) Would the wrapper always zero errno when it's planning to
return errno after the library call?
There are more cases where errno is wanted along with the
result of a library call than there are cases where errno
needs to be set prior to the call ... but OTOH we can hope
this whole situation is rare so maybe the overhead of
setting errno if errno is going to be used at all is worth
the simplicity of not breaking the cases down further.
I'm assuming here that there is no case where errno must be
set before a library call but that zero is not the
appropriate value to set it to.
Surely nobody has been so lost to good sense that they
require errno to be set to any non-zero value? (If anyone
knows of a case, this would be a really good time to
enlighten me, and get it on record in the list archives.)
b) All the synchronisation difficulties still apply to ensure
that the right thread's errno is set and retrieved without
a goroutine migration, other goroutine's call via cgo
sneaking in, or a signal handler intervening at the "wrong"
moment.
I doubt that that this synchronistion can be done entirely
in cgo without runtime support, but at this point I'm
speculating and lack knowledge of the runtime's workings.
Maybe there's a facility for taking sole control of an OS
thread that could be exposed to the "C" package for cgo's
use.
Cheers,
Giles
Russ Cox
that the two or so instructions it takes to grab errno
are not going to add noticeably to it. I think cgo
should make references to C.errno just work,
without special prior arrangement.
Russ
James Aguilar
On the other hand, how does C handle this when you're running multiple
threads? My guess is that errno access would be unsafe in that case.
Does Go actually need to make any more guarantee than "the C code will
behave the same way as it would if you called it in the same context
from C code?" I.e. accessing errno is safe as long as C foreign
functions are only being called from one of the running goroutines?
Russ Cox
> threads?
C makes it safe by making errno a thread-local variable
in a threaded program.
Cgo can easily, and should, make it equally safe.
Russ
James Aguilar
Oh, I see. I should have read Giles' first post more completely and I
would have understood that.
This does bring up another concern: that any kind of thread-local
storage done in C will be equally unusable from Go, especially in
cases where C code is "thread-hostile". That makes me think that maybe
a more general solution (which will be automatically turned on for
errno, perhaps) is necessary.
-- James
Giles Lean
Russ Cox <r...@golang.org> wrote:
Russ,
I've created http://codereview.appspot.com/798041 so that this
won't be forgotten. It includes a link to this email thread.
Maybe I'll get to look at this change myself, but I shan't
until I'm into the runtime support for my NetBSD Go port at
least. To date I've not needed to read any of the runtime code
and want to maintain _some_ focus!
FYI I didn't nominate you as the reviewer, and the TODO in the
diff (the only thing in the diff, ahem) doesn't have anyone's
name against it either, so it's as open as I can make it for
anyone who's interested to pick up.
Regards,
Giles
Giles Lean
James Aguilar <jagu...@google.com> wrote:
> This does bring up another concern: that any kind of thread-local
> storage done in C will be equally unusable from Go, especially in
> cases where C code is "thread-hostile". That makes me think that maybe
> a more general solution (which will be automatically turned on for
> errno, perhaps) is necessary.
I'd hold off on the more general solution. If we can, let's
see what it takes to get errno up and working first: there are
clear use cases for that already (e.g. openpty() mentioned in
the subject line ;-).
I mentioned thread local storage as a possible problem for
completeness, not as something that necessarily should be
addressed.
Remember that the Go designers chose goroutines over threads;
trying to use threads /and/ goroutines simultaneously looks
like inviting trouble to me, and the thought of debugging such
an application ... *shudder*.
If it /is/ necessary to allow calling of foreign functions
that use thread local storage via cgo then I suspect a
solution would be to allow a goroutine to take over a whole OS
thread for itself. Hardly elegant, but sometimes a bigger
hammer is what you want.
Finally, if I really had to write something using POSIX theads
with thread local storage (and I couldn't change jobs to get
out of it ;-) I'd think hard about just biting the bullet and
just writing the damn thing in C.
Giles