OpenBSD's strlcpy(3) and strlcat(3)
Paul Hart
Conference where Todd Miller and Theo de Raadt presented a paper on two
new functions that OpenBSD has integrated into libc. The new functions,
strlcpy(3) and strlcat(3), are intended to provide an easily understood
means of safe string copying and concatenation to programmers. Of course,
strcpy(3) and strcat(3) have obvious dangers, but their standardized
intended replacements, strncpy(3) and strncat(3), suffer from some subtle
dangers as well that can trip up even experienced programmers.
I was impressed by the paper and wondered if anyone besides myself would
be amenable to including them in FreeBSD's libc. Are there members of the
FreeBSD core and community that would be interested in importing these new
functions? The semantics of strncpy(3) and strncat(3) have struck me as
warts on the C standard for some time. I'm not sure what debate took
place on the standardization committee, but whatever it was seems to have
produced some strange results.
If you are a USENIX member you can access the text of the paper at:
http://www.usenix.org/events/usenix99/millert.html
Paul Hart
--
Paul Robert Hart ><8> ><8> ><8> Verio Web Hosting, Inc.
ha...@iserver.com ><8> ><8> ><8> http://www.iserver.com/
To Unsubscribe: send mail to majo...@FreeBSD.org
with "unsubscribe freebsd-security" in the body of the message
Matthew Dillon
:Conference where Todd Miller and Theo de Raadt presented a paper on two
:new functions that OpenBSD has integrated into libc. The new functions,
:strlcpy(3) and strlcat(3), are intended to provide an easily understood
:I was impressed by the paper and wondered if anyone besides myself would
:be amenable to including them in FreeBSD's libc. Are there members of the
:FreeBSD core and community that would be interested in importing these new
:functions? The semantics of strncpy(3) and strncat(3) have struck me as
:warts on the C standard for some time. I'm not sure what debate took
:
:If you are a USENIX member you can access the text of the paper at:
:
: http://www.usenix.org/events/usenix99/millert.html
:
:Paul Hart
:Paul Robert Hart ><8> ><8> ><8> Verio Web Hosting, Inc.
I was quite impressed with that paper too and would love to see
the functions added to libc.
-Matt
Matthew Dillon
<dil...@backplane.com>
Garrett Wollman
> The semantics of strncpy(3) and strncat(3) have struck me as warts
> on the C standard for some time. I'm not sure what debate took
> place on the standardization committee, but whatever it was seems to
> have produced some strange results.
These functions were not creations of the committee -- they have been
in C for a very long time. They (along with strncmp()) were
originally created for the purpose of dealing with `struct direct' in
Seventh Edition, which looked something like this (I've probably got
the member names wrong):
struct direct {
int d_ino;
char d_name[MAXNAMLEN];
};
The `d_name' member defined the semantics of strncpy() and strncmp().
X3J11 standardized these functions as they were.
-GAWollman
--
Garrett A. Wollman | O Siem / We are all family / O Siem / We're all the same
wol...@lcs.mit.edu | O Siem / The fires of freedom
Opinions not those of| Dance in the burning flame
MIT, LCS, CRS, or NSA| - Susan Aglukark and Chad Irschick
Brett Glass
I've looked it over on my own, however, and think that Theo has some
good points. In particular, the notion that one should be able to
detect and react to unexpected truncation of a string is a good one. I'd
like to see these functions in FreeBSD's standard libraries, and to see
them used in FreeBSD's kernel and userland.
--Brett
At 10:47 AM 7/15/99 -0600, Paul Hart wrote:
>I was just reviewing the proceedings from the USENIX 1999 Annual Technical
>Conference where Todd Miller and Theo de Raadt presented a paper on two
>new functions that OpenBSD has integrated into libc. The new functions,
>strlcpy(3) and strlcat(3), are intended to provide an easily understood
>means of safe string copying and concatenation to programmers. Of course,
>strcpy(3) and strcat(3) have obvious dangers, but their standardized
>intended replacements, strncpy(3) and strncat(3), suffer from some subtle
>dangers as well that can trip up even experienced programmers.
>
>I was impressed by the paper and wondered if anyone besides myself would
>be amenable to including them in FreeBSD's libc. Are there members of the
>FreeBSD core and community that would be interested in importing these new
>functions? The semantics of strncpy(3) and strncat(3) have struck me as
>warts on the C standard for some time. I'm not sure what debate took
>place on the standardization committee, but whatever it was seems to have
>produced some strange results.
>
>If you are a USENIX member you can access the text of the paper at:
>
> http://www.usenix.org/events/usenix99/millert.html
>
>Paul Hart
>
>--
>Paul Robert Hart ><8> ><8> ><8> Verio Web Hosting, Inc.
>ha...@iserver.com ><8> ><8> ><8> http://www.iserver.com/
>
>
>
Brett Glass
Paper: http://www.openbsd.org/papers/strlcpy-paper.ps
Slides (worth looking at too): http://www.openbsd.org/papers/strlcpy-slides.ps
The USENIX site (mentioned in an earlier message) denies access to the full text of the paper unless you're a member. But the URLs I've listed above are available to anyone with a PostScript viewer. Again, I think the paper makes some great points, and can't understand why Mike was so negative about the paper or the adoption of these functions. They appear to be both faster AND safer than what's commonly used now.
--Brett
Warner Losh
them in my local library for a long time, but haven't had the time to
commit them.
Solaris is also adding them.
Warner
Warner Losh
: What I wanted to do was have "estr" routines, where the destination
: is specified as the starting point and the ending point of the area
: available for the string (as two parameters). The routines would
: return the position of the current string-terminator. So you could
: do things like:
: eos = estrcpy(dest, endp, src1);
: eos = estrcat(eos, endp, src2);
: eos = estrcat(eos, endp, src3);
: and you could check for "string is full" by comparing 'eos' (the
: return value) to 'endp' (which you'd already have). Strictly
: speaking that won't work quite right in some cases, so the strlcpy
: routines also have an advantage there.
Yes. It would take a lot of talking and convincing to get me to
support adding something other than strl* routines.
Matthew Dillon
:I *STRONGLY* support adding strl routines to FreeBSD's libc. I've had
:them in my local library for a long time, but haven't had the time to
:commit them.
:
:Solaris is also adding them.
:
:Warner
I'd say go for it. I would sure use them!
-Matt
Matthew Dillon
<dil...@backplane.com>
Brett Glass
This would allow the programmer to wrap an "if" right around the function
call and handle the error easily if the string was truncated. Making a
check convenient would encourage programmers to insert it into their code.
Having to write a separate test would actually discourage this practice
and could lead to malfunctioning code.
--Brett
At 01:38 AM 7/16/99 +0200, Sheldon Hearn wrote:
>On Thu, 15 Jul 1999 17:19:05 CST, Warner Losh wrote:
>
> > I *STRONGLY* support adding strl routines to FreeBSD's libc. I've had
> > them in my local library for a long time, but haven't had the time to
> > commit them.
>
>What do you think of this?
>
>"
>size_t
>strlcpy(char *dst, char *src, size_t len [, shortfall]);
>
>size_t
>strlcat(char *dst, char *src, size_t len [, shortfall]);
>
>[...]
>
>RETURN VALUES
>
>If the optional shortfall argument is passed non-zero, the functions
>return the number of characters from src that are missing in dst after
>the operation. Otherwise, they return the length of dst. In either case,
>the return value does not include the NUL terminator.
>"
>
>This way, we get compatibility with the other vendors who've chosen to
>implement the functions, but we also get the cheaper option Tim wants.
>It'd be up to the other vendors to choose to implement the extension.
>
>I'll come up with a commit candidate in the next 48 hours and post a
>URL, including a manpage replacement. The OpenBSD manpage for these
>functions includes in DESCRIPTION too much that should be in HISTORY
>(and perhaps IMPLEMENTATION NOTES).
>
>The only thing I can think of that would make this extension a bad idea
>is va_alist processing cost. Is it significant?
>
>Ciao,
>Sheldon.
Brett Glass
of the argument. This is an invitation to confusion.
A more consistent way to do it would be to have the function return zero
if the programmer opted not to know about any shortfall.
Or, even better, ALWAYS return the shortfall. The programmer can then discard
the return value if he's really willing to ignore it (perhaps at his peril).
--Brett
At 01:52 AM 7/16/99 +0200, Sheldon Hearn wrote:
>On Thu, 15 Jul 1999 17:47:03 CST, Brett Glass wrote:
>
> > How about returning the shortfall as the return value of the function?
>
>Um, hello? That's exactly what I'm proposing:
>
> > >RETURN VALUES
> > >
> > >If the optional shortfall argument is passed non-zero, the functions
> > >return the number of characters from src that are missing in dst after
> > >the operation. Otherwise, they return the length of dst. In either case,
> > >the return value does not include the NUL terminator.
>
>:-)
Warner Losh
: If the optional shortfall argument is passed non-zero, the functions
Impossible to implement, gratuitously incompatible with OpenBSD. I'd
say no.
Warner
Warner Losh
: Or, even better, ALWAYS return the shortfall. The programmer can then discard
: the return value if he's really willing to ignore it (perhaps at his peril).
That's what strl* are defined to do. They always return the length of
the string that would have resulted, had it not been truncated. That
way it can either be used or ignored as the programmer sees fit. I
don't see much value in computing return-value - size as another,
incompatible argument.
Garance A Drosihn
>On Thu, 15 Jul 1999 17:19:05 CST, Warner Losh wrote:
>
>> I *STRONGLY* support adding strl routines to FreeBSD's libc.
>> I've had them in my local library for a long time, but haven't
>> had the time to commit them.
>
> What do you think of this?
>
>"
>size_t
>strlcpy(char *dst, char *src, size_t len [, shortfall]);
>
>size_t
>strlcat(char *dst, char *src, size_t len [, shortfall]);
>
>[...]
I like the idea of strl* routines, but I think we should just
add them the way they'll be implemented everywhere else. If
everyone else (openbsd, solaris, etc) likes the above idea
then I'm certainly all for it. If they don't, then I see no
benefit in making slightly different routines for freebsd.
If I'm writing code, I'm writing it for more than one platform.
---
Garance Alistair Drosehn = g...@eclipse.acs.rpi.edu
Senior Systems Programmer or dro...@rpi.edu
Rensselaer Polytechnic Institute
Warner Losh
: Reality check: we're talking about portability here. If we take these
: functions into our own libc, we really should make them work as expected
: on other platforms. However, there's nothing to stop us extending them
: beyond those expectations.
No. They will be 100% compatible, or not imported. No extensions
that could bite people when porting *TO* OpenBSD.
Warner
Jordan K. Hubbard
> hoping that an implementation of an extension will be backported to the
> platform of origin, I need to contact the vendor of that platform.
Naw, I think this is REALLY SIMPLE and very definitely unworth the
length of this thread so far. Not that "proper defensive coding" is
an unworthy topic in its own right, and perhaps we should all have
that discussion again some time, but it's not immediately germin to
the issue of whether or not to bring the OpenBSD strlcpy()/strlcat()
routines in.
I'd say that's a decision made on simple compatibility grounds and if
X or more sister OSen have already adopted them into their libc, where
X is a positive integer greater than or equal to 2, then it's a
foregone conclusion that apps writers will begin using them and we
should provide compatibility. Unvarnished, uncomplicated, just a 100%
compatible implementation with the standard that OpenBSD has already
set.
Now if other security mavens have their own ideas about a much better
family of additions to libc which promote even better and secure
programming practices, by all means knock yourselves out in producing
this family of routines and maybe even someday the man pages for these
first OpenBSD efforts can bear the "SUPERSEDED BY" tag. :-) Until
then, let's kindly not debate this particular bike shelter into the
ground.
- Jordan
Brett Glass
>That'd break code that doesn't expect to have to pass the additional
>argument that we've opted to allow for.
Well, at this point, it's early enough to specify the function so that
no such code is written.
> > Or, even better, ALWAYS return the shortfall. The programmer can then
> > discard the return value if he's really willing to ignore it (perhaps
> > at his peril).
>
>Reality check: we're talking about portability here.
That's why this is the time to set the standard sensibly -- before Solaris
and other standard bearers have decided one way or the other.
>If we take these
>functions into our own libc, we really should make them work as expected
>on other platforms.
Right now, there's only one other platform doing it: OpenBSD. And it's
still early enough to make OpenBSD consistent if there are strong arguments
for the approach that is taken.
>What I'm getting at here is that, while the strl* functions may be nice
>(and Mike Smith's arguments are casting some serious doubt over that
>idea)
What arguments? (They must be on some list to which I'm not subscribed.)
However, based on what Mike said to be at Def Con, I'm concerned that
Mike's arguments might arise from a personality conflict with Theo.
>they could certainly be nicer. At least two other vendors already
>have a defined API for the functions. If we use them, we shouldn't break
>that API. What I propose doesn't, put it does allow for more convenient
>use of the functions.
Alas, the approach you've taken makes the functions slower (You have to check an
argument that's going to be a constant in all cases -- why not just provide two
separate functions?), bigger (you have to check the number of arguments AND the
value of the extra one), and semantically confusing (the meaning of the return
value is influenced both by the presence and the value of an argument). I would
not take the approach you've mentioned for this reason.
--Brett
Brett Glass
the return value. Why? Because it facilitates a conditional test for
truncation. If the return value is the number of bytes copied, one must do
additional arithmetic to test for an error.
Since OpenBSD is the only platform currently integrating the functions,
there's time to work with them to make this the standard if we'd like.
--Brett
At 06:16 PM 7/15/99 -0600, Warner Losh wrote:
>In message <4.2.0.58.19990715180119.04723d20@localhost> Brett Glass writes:
>: Or, even better, ALWAYS return the shortfall. The programmer can then discard
>: the return value if he's really willing to ignore it (perhaps at his peril).
>
>That's what strl* are defined to do. They always return the length of
>the string that would have resulted, had it not been truncated. That
>way it can either be used or ignored as the programmer sees fit. I
>don't see much value in computing return-value - size as another,
>incompatible argument.
>
>Warner
To Unsubscribe: send mail to majo...@FreeBSD.org
Warner Losh
: Since OpenBSD is the only platform currently integrating the functions,
: there's time to work with them to make this the standard if we'd like.
Too late. Linux (glibc I'm told) and Solaris 7 (or is that Solaris 8)
implements that. It is too late to change. So arguing about it is
useless.
Brett Glass
--Brett
Warner Losh
: Darn. In that case, I'll just have to create macros that do it right. ;-)
Sure would be easier than arguing with Theo :-)
#define BrettStrlcat(a,b,c) ((c) - strlcat((a),(b),(c)))
Brett Glass
>Sure would be easier than arguing with Theo :-)
Arguing with Theo is kinda fun, actually. We had good discussions about
gun control and health care during dinner at Def Con. (Of course, Theo
thought that the Canadian approaches to both were absolutely The Right
Thing. ;-)
--Brett
Sheldon Hearn
On Thu, 15 Jul 1999 17:19:05 CST, Warner Losh wrote:
> I *STRONGLY* support adding strl routines to FreeBSD's libc. I've had
> them in my local library for a long time, but haven't had the time to
> commit them.
What do you think of this?
"
size_t
strlcpy(char *dst, char *src, size_t len [, shortfall]);
size_t
strlcat(char *dst, char *src, size_t len [, shortfall]);
[...]
RETURN VALUES
If the optional shortfall argument is passed non-zero, the functions
return the number of characters from src that are missing in dst after
the operation. Otherwise, they return the length of dst. In either case,
the return value does not include the NUL terminator.
"
This way, we get compatibility with the other vendors who've chosen to
implement the functions, but we also get the cheaper option Tim wants.
It'd be up to the other vendors to choose to implement the extension.
I'll come up with a commit candidate in the next 48 hours and post a
URL, including a manpage replacement. The OpenBSD manpage for these
functions includes in DESCRIPTION too much that should be in HISTORY
(and perhaps IMPLEMENTATION NOTES).
The only thing I can think of that would make this extension a bad idea
is va_alist processing cost. Is it significant?
Ciao,
Sheldon.
Sheldon Hearn
On Thu, 15 Jul 1999 17:47:03 CST, Brett Glass wrote:
> How about returning the shortfall as the return value of the function?
Um, hello? That's exactly what I'm proposing:
> >RETURN VALUES
> >
> >If the optional shortfall argument is passed non-zero, the functions
> >return the number of characters from src that are missing in dst after
> >the operation. Otherwise, they return the length of dst. In either case,
> >the return value does not include the NUL terminator.
:-)
Sheldon Hearn
On Thu, 15 Jul 1999 18:05:06 CST, Brett Glass wrote:
> A more consistent way to do it would be to have the function return zero
> if the programmer opted not to know about any shortfall.
That'd break code that doesn't expect to have to pass the additional
argument that we've opted to allow for.
> Or, even better, ALWAYS return the shortfall. The programmer can then
> discard the return value if he's really willing to ignore it (perhaps
> at his peril).
Reality check: we're talking about portability here. If we take these
functions into our own libc, we really should make them work as expected
on other platforms. However, there's nothing to stop us extending them
beyond those expectations.
What I'm getting at here is that, while the strl* functions may be nice
(and Mike Smith's arguments are casting some serious doubt over that
idea) they could certainly be nicer. At least two other vendors already
have a defined API for the functions. If we use them, we shouldn't break
that API. What I propose doesn't, put it does allow for more convenient
use of the functions.
Ciao,
Sheldon Hearn
On Thu, 15 Jul 1999 18:17:45 CST, Warner Losh wrote:
> No. They will be 100% compatible, or not imported. No extensions
> that could bite people when porting *TO* OpenBSD.
Ah yes. The gotcha. :-)
Cool, so now I face that icky sticky problem of "who first". Rather than
hoping that an implementation of an extension will be backported to the
platform of origin, I need to contact the vendor of that platform.
I can see where that'll get me, but it's worth a shot.
Nonetheless, your call is definitely the decisive blow to the idea. I
shoulda thought of it myself. :-)
Johan Danielsson
> I was impressed by the paper and wondered if anyone besides myself
> would be amenable to including them in FreeBSD's libc.
I must admit that I didn't read the paper that thoroughly , but I was
at the presentation. The only thing they missed was `prior art'. We
did implemented the exact same functionality (but with other names)
for our Krb4 distribution, and I beleive that those functions got
imported into OpenBSD before they added strl* to libc. :-)
/Johan
Wes Peters
>
> How about returning the shortfall as the return value of the function?
>
> call and handle the error easily if the string was truncated. Making a
> check convenient would encourage programmers to insert it into their code.
> Having to write a separate test would actually discourage this practice
> and could lead to malfunctioning code.
A good idea, but it's already provided. As pointed out on Slide 9, if
(strlcat(..., size) >= size) an overflow occured and should be handled.
I agree with Mike that for future development or audits of existing code,
moving away from static buffers is THE way to make the codebase less
fragile. strl* does seem to have some compelling features for fixing
existing code when a complete audit is either not warranted or just not
feasible given the available "headcount." Relatively inexperienced
programmers could be given a set of rules for replacing strcat and strcpy
with strlcat and strlcpy to improve, if not perfect, many programs quite
quickly.
--
"Where am I, and what am I doing in this handbasket?"
Wes Peters Softweyr LLC
http://softweyr.com/ w...@softweyr.com
John J. Rushford Jr.
to the libalias library and NATD. The latest feature added that I
use is pptpalias. This allows you to have an NT server behind a
FreeBSD firewall that MS Windows clients may login to with pptp from the
internet. pptpalias first appears in FreeBSD 3.2.
Modred
[snip]
> thought that the Canadian approaches to both were absolutely The Right
> Thing. ;-)
*bow chikka bow wow*
'continuing a thread that's about as relevant to -security
as 70's porno funk...'
*chikka chikka bow wow*