KKIS.05051999.003b
Don Lewis
} Subject: Re: KKIS.05051999.003b
} > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~[ Informations ]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
} > Report title : Security problem with sockets in FreeBSD's
} > implementation of UNIX-domain protocol family.
} > Problem found by : Lukasz Luzar (llu...@security.kki.pl)
} > Report created by : Robert Pajak (sha...@security.kki.pl)
} > Lukasz Luzar (llu...@security.kki.pl)
} > Raport published : 5th May 1999
} > Raport code : KKIS.05051999.003.b
} > Systems affected : FreeBSD-3.0 and maybe 3.1,
} > Archive : http://www.security.kki.pl/advisories/
} > Risk level : high
} >
} > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~[ Description ]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
} > As you know, "The UNIX-domain protocol family is a collection of protocols
} > that provides local interprocess communication through the normal socket
} > mechanism. It supports the SOCK_STREAM and SOCK_DGRAM soceket types and uses
} > filesystem pathnames for addressing."
} > The SOCK_STREAM sockets also supports the communication of UNIX file
} > descriptors through the use of functions sendmsg() and recvmsg().
} > While testing UNIX-domain protocols, we have found probable bug in
} > FreeBSD's implementation of this mechanism.
} > When we had run attached example on FreeBSD-3.0 as local user, system
} > had crashed imediatelly with error "Supervisor read, page not present"
} > in kernel mode.
} >
}
} Here's my testing so far:
}
} 2.2.2 - Vulnerable
} 2.2.6 - Vulnerable
} 2.2.8 - Vulnerable
} 3.1-RELEASE - Ran 15 minutes, no crash.
I'd be willing to bet that 3.0-RELEASE is also vulnerable. I believe
Matt Dillon fixed this earlier this year in revisions 1.38/1.39 (-CURRENT
branch January 21, 1999) and 1.37.2.1 (RELENG_3 branch February 15, 1999) of
sys/kern/uipc-usrreq.c. The RELENG_3 branch fix was committed just before
3.1-RELEASE.
To Unsubscribe: send mail to majo...@FreeBSD.org
with "unsubscribe freebsd-security" in the body of the message
Wes Peters
>
> On May 6, 2:10pm, Kevin Day wrote:
> }
> } Here's my testing so far:
> }
> } 2.2.2 - Vulnerable
> } 2.2.6 - Vulnerable
> } 2.2.8 - Vulnerable
> } 3.1-RELEASE - Ran 15 minutes, no crash.
Let it keep running. It will (apparently) eventually exhaust all
available file handles in an unrecoverable manner. 3.1-R is better,
but not invulnerable.
--
"Where am I, and what am I doing in this handbasket?"
Wes Peters Softweyr LLC
http://www.softweyr.com/~softweyr w...@softweyr.com
Don Lewis
} Subject: Re: KKIS.05051999.003b
} Don Lewis wrote:
} >
} > On May 6, 2:10pm, Kevin Day wrote:
} > }
} > } Here's my testing so far:
} > }
} > } 2.2.2 - Vulnerable
} > } 2.2.6 - Vulnerable
} > } 2.2.8 - Vulnerable
} > } 3.1-RELEASE - Ran 15 minutes, no crash.
}
} Let it keep running. It will (apparently) eventually exhaust all
} available file handles in an unrecoverable manner. 3.1-R is better,
} but not invulnerable.
I don't see any obvious descriptor leaks, but the fact that FreeBSD < 3.1
panics (probably in unp_gc(), which Matt fixed) indicates that I'm missing
something. The exploit code should not result in any calls to unp_gc(),
because the client receives all the descriptors that are sent by the server.
This should result in unp_rights being 0 except when the descriptor is
in flight. If unp_rights is 0 when the socket is closed, unp_gc() should not
be called. unp_gc() should only be called if the client closes socket before
receiving the descriptor.
Maybe a third process occasionally get scheduled while the exploit code
has the descriptor in flight and causes unp_gc() to get executed. If so,
then the exploit shouldn't cause a problem in single user mode.
Wes Peters
>
> On May 7, 11:34pm, Wes Peters wrote:
> } Subject: Re: KKIS.05051999.003b
> } Don Lewis wrote:
> } >
> } > On May 6, 2:10pm, Kevin Day wrote:
> } > }
> } > } Here's my testing so far:
> } > }
> } > } 2.2.2 - Vulnerable
> } > } 2.2.6 - Vulnerable
> } > } 2.2.8 - Vulnerable
> } > } 3.1-RELEASE - Ran 15 minutes, no crash.
> }
> } Let it keep running. It will (apparently) eventually exhaust all
> } available file handles in an unrecoverable manner. 3.1-R is better,
> } but not invulnerable.
>
> I don't see any obvious descriptor leaks, but the fact that FreeBSD < 3.1
> panics (probably in unp_gc(), which Matt fixed) indicates that I'm missing
> something. The exploit code should not result in any calls to unp_gc(),
> because the client receives all the descriptors that are sent by the server.
Actually it doesn't. If you look up the first message I posted on this
subject, I listed the error messages it produces, many of which indicated
the client didn't get a descriptor from the server IIRC. Maybe that's
how the descriptors are being lost; they've been sent on a UNIX domain
socket and so have to remain open, have been closed by the server, working
around it's limits, and have not been read by the client?
> This should result in unp_rights being 0 except when the descriptor is
> in flight. If unp_rights is 0 when the socket is closed, unp_gc() should not
> be called. unp_gc() should only be called if the client closes socket before
> receiving the descriptor.
>
> Maybe a third process occasionally get scheduled while the exploit code
> has the descriptor in flight and causes unp_gc() to get executed. If so,
> then the exploit shouldn't cause a problem in single user mode.
I haven't had time to research this any further, I spent today chasing
a VERY engergetic toddler. I'm too old for this. ;^)
--
"Where am I, and what am I doing in this handbasket?"
Wes Peters Softweyr LLC
http://www.softweyr.com/~softweyr w...@softweyr.com
sth...@nethelp.no
> has the descriptor in flight and causes unp_gc() to get executed. If so,
> then the exploit shouldn't cause a problem in single user mode.
It happens in single user mode too.
In general, this program leaks one file descriptor for each time round
the client/server loops - this is easy to see if you add some debugging
printout to falloc()/ffree() in sys/kern/kern_descrip.c.
If you parametrize the client loop, ie.
case 0:
for (n=0;n<rounds;n++)
client();
you'll find that it leaks N-1 file descriptors if the client loop is run
N times and the program is aborted with ^C. It's eminently reproducible.
Other interesting points about this program:
- The client shouldn't receive anything at all, because it's listening
on a different socket (using PATH_TMP) than the server (using PATH) is
sending on.
- If you remove the following part of the client() routine:
if ( sendto( sockfd,&data,sizeof( data),0,(struct sockaddr *) &addr_s,
addr_s.sun_len) == -1)
printf( "client: sendto error %d\n",errno);
there is no longer any leak.
- The client is asking for messages with zero iov's, and length 0. To
me, this means it shouldn't receive *anything* (file descriptors or
otherwise). But the program included below, slightly modified from the
client() routine, receives one message of length zero. The same thing
happens on for instance NetBSD 1.4-BETA or NetBSD 1.3.2. Does this mean
the semantics of receiving zero length messages aren't sufficiently
well defined?
Steinar Haug, Nethelp consulting, sth...@nethelp.no
----------------------------------------------------------------------
#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/un.h>
#define PATH_TMP "/tmp/123.tmp"
main()
{
struct sockaddr_un addr_c;
struct msghdr mymsghdr;
int sockfd, l;
if ((sockfd = socket(AF_UNIX, SOCK_DGRAM, 0)) == -1)
perror("socket");
strcpy(addr_c.sun_path, PATH_TMP);
addr_c.sun_len = sizeof(addr_c.sun_len) + sizeof(addr_c.sun_family)
+ strlen(addr_c.sun_path) + 1;
addr_c.sun_family = AF_UNIX;
if (bind(sockfd, (struct sockaddr*) &addr_c, addr_c.sun_len) == -1)
perror("bind");
mymsghdr.msg_name = NULL;
mymsghdr.msg_namelen = 0;
mymsghdr.msg_iov = NULL;
mymsghdr.msg_iovlen = 0;
mymsghdr.msg_control = NULL;
mymsghdr.msg_controllen = 0;
mymsghdr.msg_flags = 0;
if ( (l = recvmsg(sockfd, &mymsghdr, 0)) == -1)
perror("recvmsg");
else
printf("client: received len %d\n", l);
close(sockfd);
if ( unlink( PATH_TMP) == -1)
perror("unlink");
sth...@nethelp.no
> panics (probably in unp_gc(), which Matt fixed) indicates that I'm missing
> something.
A 2.2.8 system I have here panics in sorflush (called from unp_gc()):
void
sorflush(so)
register struct socket *so;
{
register struct sockbuf *sb = &so->so_rcv;
register struct protosw *pr = so->so_proto;
register int s;
struct sockbuf asb;
sb->sb_flags |= SB_NOINTR;
(void) sblock(sb, M_WAITOK);
s = splimp();
socantrcvmore(so);
sbunlock(sb); asb = *sb;
bzero((caddr_t)sb, sizeof (*sb));
splx(s);
if (pr->pr_flags & PR_RIGHTS && pr->pr_domain->dom_dispose)
(*pr->pr_domain->dom_dispose)(asb.sb_mb);
sbrelease(&asb);
}
because so->so_proto is 0. Backtrace:
#0 boot (howto=256) at ../../kern/kern_shutdown.c:275
#1 0xf01128ba in panic (fmt=0xf01bdf0f "page fault") at ../../kern/kern_shutdown.c:409
#2 0xf01beafa in trap_fatal (frame=0xefbffde4) at ../../i386/i386/trap.c:772
#3 0xf01be5bc in trap_pfault (frame=0xefbffde4, usermode=0) at ../../i386/i386/trap.c:681
#4 0xf01be247 in trap (frame={tf_es = 16, tf_ds = 16, tf_edi = -272630184, tf_esi = -260321820,
tf_ebp = -272630184, tf_isp = -272630260, tf_ebx = -260321856, tf_edx = 1073610751, tf_ecx = 0,
tf_eax = 0, tf_trapno = 12, tf_err = 0, tf_eip = -267232200, tf_cs = 8, tf_eflags = 66118,
tf_esp = 0, tf_ss = -259461120}) at ../../i386/i386/trap.c:324
#5 0xf0125c38 in sorflush (so=0xf07bcd80) at ../../kern/uipc_socket.c:854
#6 0xf01297de in unp_gc () at ../../kern/uipc_usrreq.c:889
#7 0xf012908f in unp_detach (unp=0xf0548694) at ../../kern/uipc_usrreq.c:420
#8 0xf0128b42 in uipc_usrreq (so=0xf0890f00, req=1, m=0x0, nam=0x0, control=0x0)
at ../../kern/uipc_usrreq.c:113
#9 0xf012720f in old_detach (so=0xf0890f00) at ../../kern/uipc_socket2.c:890
#10 0xf0124902 in soclose (so=0xf0890f00) at ../../kern/uipc_socket.c:209
#11 0xf011c607 in soo_close (fp=0xf0906540, p=0xf07d8800) at ../../kern/sys_socket.c:206
#12 0xf010b1bc in closef (fp=0xf0906540, p=0xf07d8800) at ../../kern/kern_descrip.c:896
#13 0xf010a8a9 in close (p=0xf07d8800, uap=0xefbfff94, retval=0xefbfff84) at ../../kern/kern_descrip.c:392
#14 0xf01bed93 in syscall (frame={tf_es = 39, tf_ds = 39, tf_edi = 0, tf_esi = -272638692,
tf_ebp = -272638740, tf_isp = -272629788, tf_ebx = -272638688, tf_edx = -272638846,
tf_ecx = -272638972, tf_eax = 6, tf_trapno = 7, tf_err = 7, tf_eip = 537330913, tf_cs = 31,
tf_eflags = 646, tf_esp = -272639024, tf_ss = 39}) at ../../i386/i386/trap.c:920
#15 0x200704e1 in ?? ()
#16 0x163d in ?? ()
#17 0x1095 in ?? ()
Steinar Haug, Nethelp consulting, sth...@nethelp.no
Don Lewis
} Subject: Re: KKIS.05051999.003b
} Don Lewis wrote:
} > I don't see any obvious descriptor leaks, but the fact that FreeBSD < 3.1
} > panics (probably in unp_gc(), which Matt fixed) indicates that I'm missing
} > something. The exploit code should not result in any calls to unp_gc(),
} > because the client receives all the descriptors that are sent by the server.
}
} Actually it doesn't. If you look up the first message I posted on this
} subject, I listed the error messages it produces, many of which indicated
} the client didn't get a descriptor from the server IIRC. Maybe that's
} how the descriptors are being lost; they've been sent on a UNIX domain
} socket and so have to remain open, have been closed by the server, working
} around it's limits, and have not been read by the client?
I think all those messages are the result of running out of descriptors.
If no more descriptors can be created, then socket will fail with ENFILE (23),
and bind(), sendto(), and recvmsg() will fail with EBADF (9), and unlink()
will fail with ENOENT (2). I'd have to study the code some more to work
out the expected sequence of the errors assuming that no more descriptors
could be created.
The message containing the descriptor to be passed holds a reference to
the descriptor, so it is permissible for the server to close the descriptor
before the client receives the message. See unp_internalize() and
unp_externalize(). BTW, how does this manage to work on the Alpha,
where sizeof(int) != sizeof(pointer)?
Don Lewis
} Subject: Re: KKIS.05051999.003b
} > I don't see any obvious descriptor leaks, but the fact that FreeBSD < 3.1
} > panics (probably in unp_gc(), which Matt fixed) indicates that I'm missing
} > something.
}
This is the bug that Matt Dillon fixed just before 3.1-RELEASE, sorflush()
is only supposed to be used on descriptors associated with sockets. This
exploit code causes sorflush() to be called on a descriptor for an open
file.
Don Lewis
} > has the descriptor in flight and causes unp_gc() to get executed. If so,
} > then the exploit shouldn't cause a problem in single user mode.
}
} It happens in single user mode too.
}
} In general, this program leaks one file descriptor for each time round
} the client/server loops - this is easy to see if you add some debugging
} printout to falloc()/ffree() in sys/kern/kern_descrip.c.
}
} If you parametrize the client loop, ie.
}
} case 0:
} for (n=0;n<rounds;n++)
} client();
}
} you'll find that it leaks N-1 file descriptors if the client loop is run
} N times and the program is aborted with ^C. It's eminently reproducible.
Hmn ...
} Other interesting points about this program:
}
} - The client shouldn't receive anything at all, because it's listening
} on a different socket (using PATH_TMP) than the server (using PATH) is
} sending on.
The client starts by doing a sendto() to send a message to the server
at address PATH. The server receives this message using recvfrom(),
which returns the source of the message (PATHTMP) in addr. When the
server uses sendmsg() to return its response (with the embedded descriptor)
to the client, it sends this message to the address that it received in
the recvfrom().
} - If you remove the following part of the client() routine:
}
} if ( sendto( sockfd,&data,sizeof( data),0,(struct sockaddr *) &addr_s,
} addr_s.sun_len) == -1)
} printf( "client: sendto error %d\n",errno);
}
} there is no longer any leak.
Right, because this is what prompts the server to wake up and send
each descriptor to the client. Without this sendto(), the server waits
forever in recvfrom(), and the client falls through to its recvmsg()
call and hangs.
} - The client is asking for messages with zero iov's, and length 0. To
} me, this means it shouldn't receive *anything* (file descriptors or
} otherwise). But the program included below, slightly modified from the
} client() routine, receives one message of length zero. The same thing
} happens on for instance NetBSD 1.4-BETA or NetBSD 1.3.2. Does this mean
} the semantics of receiving zero length messages aren't sufficiently
} well defined?
I believe the length refers to the length of any data that might
accompany the descriptors. It should be OK to specify a length of 0.
Even if the server was sending data in its reply, I believe it would
not be an error to specify a zero length buffer. The data would just
be truncated to fit the buffer size.
sth...@nethelp.no
> } me, this means it shouldn't receive *anything* (file descriptors or
> } otherwise). But the program included below, slightly modified from the
> } client() routine, receives one message of length zero. The same thing
> } happens on for instance NetBSD 1.4-BETA or NetBSD 1.3.2. Does this mean
> } the semantics of receiving zero length messages aren't sufficiently
> } well defined?
>
> I believe the length refers to the length of any data that might
> accompany the descriptors. It should be OK to specify a length of 0.
> Even if the server was sending data in its reply, I believe it would
> not be an error to specify a zero length buffer. The data would just
> be truncated to fit the buffer size.
Okay, but why should the *standalone* version of the client receive any
message at all (which it does: a zero length message) when there's no
sender involved at all?
Steinar Haug, Nethelp consulting, sth...@nethelp.no
Don Lewis
} > } me, this means it shouldn't receive *anything* (file descriptors or
} > } otherwise). But the program included below, slightly modified from the
} > } client() routine, receives one message of length zero. The same thing
} > } happens on for instance NetBSD 1.4-BETA or NetBSD 1.3.2. Does this mean
} > } the semantics of receiving zero length messages aren't sufficiently
} > } well defined?
} >
} > I believe the length refers to the length of any data that might
} > accompany the descriptors. It should be OK to specify a length of 0.
} > Even if the server was sending data in its reply, I believe it would
} > not be an error to specify a zero length buffer. The data would just
} > be truncated to fit the buffer size.
}
} Okay, but why should the *standalone* version of the client receive any
} message at all (which it does: a zero length message) when there's no
} sender involved at all?
Darned if I know. Maybe it is because neither a data buffer or a control
buffer is specified in your modified version. If you modify your example
code to loop on recvmsg(), you'll find that recvmsg() will return 0 as
many times as you want.
FLASH! Now this is really wierd. The original exploit code doesn't show
any signs leaking descriptors on one of our 3.1-stable machines, but
/tmp (where the sockets are created) is mfs. If I change PATH and
PATH_TMP so that they point to /var/tmp, sendmsg() fails with with
ECONNREFUSED after the first iteration and descriptors are leaked.
I might believe that descriptors could be leaked if sendmsg() fails this
way, but why would sendmsg() fail if the sockets live in a ufs filesystem
but not if the sockets live in a mfs filesystem?
Don Lewis
} Subject: Re: KKIS.05051999.003b
} FLASH! Now this is really wierd. The original exploit code doesn't show
} any signs leaking descriptors on one of our 3.1-stable machines, but
} /tmp (where the sockets are created) is mfs. If I change PATH and
} PATH_TMP so that they point to /var/tmp, sendmsg() fails with with
} ECONNREFUSED after the first iteration and descriptors are leaked.
}
} I might believe that descriptors could be leaked if sendmsg() fails this
} way, but why would sendmsg() fail if the sockets live in a ufs filesystem
} but not if the sockets live in a mfs filesystem?
Ok, the first bug is in uipc_send(). If an error occurs between
unp_internalize() and release:, then the work that unp_internalize()
did needs to be undone. Probably something like:
if (control && error != 0)
unp_dispose(control);
right before release: would do the trick.
I haven't a clue about what appears to be bug number 2.
Don Lewis
} Subject: Re: KKIS.05051999.003b
} Okay, but why should the *standalone* version of the client receive any
} message at all (which it does: a zero length message) when there's no
} sender involved at all?
I think this is part of bug number 2. What appears to be happening is
that the client does sendto() and then does recvmsg() which returns and
then the client continues on to do the next unlink() before the server
wakes up and does its sendmsg(). When the server finally gets around
to executing sendmsg(), the client has already unlinked the socket that
the server is trying to send its reply to, causing the sendmsg() to
fail and leak the descriptor that is being passed.
The UFS vs MFS effect that I observed apparently affects the timing.
If I add a sleep() call in the client before the recvmsg(), everything
works as it should.
Don Lewis
} Subject: Re: KKIS.05051999.003b
} } Subject: Re: KKIS.05051999.003b
} } Don Lewis wrote:
}
} } > panics (probably in unp_gc(), which Matt fixed) indicates that I'm missing
} } > because the client receives all the descriptors that are sent by the server.
} }
} } Actually it doesn't. If you look up the first message I posted on this
} } subject, I listed the error messages it produces, many of which indicated
} } the client didn't get a descriptor from the server IIRC.
] On my 3.1-RELEASE system it reports numerous sendmsg ECONNREFUSED and
] ENOENTs, then eventually dives into repeats of:
]
] unlink error 2
] bind error 9
] sento error 9
] recvmsg error 9
] socket error 23
Steinar Haug noticed something odd in the exploit code - the client
process doesn't block in recvmsg() like one would expect. If a message
is waiting, the client process will receive it, but if not, the client
will fall through the recvmsg() without getting an error indication.
This means that the client and server don't run in lockstep. With
the default UFS mount options for /tmp, the client process will block
for short periods of time while it executes bind() and unlink(). If
/tmp is MFS or is mounted async or with softupdates, the client process
will probably only block when its scheduling quantum expires. This oddity
may allow multiple requests to accumulate for the server process, and
allows the server process to attempt to send responses at possibly
inconvenient times for the client.
If the server calls sendmsg() between the client's bind() and
recvmsg() syscalls, then everything works normally.
If the server calls sendmsg() between the client's recvmsg() and
close(sockfd) syscalls, or if the server manages to do more than
one sendmsg() call during one iteration of the client, which might
happen if the client exhausts its scheduling time quantum, one or
more messages will be outstanding when the client executes
close(sockfd). This will result in unp_gc() being called to garbage
collect the descriptors that are in flight. In FreeBSD < 3.1-RELEASE,
this will result in a panic due to sorflush() being improperly called
on a non-socket descriptor.
If the server calls sendmsg() after the client's close(sockfd) and
before the client unlinks the socket, the sendmsg() call will fail
with ECONNREFUSED. If the server calls sendmsg() after the client
unlinks the socket and before the client calls bind(), then the
sendmsg() will fail with ENOENT. Either of these failures will
cause the descriptor being sent to be permanently leaked away.
The rate at which descriptors are leaked depends on how the client
and server processes are scheduled.
Once enough descriptors have been leaked away, the client's socket()
syscall will fail with ENFILE, the bind(), sendto(), and recvmsg()
syscalls will fail with EBADF because the socket was not created,
and unlink() will fail because the socket name was not created by
bind().
The following patch appears to fix the descriptor leak. I hope to
commit it later today.
Index: uipc_usrreq.c
===================================================================
RCS file: /home/ncvs/src/sys/kern/uipc_usrreq.c,v
retrieving revision 1.43
diff -u -u -r1.43 uipc_usrreq.c
--- uipc_usrreq.c 1999/04/28 11:37:07 1.43
+++ uipc_usrreq.c 1999/05/09 23:50:45
@@ -367,6 +367,9 @@
unp_shutdown(unp);
}
+ if (control && error != 0)
+ unp_dispose(control);
+
release:
if (control)
m_freem(control);