Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

'X11 connection uses different authentication protocol'

1,258 views
Skip to first unread message

Alan J. Flavell

unread,
Jan 6, 2000, 3:00:00 AM1/6/00
to

We're getting some problems with a particular commercial application
that's supposed to throw X windows. Due to firewalling, we need to
tunnel the X protocol through ssh version 1 (more details later, but I
suspect they are not relevant).

Ordinary X applications execute just fine, for example xlogo or
(X) Netscape. But this particular application won't work:

- when the session is initiated from ttssh, then on starting up the
application, ttssh puts up an alert box saying "Remote X application
sent incorrect authentication data".

- when the session is initiated from ssh with the -v option, then it
reports (typed-in from my handwritten notes - beware typos):

X11 connection uses different authentication
protocol 'MIT-MAGIC-COOKIE-1' vs. ''

Now, I know that 'MIT-MAGIC-COOKIE-1' is what the ssh X forwarding is
expecting. So presumably '' is what the application is offering(?)

Doesn't it look as if this commercial application is attempting its
own botched authentication somehow?

Any suggestions on how to get around this? There doesn't seem to be a
way of persuading the ssh X forwarding to use any other authentication
scheme(?).


Now here are the "More details" that I promised. In reality the
dedicated host (let's call it AAA) where this commercial package is
running is not offering any ssh service - it's behind the firewall. For
remote access we use ssh to an intermediate machine, say BBB, that's
accessible from outside the firewall by this method, then we connect
locally from BBB into the machine AAA (using rsh, but this is probably
irrelevant, telnet would do the same), we then set DISPLAY variable to
BBB:n.0 for the appropriate value of n. As I remarked, at this point we
have no problems with executing xlogo or other "normal" X applications
on the host AAA, and they happily throw their windows down the ssh
tunnel that exists between the intermediate host BBB and the distant
user's station.

It's just this particular commercial application that's causing grief.
But that was the whole point of making the connection, so it's a real
problem. Its system manager doesn't seem to understand ssh tunnelling,
so we've made no progress on that front. OK, fairynuff, I don't very
well understand ssh tunnelling either, but I'm happily using it in many
other contexts...

Any suggestions for a next move, please?


Per Hedeland

unread,
Jan 7, 2000, 3:00:00 AM1/7/00
to
In article <Pine.HPP.3.95a.100010...@hpplus01.cern.ch>

"Alan J. Flavell" <fla...@mail.cern.ch> writes:
> X11 connection uses different authentication
> protocol 'MIT-MAGIC-COOKIE-1' vs. ''
>
>Now, I know that 'MIT-MAGIC-COOKIE-1' is what the ssh X forwarding is
>expecting. So presumably '' is what the application is offering(?)
>
>Doesn't it look as if this commercial application is attempting its
>own botched authentication somehow?

Probably not, X clients don't get to choose authentication protocol
randomly, they have to use one that is listed for the display in the
$HOME/.Xauthority file (or the file that environment $XAUTHORITY points
to, if present). A more likely reason is that it can't find the display
in the file - not sure why if other clients can, though.

>Any suggestions on how to get around this?

One thing to try would be to use the IP address for the $DISPLAY
variable, I think that's what is in the .Xauthority file - the
application might have some problem with name resolution. Also, if it
runs setuid it probably won't be able to read .Xauthority as it should
be readable only by the owner (giving world read access to it is not a
good idea of course, but could be used for testing at least).

> There doesn't seem to be a
>way of persuading the ssh X forwarding to use any other authentication
>scheme(?).

There shouldn't be any need for this, "everything" can do
MIT-MAGIC-COOKIE-1, and in many/most cases it's the only protocol the X
server does - i.e. without it the app might not be able to talk directly
to the X server either (unless Bad Things like xhost were used).

--Per Hedeland
p...@erix.ericsson.se

Alan J. Flavell

unread,
Jan 7, 2000, 3:00:00 AM1/7/00
to
On 7 Jan 2000, Per Hedeland wrote:

> Probably not, X clients don't get to choose authentication protocol
> randomly,

OK, I discuss this again below

> they have to use one that is listed for the display in the
> $HOME/.Xauthority file (or the file that environment $XAUTHORITY points
> to, if present).

OK, I tried playing with that setting, but it didn't help. I verified
with "xauth -v list" that, at least outside of this wretched
application, the authentication was getting its data from the place that
I thought it was.

I also looked at the provided script that was starting up the binary of
the application - I didn't find anything relevant there either. So if
there is any trickery, it seems it's stashed in the binary somewhere.

> A more likely reason is that it can't find the display
> in the file

Well, I did have a play around with extra xauth entries for the FQDN and
for the IP address, based on the existing unqualified name entry that I
found there. It didn't help though.

> - not sure why if other clients can, though.

Thanks. My thoughts too.

> One thing to try would be to use the IP address for the $DISPLAY
> variable, I think that's what is in the .Xauthority file

Hmm, I played around with that a bit and added some likely extra
entries into .Xauthority, but it didn't seem to make any difference.
I can try playing around there some more, anyhow.

> Also, if it
> runs setuid it probably won't be able to read .Xauthority as it should
> be readable only by the owner

This is an interesting idea. Thanks, I'll take a look at this.

> (giving world read access to it is not a
> good idea of course,

understood!

I suppose it _could_ be that the application was munging its own
$XAUTHORITY setting internally - maybe to some temporary file that it
creates - before it starts opening x windows? If so, that would seem
to contradict your idea that "X clients don't get to choose
authentication protocol", no?


I have to admit this is now no longer urgent, as the user in question
has negotiated an easement in the firewall to allow X windows to pass.
Of course, this leaves him much more vulnerable than if he'd been able
to use the ssh tunnel. But I'm sure the problem will come up again in
our community, so I'm still interested in finding an answer.

thanks a lot, all the best

p.s for CERN-based readers: I'm not talking about CERN's firewall!


Per Hedeland

unread,
Jan 8, 2000, 3:00:00 AM1/8/00
to
In article <Pine.HPP.3.95a.100010...@hpplus01.cern.ch>
"Alan J. Flavell" <fla...@mail.cern.ch> writes:
>On 7 Jan 2000, Per Hedeland wrote:
>> A more likely reason is that it can't find the display
>> in the file
>
>Well, I did have a play around with extra xauth entries for the FQDN and
>for the IP address, based on the existing unqualified name entry that I
>found there. It didn't help though.

Entries for TCP connections (such as those forwarded by ssh) are always
by IP address (I looked it up in xc/lib/Xau/README:-) - if 'xauth list'
gives you unqualified hostnames, it's because that's what your reverse-
resolution gives.

>> One thing to try would be to use the IP address for the $DISPLAY
>> variable, I think that's what is in the .Xauthority file
>
>Hmm, I played around with that a bit and added some likely extra
>entries into .Xauthority, but it didn't seem to make any difference.
>I can try playing around there some more, anyhow.

Adding entries doesn't seem meaningful (and is error-prone), the trick
would be to set $DISPLAY to something that made the app find the entry
that is already there - and the IP address is guaranteed to be. However
on second thought, if the app was unable to map the hostname to an IP
address, it wouldn't be able to make any connection at all, so this is
probably a dead end.

>I suppose it _could_ be that the application was munging its own
>$XAUTHORITY setting internally - maybe to some temporary file that it
>creates - before it starts opening x windows?

The idea occured to me, but it doesn't make any sense - the purpose of
the .Xauthority file is to share info between clients and server, the
client can't share if it uses its own file.:-)

> If so, that would seem
>to contradict your idea that "X clients don't get to choose
>authentication protocol", no?

It's not "my idea", it's the law:-) - RTF Xsecurity M.

>I have to admit this is now no longer urgent, as the user in question
>has negotiated an easement in the firewall to allow X windows to pass.

Hm, using host-based access I presume? If that is acceptable, it can
certainly be done via ssh too - perhaps a bit more secure, and without
needing to open up the firewall. Just do a normal remote forward,
e.g. -R6100:localhost:6000, allow X access for localhost at the local
end, and set (with the suggested forward) DISPLAY to localhost:100 at
the remote end (or sshd-host:100 if there is an extra hop needed).
Tcp-wrappers (if built into sshd) can be used to restrict allowed
connections to the remote port somewhat.

--Per Hedeland
p...@erix.ericsson.se

Per Hedeland

unread,
Jan 8, 2000, 3:00:00 AM1/8/00
to
In article <858gtr$epb$1...@news.du.uab.ericsson.se> p...@erix.ericsson.se

(Per Hedeland) writes:
>Adding entries doesn't seem meaningful (and is error-prone), the trick
>would be to set $DISPLAY to something that made the app find the entry
>that is already there - and the IP address is guaranteed to be. However
>on second thought, if the app was unable to map the hostname to an IP
>address, it wouldn't be able to make any connection at all, so this is
>probably a dead end.

Hm, on third thought, perhaps not - the host running sshd wouldn't
happen to be reachable by multiple IP addresses (multihomed, virtual
interfaces, etc), and looking up the name in different name services
happen to give different addresses in response, I suppose? If that were
the case, it could be an explanation (the problem app could be using a
different name service than the others e.g. due to being linked with
different versions of libraries) - and the simplest fix to set $DISPLAY
using the IP address - but it has to be the *right* IP address...

--Per Hedeland
p...@erix.ericsson.se


Reinier Post

unread,
Jan 10, 2000, 3:00:00 AM1/10/00
to
"Alan J. Flavell" <fla...@mail.cern.ch> wrote:

>- when the session is initiated from ssh with the -v option, then it
>reports (typed-in from my handwritten notes - beware typos):
>

> X11 connection uses different authentication
> protocol 'MIT-MAGIC-COOKIE-1' vs. ''

I've been getting the same error using ssh to connect to a Solaris
host. It turned out sshd was unable to guess the pathname to xauth.

After adding

XAuthLocation /usr/openwin/bin/xauth

to sshd_config, the problem went away. If you can't edit there, use

~/.ssh/rc

--
Reinier Post
TU Eindhoven ('TUE', for short. 'TU/e' is not allowed.)

Andreas Gunnarsson

unread,
Jan 10, 2000, 3:00:00 AM1/10/00
to
In article <858hls$f5h$1...@news.du.uab.ericsson.se>,

Per Hedeland <p...@erix.ericsson.se> wrote:
>and the simplest fix to set $DISPLAY using the IP address - but it has to
>be the *right* IP address...

If it is standard ssh X forwarding, $DISPLAY should point to the machine
itself and sshd should take care of that. Maybe some wrapper around the
application tries to be smart and changes $DISPLAY to the remote host?
Or maybe the application tries to run on a third machine and fails to
set up $DISPLAY and .Xauthority appropriately?

It is also possible that the application doesn't use the standard X
libraries and isn't aware or MIT-MAGIC-COOKIE-1, I've seen that once
and I had to make my own X proxy that forwards unauthenticated X and
adds authentication (yes, that sounds really bad but was acceptable in
this case).

Andreas
--
--
Andreas Gunnarsson - andreas.g...@emw.ericsson.se - +46 31 7476081
Using a metaphor as proof is like selling water and charging for single malt
QED

Alan J. Flavell

unread,
Jan 10, 2000, 3:00:00 AM1/10/00
to
On 10 Jan 2000, Andreas Gunnarsson wrote:

> In article <858hls$f5h$1...@news.du.uab.ericsson.se>,
> Per Hedeland <p...@erix.ericsson.se> wrote:
> >and the simplest fix to set $DISPLAY using the IP address - but it has to
> >be the *right* IP address...
>
> If it is standard ssh X forwarding, $DISPLAY should point to the machine
> itself and sshd should take care of that.

Evidently you haven't been following this thread.

The scenario is an ssh across the Internet to a firewall host (AAA), and
then a non-ssh connection on-site to a host (BBB) behind the firewall.
Host BBB then has to set its display to AAA:n.0 (for appropriate choice
of n).

As it happens, both hosts get access to the user's AFS home filesystem,
so they are both seeing the same ~/.Xauthority file. I stress that
executing something like xlogo on host BBB _is_ successful, and so it
evidenly does throw the X window "across the site and into the ssh
pipe": it's only when a specific application is executed on BBB that
this fails. "Now read on".

> Maybe some wrapper around the
> application tries to be smart and changes $DISPLAY to the remote host?

No, because the rejection is reported by ssh. On the basis of your
theory, ssh would not have been involved.

> It is also possible that the application doesn't use the standard X
> libraries and isn't aware or MIT-MAGIC-COOKIE-1,

Now this seems to me to be one plausible explanation, indeed.

thanks.


Alan J. Flavell

unread,
Jan 10, 2000, 3:00:00 AM1/10/00
to
On 10 Jan 2000, Reinier Post wrote:

> > X11 connection uses different authentication
> > protocol 'MIT-MAGIC-COOKIE-1' vs. ''
>
> I've been getting the same error using ssh to connect to a Solaris
> host. It turned out sshd was unable to guess the pathname to xauth.

Ah yes, I recognise that scenario.

However (please see more-detailed f'up that I posted a moment ago) this
explanation doesn't hold water in relation to the specific problem that
I was tackling. As you can see from the fact that xlogo worked, but the
commercial application didn't. Since the sshd was on a totally
different machine (the firewall relay machine), it couldn't suddenly
lose sight of xauth just because a different application was executed on
the target machine.

> XAuthLocation /usr/openwin/bin/xauth

This will definitely help people who have that specific problem, so it
_was_ useful to post it, I'm not complaining. But as it happens, in
this case it doesn't seem to apply.

all the best


Terje Elde

unread,
Jan 13, 2000, 3:00:00 AM1/13/00
to
In article <Pine.HPP.3.95a.100011...@hpplus01.cern.ch>,

Alan J. Flavell wrote:
>However (please see more-detailed f'up that I posted a moment ago) this
>explanation doesn't hold water in relation to the specific problem that
>I was tackling. As you can see from the fact that xlogo worked, but the
>commercial application didn't. Since the sshd was on a totally
>different machine (the firewall relay machine), it couldn't suddenly
>lose sight of xauth just because a different application was executed on
>the target machine.

Possible solution:

xlogo is doing everything correctly, and connects to the display listen in
the DISPLAY environment variable, while the commercial program is not.
I've had this problem myself, rxvt worked, while for example xv tried to
connect to local:0 of the box I started it on, as it ignored the DISPLAY
variable.

For xv I found the solution was to add the options "-display [box I was
on]:0" tho if you want to use the ssh tunnel you should obviously use
localhost:10.

Check the man page and see how you can force the program to use the
display you want.

Terje Elde
--

Hi! I'm a .signature virus! Copy me into your ~/.signature to help me
spread!


Alan J. Flavell

unread,
Jan 13, 2000, 3:00:00 AM1/13/00
to
On Thu, 13 Jan 2000, Terje Elde wrote:

> Possible solution:

'fraid not

> For xv I found the solution was to add the options "-display [box I was
> on]:0" tho if you want to use the ssh tunnel you should obviously use
> localhost:10.

The commercial app _is_ using the correct display (the ssh-tunneled
display), or at least trying to. It's the ssh-tunneled display that is
complaining about its attempt to use the wrong authentication.

So you've got the right solution to the wrong problem, I'm afraid.

Terje Elde

unread,
Jan 13, 2000, 3:00:00 AM1/13/00
to
In article <Pine.HPP.3.95a.100011...@hpplus01.cern.ch>,
Alan J. Flavell wrote:
>> For xv I found the solution was to add the options "-display [box I was
>> on]:0" tho if you want to use the ssh tunnel you should obviously use
>> localhost:10.
>
>The commercial app _is_ using the correct display (the ssh-tunneled
>display), or at least trying to. It's the ssh-tunneled display that is
>complaining about its attempt to use the wrong authentication.
>
>So you've got the right solution to the wrong problem, I'm afraid.

Then I shall try again =)

But before I do so, how can you be sure it's trying the correct host?

Anyways, I once had a similar problem, tho it's a while ago, so I don't
recall if it was the exact same. Anyways, I maually synced .Xauthority,
and it worked. This should IMHO not fix anything, but it did.

Another thing that COULD work is "xhost localhost", allowing any
connection from localhost without needing authentication (ssh runs on
localhost). Keep in mind the security side of this tho.

Kenneth Herron

unread,
Jan 15, 2000, 3:00:00 AM1/15/00
to
This commercial application wouldn't be HP Openview, would it? Template
administrator, right? :-)

Try this on for size: Your .Xauthority file is NFS-mounted on AAA, and
the commercial application is trying to connect to your display from a
process running as root. Due to the NFS root-is-nobody feature, it
can't read your .Xauthority file, so it falls back to no
authentication. Ssh refuses to let an unauthenticated session through
and gives precisely the error message that you're receiving. Using xhost
to control access to the server has absolutely no effect on any of this,
because the check is hardcoded into ssh.

You can change the NFS mounting parameters to shut off the
root-is-nobody feature. Or you can set up you login environment on AAA
to copy the relevent xauth keys to /var/tmp/.userid/.Xauthority, and
use that as your xauth file on that machine.
--
Kenneth Herron -- khe...@sgum.mci.com
"Netscape pollution must be eradicated."
-- Jeff Raikes, Vice president, Microsoft

Alan J. Flavell

unread,
Jan 15, 2000, 3:00:00 AM1/15/00
to
On 15 Jan 2000, Kenneth Herron wrote:

> This commercial application wouldn't be HP Openview, would it?

It's some kind of CAD application. Our user could tell you exactly what
it was, but I'm afraid I don't know what it is myself. Anyhow, you have
made some very interesting points, which I may be able to use as clues.

> Try this on for size: Your .Xauthority file is NFS-mounted on AAA,

Actually, the user's home file system is in AFS rather than NFS, and
permissions work quite differently in AFS, but the consequences are
pretty much equivalent.

> and
> the commercial application is trying to connect to your display from a
> process running as root.

If that is so in our case (I'm not quite sure how to determine it)
then we would be a step forward, as I think the AFS accesslist can be
adjusted to let this work (hopefully without compromising security too
much...?)

> Due to the NFS root-is-nobody feature, it
> can't read your .Xauthority file, so it falls back to no
> authentication.

Well, again AFS is different, but the consequences seem to be the same:
here's my own home directory accesslist for instance:

% fs listacl .
Access list for . is
Normal rights:
system:anyuser l
flavell rlidwka

and the .Xauthority file is right there in the home directory,
readable by me but no-one else.

{Aside: files that need to be world-readable and that are expected to
be in the home directory have to be put somewhere else, e.g ~/public,
and a symlink created to them from the home directory, in order to
work-around the fact that AFS permissions are directory-wide).

% fs listacl public
Access list for public is
Normal rights:
system:anyuser rl
flavell rlidwka

> Ssh refuses to let an unauthenticated session through
> and gives precisely the error message that you're receiving. Using xhost
> to control access to the server has absolutely no effect on any of this,
> because the check is hardcoded into ssh.

Check!



> Or you can set up you login environment on AAA
> to copy the relevent xauth keys to /var/tmp/.userid/.Xauthority, and
> use that as your xauth file on that machine.

A splendid series of ideas, and it all seems to make sense, thanks very
much. What's more, even if this app were running under some arbitrary
userid rather than specifically "root", we could make that idea work.

Now all that I need to do, as I say, is to adapt those to our particular
situation.

Much appreciated. I have the feeling that this is going to work ;-)
Now all that I have to do is persuade our user to try it (he's been
entirely happy since he got an easement to let his X windows through
the firewall, but it makes me more than a little nervous).

cheers


0 new messages