Problems with Waffle

[Josep Marí]

Hi, after a few days trying I decided to ask the community for a little help, because I can't figure out the solution.

I've been using a Valve: https://github.com/dblock/waffle/blob/master/Docs/tomcat/TomcatSingleSignOnValve.md  (The filter didn't work for me, I don't know why, so I used the valve. Actually I don't know when or why use each one. )

I had it all working until I changed the Tomcat service user owner from LocalSystem to an user from the domain. Now, it works on localhost but if I try to access from outside (same domain), SSO fails, bringing me the logon popup.

I've tried with to set the SPN to the domain controller but does not work either.

Any idea? Thank you.

[Daniel Doubrovkine]

I think the user under which the service runs doesn't have the necessary privileges to authenticate users. When a service runs as LocalHost it uses the machine's security context to do authentication - logging on users is something you'd expect from a server that's on the domain :)

TBH I am not sure what that account needs to have to do the same. If you figure it out, I'd like an FAQ. It's definitely possible, there was at least one discussion (and link from FAQ) (http://waffle.codeplex.com/discussions/243106, and yes CodePlex is not responding now) where someone solved their SSO issues by running the service as a domain account.

A filter uses a generic servlet interface for Java web servers.

A valve is specific for Tomcat, and specific to a given tomcat version, and allows some extra features.

--
You received this message because you are subscribed to the Google Groups "waffle" group.
To unsubscribe from this group and stop receiving emails from it, send an email to waffle-users...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

dB. | Moscow - Geneva - Seattle - New York
code.dblock.org - @dblockdotorg - artsy.net - github/dblock

[Josep Marí]

Yes, that was what I was thinking. The problem is that the user is already in the admin group (both at the domain controller and tomcat server), so I don't know which privileges are missing.

Thanks for the reply.

[Daniel Doubrovkine]

I wouldn't use a domain admin account for a service! That's the keys to the kingdom. Localhost only gives you the machine.

[Josep Marí]

No, I know I know, I was just trying to give him more privileges. :)

[Josep Marí]

I noticed that the Event Viewer of the domain controller is logging repeatedly the same error every 15 minutes aprox.

ErrorCode

0xd

ErrorMessage

KDC_ERR_BADOPTION

ExtendedError

0xc00000bb KLIN(0)

[Ed Pike]

I'm getting the same error in Windows System Log.

I'm curious if my serverrealm, servername and targetname are "normal"? The same event says:

ServerRealm XYZ.COM

ServerName d001233$@XYZ.COM

TargetName d001233$@XYZ...@XYZ.COM

What is the dollar sign? 

Why is the domain name repeated in the Servername?

Why is XYZ.COM in the name twice in targetname?

The larger context is:

I am trying the waffle-filter demo on my Win 7 dev box.

Windows SSO from any machine in the domain fails.

********** 

The really weird part: I have the exact same set up on a Win Server 2012 staging box and it works.

**********

Stack on both boxes:

Waffle 1.6 on Tomcat 7, JDK1.7.0_55 64 bit

On domain xyz.com (we have no others)

The Waffle 1.6 zip did not have guava, so I got guava-17.0.jar. 

All the libs are in the tomcat/libs folder.

Tomcat 7 is installed as a service using the tomcatdev domain service account.

In the service interface it uses tomc...@xyz.com

If I try just using tomcat, with no domain, the login check fails when I apply the change. 

Web browsers on both boxes have both boxes in the intranet zone:

http://*.xyz.com

https://*.xyz.com

Both have win auth enabled.

Win 7 box (d001233) is physical and on our 10.10 net, the Server 2012 box (rz1s) is a vm and on our 10.34 subnet.

The clocks on both machines are the same.

The error is from my Win 7 dev box.

Wireshark on the win 7 server says (compressed for your convenience):

KRB Error: KRB5KRB_AP_ERR_MODIFIED, Server Name (Principal): tomcatdev

*one odd thing*: the date on the packet with the err msg says correct day but says 16:14:30 for time when it should be about 9:30 AM.

Checked the clocks on both machines (in the taskbar) and they are the same.

Checked the time on the kerb err in the system log and it says logged 9:08 AM, but the "ServerTime" in the general information says "16:8:50"

Ah, part of the error msgs is in GMT (Z?) and part is pacific coast. 8 hours minus 1 for daylight savings, I guess.  

I enabled kerberos logging and got the KDC_ERR_BADOPTION when doing a failed login from another machine.

Remember, the filter demo works fine on Server 2012.

In AD, neither machine had "allow delegation" turned on.

Both tomcat services use the same service user: tomcatdev, a domain user

The spns are as follows:

setspn -L tomcatdev

HTTP/rz1s.warn.com (srvr 2012 staging server)

HTTP/rz1s

HTTP/d001233.xyz.com (win 7 dev box)

HTTP/d001233

setspn -L d001233

RestrictedKrbHost/D001233

RestrictedKrbHost/D001233.xyz.com

HOST/D001233

HOST/D001233.xyz.com

setspn -L rz1s

WSMAN/rz1s (what is this?)

ditto xyz.com

RestrictedKrbHost/RZ1S

ditto xyz.com

HOST/RZ1S

HOST/rz1s.xyz.com (caps differences should not matter right?)

Checked for duplicate spns, nothing.

WAffle logging is turned on.

Tomcat logs/tomcat7/stdout has this on last line: 

[http-bio-80-exec-3] WARN  w.servlet.NegotiateSecurityFilter - error logging in user: The handle specified is invalid

I stopped the test with the basic auth prompt still waiting in the requesting browser, btw.

I have spent the whole week learning Kerb, Wireshark, etc. And I still can't get it to work on the Win 7 box. I've tried on another win 7 machine and get the same problem.

I want to end the week "winning" and enjoy my weekend, please help!

I'm going to try getting a win 7 box on the VMsphere (vs physical) to control for that var (the server 2012 is a vm).

I'll have it in 10.10 then if that doesnt work I'll put it on 10.34. 

After that I'm stumped.

[Daniel Doubrovkine]

In Windows machine accounts registered with the domain end with a $ sign, http://msdn.microsoft.com/en-us/library/cc246064.aspx.

The @XYZ thing looks suspicious, but I don't know.

Do two things:

1) Disable Kerberos in Waffle, tell it to use NTLM instead of Negotiate. Something like this:

       <filter>
        <filter-name>SecurityFilter</filter-name>
        <filter-class>waffle.servlet.NegotiateSecurityFilter</filter-class>
        <init-param>
            <param-name>allowGuestLogin</param-name>
            <param-value>false</param-value>
        </init-param>
        <init-param>
         <param-name>waffle.servlet.spi.NegotiateSecurityFilterProvider/protocols</param-name>
         <param-value>
            NTLM
             Negotiate
         </param-value>
        </init-param>
    </filter>
    <filter-mapping>
        <filter-name>SecurityFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

This will make all that Kerberos SPN stuff irrelevant for the test and we'll see if the system is even capable of doing a Windows logon.

2) Setup a plain IIS web server on one of those machines, lets see if this is a waffle-related problem or a domain setup one.

--
You received this message because you are subscribed to the Google Groups "waffle" group.
To unsubscribe from this group and stop receiving emails from it, send an email to waffle-users...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Ed Pike]

I disabled guest login, and now it does windows auth from other machines, I assume with NTLMv2.

I will now try IIS and get back to you.

BTW, I do believe that it will be something to do with our AD setup, other than that, something specific to Win 7.

Thanks a bunch!

[Daniel Doubrovkine]

At which level did you disable guest login? And are you saying that it started working after that?

You can look at server-side logs for which protocol is chosen. Or if you look at the headers, very long ones are Kerberos, short ones are NTLMv2.

[Ed Pike]

I modified the web.xml of the waffle-filter demo app.

It was untouched since I dropped in the demo app.

I set allowGuestLogin to false.

The other thing I did was switch the protocol order to match what you posted.

It had been negotiate then NTLM, now it is switched. 

I enabled IIS, and was going through the checklist to enable Kerb on IIS. 

I got to the set SPN part. I decided to check something.

setspn -Q for HTTP/d001233 (my workstation where kerb does not work)

and it was not found.

*******

Spoiler Alert: from here down is saying how the actual problem was with the SPN but I'm putting it in because it might help someone else

******

I did same thing for one of the machines that works, and it is found, under user tomcatdev.

HTTP/host1

HTTP/host1.xyz.com

I then did setspn -L tomcatdev, and the good machines *and my workstation are listed*.

as in:

HTTP/d001233:8080

HTTP/d001233.xyz.com:8080

HTTP/host1

HTTP/host1.xyz.com

HTTP/d001233

HTTP/d001233.xyz.com

The 8080's are from a previous effort that I was going to try but I fell back to port 80, the original spn setup, to control for that variable because the good boxes are port 80 as well.

Decided to setspn -D HTTP/d001233:8080 tomcatdev, to remove the SPN mapping, it worked. 

Tried same for HTTP/d001233 and got the confirmation that the object had been modified.

typed setspn -L for tomcatdev to confirm, and the 8080's were gone but not HTTP/d001233! wth?

**Then it dawned on me** one of our sys admins set the first one up for me (HTTP/d001233) and used capital o's instead of zeroes!!!!!!!

I had set up all subsequent SPNs myself (I left various other ones out of the post to spare you guys). They had proper zeroes.

Days and days of hell because of this.

Also, in the meantime, I logged into one of the good boxes, which are VMs and live in a diff subnet from our workstations. 

I ran:

setspn -L tomcatdev

And one time it would list the same as same command on my workstation, next time it would list some SPNs that I had removed. 

I did this about 10 times and it was random what I'd get back.

I was getting random results for setspn -L tomcatdev!!!!

These new tomcat vms were recently setup by our sys admins on a new 10.34 subnet. We were bought and our parent corp wants to integrate networks, so the old 172 would not work anymore.

From there, setspn is apparently resolving DNS/DC to "whereever" and so sometimes I am getting our local one and sometimes I am getting AD/DCs from one of our other physical locations.

God knows how long it takes for those entries to sync, the network admins certainly couldnt tell me. I have a ticket to them now to set the vms to consistently use our local boxes first.

I deeply appreciate your help and this product.

[Daniel Doubrovkine]

It'd be great if some of this was written up in an FAQ, please contribute!