CF11 Scheduled Task Not Running - 401 Unauthorized

[Xiaofeng Liu]

Hi folks,

I'm running out of options. Basically I have a CF site running on Windows 2012 R2 & IIS 8.5.

I have two scheduled tasks sit under a sub directory of the web root directory.

In IIS 8.5 even from the webroot folder I set the IIS authentication to 'Anonymous Authentication' Enabled and it is using the option 'Application pool identity'.

Now when I run the scheduled task from the CF Admin, I got status code 401 - Unauthorized in the CF http.log.

But, if I directly run the URL of the scheduled task in a browser, it works without any problem.

When I googled, I realized that other people got the same issue but pretty much is because of Windows Authentication is turned on and anonymous authentication is turned off. But in my case, I got anonymous authentication even from the root.

Also I tried to set 'IUSR' for the anonymous authentication but still got the same issue.

Could anyone kindly shed some light if you had similar experience before?

--

Best regards,

Xiaofeng,

[Charlie Arehart]

Xiaofeng, when a sched task won’t work but the URL works in a browser, it’s nearly always an issue of the URL being behind a form of authentication that CF can’t support (as a scheduled task or CFHTTP call) but the browser can.

As for why it could “no problem” from the browser, it’s usually that your browser has cached the credentials, so you’re not seeing that they are required and requested, and passed in by the browser.

Here’s a simple test you can do: if using Chrome, open a “new incognito window” (via the “hamburger” menu in the top right). If in FF, use the same to open a “new private window”. In either case, visit the URL, and if there is a need of authentication, you will be prompted for it. If you are not, let us know. If you ARE prompted, then you need to find out if IIS is configured (whether at the folder or site level) to require authentication (see the “authentication” feature in the IIS control panel), and if so, and it’s only supporting “windows authentication”, then that’ why you can’t access the URL as a sched task. CF does not support that.

Let us know what you find.

/charlie

--
You received this message because you are subscribed to the Google Groups "cfaussie" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cfaussie+u...@googlegroups.com.
To post to this group, send email to cfau...@googlegroups.com.
Visit this group at http://groups.google.com/group/cfaussie.
For more options, visit https://groups.google.com/d/optout.

[Xiaofeng Liu]

Hi Charlie,

Thanks for offering the tips of testing from "incognito and private window". I tried both and both of them did not prompt me for authentication. So I am able to run the scheduled task even from there without problem.

Back to IIS, from the Basic Settings I can see the site is using "Pass-through authentication" - ApplicationPoolIdentity. Basically I got "Anonymous Authentication" turned on already. I also noticed that there is an warning (exclamation mark) for Authorization which says "Cannot verify access to path - [webroot of the site]".

It looks like this is to do with the built-in ApplicationPoolIdentity I'm using in IIS for anonymous authentication does not have Read access to my web root. But I'm not sure how that can be configured. Any more thoughts?

Thanks,

Xiaofeng
​

--

Best regards,

Xiaofeng,^_^

[Charlie Arehart]

Well, I have four thoughts for you to consider:

A) Focusing on it first as a scheduled task in CF, if the request works from a browser (including one like an incognito window) and yet it does not work in the scheduled task mechanism—for the exact same URL—then I would think there should be no problem with IIS. If there were, the request from the browser wouldn’t work.

And CF would not itself cause the request to act differently in requesting the URL than the browser would—other than not being able to do Windows auth, which you’ve proven with the incognito window is not the problem.

As such, I don’t know what is making that fail.

B) As for the rest of your note, well, you can add authentication info in the app pool (in its advanced settings page), as well as at the site level (in its advanced settings page). Again, though, if the request works from a browser, then I’d not think you have a problem in IIS.

I’ll just say that one can get lost in the weeds of permissions in IIS, at the app pool, site, and folder levels. If you’re needing to do all that sort of tweaking just in order for things to work, it could be sticky finding a solution. (But gain, if it works in a browser, then I’d not see how any such permissions issues would change how the EXACT SAME URL works from CF.)

C) Actually, I just had a thought: when you say you are running the URL in a browser, are you running that browser on the same machine where CF is running? If not, beware that you may have a DNS or other issue, where the domain in the URL may resolve differently from your workstation to the destination server, than it may from the CF server to itself. (I’m assuming that the scheduled task URL is a CFM page running on your own server, since you’re looking at IIS with regard to it.)

If you try the URL from a browser on the server running CF, is it the same issue? And in an incognito or private window?

D) If that’s not it, one more thought (which I’ve seen work before) would be to try creating a new site (and a new app pool) and seeing if you (in a browser) and then CF (as a sched task) could access the URL. You may need to come up with a new IP, domain name, or port to use for the bindings to distinguish it from the existing site. If it then worked, then you could consider how this new site and app pool may differ from the failing one, if perhaps you or someone else has been doing some tinkering with settings.

If that’s not helpful, I’m afraid I’m out of ideas for now.

/charlie

 

From: cfau...@googlegroups.com [mailto:cfau...@googlegroups.com] On Behalf Of Xiaofeng Liu
Sent: Monday, November 16, 2015 5:45 PM
To: cfau...@googlegroups.com
Subject: Re: [cfaussie] CF11 Scheduled Task Not Running - 401 Unauthorized

 

Hi Charlie,

 

Thanks for offering the tips of testing from "incognito and private window". I tried both and both of them did not prompt me for authentication. So I am able to run the scheduled task even from there without problem.

 

Back to IIS, from the Basic Settings I can see the site is using "Pass-through authentication" - ApplicationPoolIdentity. Basically I got "Anonymous Authentication" turned on already. I also noticed that there is an warning (exclamation mark) for Authorization which says "Cannot verify access to path - [webroot of the site]".

<snip>

[Mark King]

Is it possible IIS is blocking requests from unknown user agents?

 

See http://www.dantor.com/support/misc/web-config-requestFiltering-user-agent.aspx

 

I doubt that is it but it is worth a look maybe.

 

Also what happens if you give the app pool identity full control in IIS (temporarily to check if it works)?

--

[Charlie Arehart]

That’s good outside-the-box thinking, there, Mark. :-)  That said, I’d note that if Request Filtering were blocking it, it would provide a 404 instead (typically), so I doubt it will be that (he had said he was seeing a 401 in the CF http log), but he should certainly check that and think of any other such possibilities.

Still, I’m looking forward to his thoughts on the other few things I’d proposed. It’s like a mystery to be solved.:-) Not so fun when you’re suffering it, of course.

/charlie

 

From: cfau...@googlegroups.com [mailto:cfau...@googlegroups.com] On Behalf Of Mark King
Sent: Monday, November 16, 2015 8:33 PM
To: cfau...@googlegroups.com
Subject: RE: [cfaussie] CF11 Scheduled Task Not Running - 401 Unauthorized

 

Is it possible IIS is blocking requests from unknown user agents?

 

See http://www.dantor.com/support/misc/web-config-requestFiltering-user-agent.aspx

 

I doubt that is it but it is worth a look maybe.

 

Also what happens if you give the app pool identity full control in IIS (temporarily to check if it works)?

 

 

From: cfau...@googlegroups.com [mailto:cfau...@googlegroups.com] On Behalf Of Charlie Arehart
Sent: Tuesday, 17 November 2015 12:12 PM
To: cfau...@googlegroups.com
Subject: RE: [cfaussie] CF11 Scheduled Task Not Running - 401 Unauthorized

 

Well, I have four thoughts for you to consider:

<snip>

[Xiaofeng Liu]

Thanks again for all these thoughts.

For C), I'm running exactly the same URL from a browser (Chrome) on the server running CF 11 - Production Secure Profile and also tried the Icognito window without any issue. But by clicking the 'Run (play)' button next to the scheduled task in CF admin it completes the task immediately with an entry added into CF http.log with status code 401.

I also went back to IIS and changed the application pool setting of that site. In the advanced settings I changed the identity from ApplicationPoolIdentity to a local Windows login with administrator privileges. That actually fixed the IIS warning I previous saw:

However when I clicked the button to run the scheduled task from CFAdmin with or without specifying username and password, still got no luck. Like you said, it is probably not to do with IIS as long as I can run it from browser on the CF server in incognito mode.

The last thing I haven't try is to reboot ColdFusion. Since this is a production server, I will have to wait until later tonight. I'm not sure it will make any difference but I doubt it has anything to do with it.

For D), it looks like I have to try this option. When you said create a new site, do you mean create an exactly same site pointing to the same web root as the existing one but just specify a different port on that server and use it to run the scheduled tasks? 

Thanks,

Xiaofeng

--

You received this message because you are subscribed to the Google Groups "cfaussie" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cfaussie+u...@googlegroups.com.
To post to this group, send email to cfau...@googlegroups.com.
Visit this group at http://groups.google.com/group/cfaussie.
For more options, visit https://groups.google.com/d/optout.

--

Best regards,

Xiaofeng,^_^

[Xiaofeng Liu]

Hi Mark,

Thanks for the thoughts. I checked web.config and it does not have the requestFiltering section.

Cheers,

Xiaofeng

--

Best regards,

Xiaofeng,^_^

[Charlie Arehart]

Sure, and thanks for the confirmation on C. And yes, on D.

Though do beware that with IIS 7 and above, when a site points to a webroot, it reads and uses any web.config there for the setup of various IIS settings. So if you create a new site and point to the same webroot, and there is a web.config which may have settings that are influencing this, then the “new” site could suffer the same problem.

But if either there is no web.config, or you look at it and see nothing that seems to be impacting this, yes there’s a chance that a new site (and its app pool) may allow things to work when the old one did not. Sometimes there’s just no explaining why this works, but I’ve seen it so. (And for those familiar with such things, this is even when the entries in the central applicationhost.config for each site, the old failing and new working one, seem otherwise identical.) I get the sense there may still be other IIS settings stored somewhere beyond those two files which can impact things, and perhaps that’s why a “new” site can work when an old one cannot.

Let us know how it goes.

/charlie

 

From: cfau...@googlegroups.com [mailto:cfau...@googlegroups.com] On Behalf Of Xiaofeng Liu
Sent: Monday, November 16, 2015 10:33 PM
To: cfau...@googlegroups.com
Subject: Re: [cfaussie] CF11 Scheduled Task Not Running - 401 Unauthorized

 

Thanks again for all these thoughts.

 

For C), I'm running exactly the same URL from a browser (Chrome) on the server running CF 11 - Production Secure Profile and also tried the Icognito window without any issue.

<snip>

 

For D), it looks like I have to try this option. When you said create a new site, do you mean create an exactly same site pointing to the same web root as the existing one but just specify a different port on that server and use it to run the scheduled tasks? 

 

<snip>

 

[Charlie Arehart]

Do beware that (as I hinted at in my last note, though written after this one), it’s NOT only the web.config that could have request filtering settings for a site. They may also be in the central applicationhost.config file, either at the site level there, or up at the server level (therefore inherited by all sites unless they override it).

But the simplest thing is to look in IIS, at the site level, at the request filtering feature, and look at the various tabs there.

But as I said in my earlier note today, if that was blocking things, it would be producing a 404 typically, not a 401.

/charlie

 

From: cfau...@googlegroups.com [mailto:cfau...@googlegroups.com] On Behalf Of Xiaofeng Liu
Sent: Monday, November 16, 2015 10:54 PM
To: cfau...@googlegroups.com
Subject: Re: [cfaussie] CF11 Scheduled Task Not Running - 401 Unauthorized

 

Hi Mark,

[Dale Fraser]

Can u browse to the url from the server

Ie can the server resolve DNS and the url

Regards

Dale Fraser

--

[Charlie Arehart]

Dale, yep. He’s confirmed that in response to some other messages in the thread here.

/charlie

[Xiaofeng Liu]

Now I don't know what else I can go ahead with this one.

I checked the web.config and I don't see any requestFiltering settings. Also from within IIS I also checked the Request Filtering section does not seem to have tricky things.

So I went ahead and create a brand new site on the same server with a different port. I made that site point to the same web root of the existing one. Re-ran the wsconfig as administrator from the cfusion instance (the only one I have at the moment) and tested again. It turned out to be the same result. I can do it from the browser (now on a different port), but when I run it from CF Admin, bang, 401.

What is not making sense to me, still, is when I tried CFHTTP to the URL of the sched task template I saw the dump for WWW-Authenticate is like this:

I mean I don't think Windows Authentication is even enabled. What?

--

You received this message because you are subscribed to the Google Groups "cfaussie" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cfaussie+u...@googlegroups.com.
To post to this group, send email to cfau...@googlegroups.com.
Visit this group at http://groups.google.com/group/cfaussie.
For more options, visit https://groups.google.com/d/optout.

--

Best regards,

Xiaofeng,^_^

[Mark King]

What happens if you add some text at the top of the page that is called and then do a <cfabort>? Do you still get the error in the CF Admin.

 

I am wondering if there is something in the page you are calling throwing the error, not necessarily CF itself.

[Ricardo Russon]

Is that "Proxy Connection" Header expected? (Can anyone comment on this)

Is your machine running through some proxy that is handling the authentication for you?

Can you see if someone outside of your network is able to run the page successfully?

Maybe try run it from a mobile phone off the WiFi.

[Xiaofeng Liu]

I tried doing a <cfabort> at the very top of the sched task template, it came back with same error - 401 :(

[dmi...@yakhnov.info]

Hi Xiaofeng,

This is not CF, it is all about IIS config.

Put a simple .txt file and try to fetch it using CFHTTP, 100% you will end up with the same HTTP error 401.

Why? Browser (no matter if it running in generic or private browsing mode) is using your credentials (domain or local account) to negotiate access with IIS, while CF server is run without these permissions.

Google "http error 401 IIS" and you can find following link with 5 reasons IIS refuses to authorize CF:
http://blogs.msdn.com/b/david.wang/archive/2005/07/14/howto-diagnose-iis-401-access-denied.aspx

So, check your IIS logs for sub-401-code to determine cause of your problem.

Cheers,
Dmitry.

[Xiaofeng Liu]

I figured this out, sort of. Now I changed sched task URL to use the public IP address of the server, I got status 200. 

So not localhost, not 127.0.0.1 but the public IP address. I guess this is due to the server got multiple NICs so that localhost cannot be resolved somehow?

[Charlie Arehart]

But tell me, Xiaofeng , when I proposed earlier much of the same that people have said today, you said in reply the other day that you HAD used the EXACT same URL in your browser (run from the server) as in the sched task.

So when you refer now to changing the sched task URL to use“not localhost, not 127.0.0.1 but the public IP address”, are you saying that IS or IS NOT what you tested before when you ran the test in your browser on the server? Perhaps you thought this was not significant, but it would be.

It’s the IP address (or domain or machine name) in the URL which influences what IIS site handles the request. You may have windows auth on in one site, and not the other. And you may have been going to different ones all along.

/charlie

 

From: cfau...@googlegroups.com [mailto:cfau...@googlegroups.com] On Behalf Of Xiaofeng Liu

Sent: Tuesday, November 17, 2015 11:48 PM
To: cfau...@googlegroups.com

Subject: Re: Re[2]: [cfaussie] CF11 Scheduled Task Not Running - 401 Unauthorized

[Xiaofeng Liu]

Hi Charlie, 

I originally was using a URL like "http://localhost:1981/cron/schedTask.cfm" to run the scheduled task. So that was the URL I specified in CF Admin. That threw 401 to me.

Also I tested from the browser by accessing the same URL which is "http://localhost:1981/cron/schedTask.cfm" (normal and icognito mode) and it is working fine.

So now I have to update the URL to use the server's public IP address in CF Admin and it works. I don't understand why CF running locally on the same server has to use the external IP, I mean not the localhost to run the sched task that is also on the same server? Given that localhost is also in the host file.

Cheers,

Xiaofeng

--

You received this message because you are subscribed to the Google Groups "cfaussie" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cfaussie+u...@googlegroups.com.
To post to this group, send email to cfau...@googlegroups.com.
Visit this group at http://groups.google.com/group/cfaussie.
For more options, visit https://groups.google.com/d/optout.

--

Best regards,

Xiaofeng,^_^

[Charlie Arehart]

Sure, that is curious. If you may ever get to the bottom of things, it would be interesting to hear. Thanks for the update.

/charlie

From: cfau...@googlegroups.com [mailto:cfau...@googlegroups.com] On Behalf Of Xiaofeng Liu
Sent: Wednesday, November 18, 2015 5:07 PM
To: cfau...@googlegroups.com
Subject: Re: Re[2]: [cfaussie] CF11 Scheduled Task Not Running - 401 Unauthorized

 

Hi Charlie, 

[Allan Cliff]

Did anyone ever get to the bottom of this?

Having exactly the same problem here.

Local IP address url calls to another server (Windows 2012 R2) from the browser work as expected.

Also urls in my desktop browser (via VPN).

When I put the url into a scheduled tasks, I just get 401 every time.

Any solution would be appreciated.

Regards,

Allan

[charlie arehart]

Did you see the message from the OP, Xiaofeng Liu, of how he solved it (for his specific situation)?

https://groups.google.com/d/msg/cfaussie/Ug1tjCcUXCk/RLsTij8XBAAJ

He solved it by changing the URL (in the sched task) to be the IP address of the machine (not localhost or 127.0.0.1).

If that’s not your solution/issue, did you assess the first few other answers, which addressed more typical explanations for problems of a sched task URL not working though a browser one does?

/charlie

--

You received this message because you are subscribed to the Google Groups "cfaussie" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cfaussie+u...@googlegroups.com.
To post to this group, send email to cfau...@googlegroups.com.

Visit this group at https://groups.google.com/group/cfaussie.

[Xiaofeng Liu]

Allan, as Charlie mentioned, I solved my case by using the public IP address of the machine. So maybe you would like to try that if haven't done so. 

Cheers

To unsubscribe from this group and stop receiving emails from it, send an email to cfaussie+unsubscribe@googlegroups.com.

To post to this group, send email to cfau...@googlegroups.com.
Visit this group at https://groups.google.com/group/cfaussie.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "cfaussie" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cfaussie+unsubscribe@googlegroups.com.

To post to this group, send email to cfau...@googlegroups.com.
Visit this group at https://groups.google.com/group/cfaussie.
For more options, visit https://groups.google.com/d/optout.

--

Best regards,

Xiaofeng,^_^

[Allan Cliff]

I was using CF10 on a windows 2012 box.

I was unable to use 127.0.0.1, localhost, local IP address, nor any normal url.

They all resulted in 401.2

From what I could gather from the resulting messages, it seems the calls made by Scheduled Tasks could not use the Integrated Windows Authentification in IIS (8.5), but calls worked when I installed and used Basic Authentification.

Thankfully it works now.

Thanks for replying Xiaofeng and Charlie.