Problems with an offline network (WSUS needs more files)

[psnarula]

I posted this to the forums at http://wsus.info but didn't get a response so
I figured I'd try here too. Sorry for lengthy post but I wanted to err on
the side of giving too much detail.

I have a WSUS 3 server connected to the internet that manages updates for an
internet-connected network and it works fine. I use group policy to force the
clients on this network to get updates from the WSUS server instead of from
Microsoft Update.

Separately, I have an offline network in a lab environment. Connecting lab
machines to the internet is not an option. Last week I set up a new lab
machine to act as a WSUS 3 server for the rest of the machines in the lab.
The lab WSUS server is running on Windows Server 2003 R2 SP2, x64 edition
that I installed from our MSDN discs. Installation was uneventful. I will
eventually roll out group policy to force all clients on the lab network to
get updates from the lab WSUS server but for now I just used local policy on
the WSUS server itself to add it to the computers it updates. So right now
the lab WSUS server updates only itself.

Both WSUS servers have tape drives attached via SCSI connections so I
transferred updates and metadata to the lab as follows:

1. used "wsusutil export" to backup metadata to companion .cab and .log files
2. used ntbackup.exe to write the contents of the WsusContent directory and
the two files created in step 1 to a tape
3. walked down the hall to the lab with said tape in hand
4. used ntbackup.exe to restore the WsusContent directory to the lab WSUS
server (more on this below)
5. used "wsusutil import" to import metadata from the .cab and .log files

More details about #4 above:

The WSUS server that is connected to the internet has a separate partition
for WSUS (the "D" drive). So on that machine, updates are stored on
D:\WsusContent. On the lab WSUS server, I forgot to create a separate
partition for the WSUS server when I was setting up Windows so I just
accepted the defaults during the WSUS install (updates are stored on
C:\WSUS\WsusContent). Since the files are stored in different locations, I
chose not to use the options in ntbackup.exe that restores files to the
original location. Rather, I chose the option to restore files to an
alternate location and then I specified c:\tape_content. Finally, I moved
everything from C:\tape_content\WsusContent to C:\WSUS\WsusContent and then
did #5 above.

It was getting late so I started the "wsusutil import" and then left for the
day. When I came back in the morning I fired up the WSUS MMC snap in from
Administrative Tools and all of my updates were there. But there is a
problem.

For updates that have not been approved, the File Status column lists one of
two messages:

1. Ready for installation (files are not downloaded)
2. The Microsoft Software License Terms for this update are downloading

If I click on an update with message #2 above, the status bar at the top of
the update (in the bottom frame) says, "This update cannot be approved for
installation because its Microsoft Software License Terms are still
downloading".

So I cannot approve updates that give message #2 above. If I approve an
update for installation that shows message #1 above, the status bar at the
top of the update (in the bottom frame) says, "The files for this update have
not yet been downloaded. The update can be approved but will not be available
to computers until the download is complete".

I've got over 20 gigs of content in the WsusContent directory. So why
doesn't the WSUS server see them there? Obviously attempts to download
content from the internet are going to fail since none of these machines are
connected to the internet. How do I convince WSUS that there is no need to
find additional files?

I know for a fact that at least one of the updates for which I get message
#2 above has been approved and rolled out on the internet-connected network
(the update is Internet Explorer 7).

Could this related to the fact that the files are stored in different
directories on my two WSUS servers? I can always add a second hard drive to
the lab WSUS server (or even use something like Partition Magic) to create a
D: drive but is this really necessary?

Things I have tried which have not fixed the problem:

1. restarting the Automatic Updates service
2. wsusutil reset

Any help or suggestions would be appreciated. Thanks.

[Lawrence Garvin [MVP]]

"psnarula" <psna...@discussions.microsoft.com> wrote in message
news:C185C155-70A3-4E56...@microsoft.com...

> The lab WSUS server is running on Windows Server 2003 R2 SP2, x64 edition
> that I installed from our MSDN discs.

Aside from this being an unlicensed use of MSDN software...... <g>

> The WSUS server that is connected to the internet has a separate partition
> for WSUS (the "D" drive). So on that machine, updates are stored on
> D:\WsusContent. On the lab WSUS server, I forgot to create a separate
> partition for the WSUS server when I was setting up Windows so I just
> accepted the defaults during the WSUS install (updates are stored on
> C:\WSUS\WsusContent). Since the files are stored in different locations, I
> chose not to use the options in ntbackup.exe that restores files to the
> original location. Rather, I chose the option to restore files to an
> alternate location and then I specified c:\tape_content. Finally, I moved
> everything from C:\tape_content\WsusContent to C:\WSUS\WsusContent and
> then
> did #5 above.

I'd venture a guess that using this procedure has totally mucked up the
necessary permissions on your ~\WSUSContent folder. Specifically it happened
when you executed the move from C:\tape_content\WsusContent to
C:\WSUS\WsusContent, and possibly specifically as result of the creation of
C:\tape_content, which probably didn't have the requisite permissions for
inheritance in the first place.

Please make sure the permissions on your new server follow these guides:

The \WSUS folder should have:
Full Control: SYSTEM, Administrators
Read/Read & Execute/List Folder Contents: Users, NetworkService
and these permissions should be inherited to \WSUS\WSUSContent, but not
\WSUS\MSSQL$WSUS

The \WSUS\MSSQL$WSUS folder should have:
Full Control: Administrators
inherited down the folder tree

Had you restored from tape directly to C:\WSUS\WSUSContent, you might have
avoided this issue, as the restored files would have inherited the correct
permissions from the \WSUS folder.

--
Lawrence Garvin, M.S., MCTS, MCP
MVP - Software Distribution (2005-2007)
MS WSUS Website: http://www.microsoft.com/wsus
My Websites: http://www.onsitechsolutions.com;
http://wsusinfo.onsitechsolutions.com
My MVP Profile: http://mvp.support.microsoft.com/profile/Lawrence.Garvin

[psnarula]

"Lawrence Garvin [MVP]" wrote:

> "psnarula" <psna...@discussions.microsoft.com> wrote in message
> news:C185C155-70A3-4E56...@microsoft.com...
>
> > The lab WSUS server is running on Windows Server 2003 R2 SP2, x64 edition
> > that I installed from our MSDN discs.
>
> Aside from this being an unlicensed use of MSDN software...... <g>

Huh? I installed the operating system by creating two discs from the ISO
images on disc 2939.3. I activated the operating system via telephone and
had no problems.

This thought had never occurred to me. But yes, I agree that you have a
valid point. A coworker used the same tape for a different offline network
and he has no problems (but he restores directly to a separate D: partition
that was created when he set up the operating system).

On Friday, I decided that I wanted to try moving the WsusContent directory
to D: (so it would mimic the setup on the internet-connected WSUS server). I
was thinking that perhaps there was some hard-coded path to the files stored
in the .cab file that gets created when I do "wsusutil export". I later
loaded the extracted package.xml and metadata.txt files into UltraEdit and no
longer believe that this is the case since text searches for hard-coded paths
did not return anything. Can somebody confirm this?

So anyway, I added an external USB hard drive (250 GB) and used diskmgmt.msc
to format it and name it the D drive. Then I used "wsusutil movecontent" to
copy the WsusContent directory to D:\WsusContent. This didn't fix my
problem. So I uninstalled WSUS, uninstalled Windows Internal Database, and
tried again. This time I installed WSUS to D:\ but when it came time to put
files in D:\WsusContent, I just copied the same set of files that I've been
using all along (from C:\tape_content\WsusContent).

I'll check the file permissions you mentioned. But I don't have
D:\MSSQL$WSUS (or anything like unto it; is this because I'm using Windows
Internal Database instead of a full-blown SQL Server 2005 installation?) I
think I'm just going to delete WSUS and Windows Internal Database again and
start over. But this time I'm going to restore from tape and hopefully let
it inherit the proper file permissions.

Thanks for a nice reply. I'll post back with an update.

Paul

[Lawrence Garvin [MVP]]

"psnarula" <psna...@discussions.microsoft.com> wrote in message

news:4849D58C-39F0-4183...@microsoft.com...

> "Lawrence Garvin [MVP]" wrote:
>
>> "psnarula" <psna...@discussions.microsoft.com> wrote in message
>> news:C185C155-70A3-4E56...@microsoft.com...
>>
>> > The lab WSUS server is running on Windows Server 2003 R2 SP2, x64
>> > edition
>> > that I installed from our MSDN discs.
>>
>> Aside from this being an unlicensed use of MSDN software...... <g>
>
> Huh? I installed the operating system by creating two discs from the ISO
> images on disc 2939.3. I activated the operating system via telephone and
> had no problems.

And.. did you activate these products using *MSDN* Product Keys?

>> Had you restored from tape directly to C:\WSUS\WSUSContent, you might
>> have
>> avoided this issue, as the restored files would have inherited the
>> correct
>> permissions from the \WSUS folder.
>>
>
> This thought had never occurred to me. But yes, I agree that you have a
> valid point. A coworker used the same tape for a different offline
> network
> and he has no problems (but he restores directly to a separate D:
> partition
> that was created when he set up the operating system).
>
> On Friday, I decided that I wanted to try moving the WsusContent directory
> to D: (so it would mimic the setup on the internet-connected WSUS server).
> I
> was thinking that perhaps there was some hard-coded path to the files
> stored
> in the .cab file that gets created when I do "wsusutil export". I later
> loaded the extracted package.xml and metadata.txt files into UltraEdit and
> no
> longer believe that this is the case since text searches for hard-coded
> paths
> did not return anything. Can somebody confirm this?

I can. The only place the pathname to the content is stored is inside the
WSUS database on each individual deployment. It's why the 'wsusutil
movecontent' utility must be used -- because that contains the code to
execute the SQL UPDATE statement to define the new pathname.

> I'll check the file permissions you mentioned. But I don't have
> D:\MSSQL$WSUS (or anything like unto it; is this because I'm using Windows
> Internal Database instead of a full-blown SQL Server 2005 installation?)

You are correct. The MSSQL$WSUS folder would only exist on a WSUS 2.0
installation. My mistake; sorry for the confusion. On a WSUS 3.0 system with
WID the database files will be located in the UpdateServicesDBFiles folder.

If you choose to uninstall WSUS/WID and start over, be sure to reference the
instructions in the WSUS documentation for uninstalling WID, and make sure
you've uninstalled WID prior to reinstalling WSUS 3.0.

[psnarula]

"Lawrence Garvin [MVP]" wrote:

> did you activate these products using *MSDN* Product Keys?

Yes.

> The only place the pathname to the content is stored is inside the
> WSUS database on each individual deployment. It's why the 'wsusutil
> movecontent' utility must be used -- because that contains the code to
> execute the SQL UPDATE statement to define the new pathname.

So if I wanted to peer inside the SUSDB.mdf and SUSDB_log.ldf files to view
the internal structure of the database, how would I do that?

> If you choose to uninstall WSUS/WID and start over, be sure to reference the
> instructions in the WSUS documentation for uninstalling WID, and make sure
> you've uninstalled WID prior to reinstalling WSUS 3.0.

For uninstalling WID, I found this:

http://technet2.microsoft.com/windowsserver/en/library/f8abcf6e-b6ef-4872-bf51-1b89994700d51033.mspx

But I noticed that WID does appear in my Add or Remove Programs menu
(contrary to what is implied in the link above) and both times I have
uninstalled WSUS/WID, this is what I have used. The UpdateServicesDBFiles
folder does get deleted when I take this action but I am not sure about the
contents under C:\WINDOWS\SYSMSI. Is it really necessary to use the msiexec
command to uninstall WID? Each time I have reinstalled WSUS, the
installation wizard asks me to install a new instance of the WID so I am
assuming (perhaps erroneously?) that the old one has been blown away.

I have decided not to uninstall/reinstall WSUS/WID right now. Your comments
about file permissions got me thinking. So I spent some time yesterday
looking at permissions for NT Authority\Network Service. I compared
permissions on all three of the WSUS servers to which I have access (one is
connected to the internet, one is offline in my lab, and a third is offline
in a coworker's lab). File permissions on all three machines were identical.
But my problems persist.

I wondered if there would be any indication of my troubles in the Event
Viewer or some of the log files that get created. So I did some poking
around. The Event Viewer on the offline WSUS server in my lab has these
three entries:

Event Type: Information
Event Source: MSSQL$MICROSOFT##SSEE
Event Category: (2)
Event ID: 17126
Date: 8/27/2007
Time: 3:17:45 PM
User: N/A
Computer: RIVERS
Description:
SQL Server is now ready for client connections. This is an informational
message; no user action is required.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.
Data:
0000: e6 42 00 00 0a 00 00 00 æB......
0008: 17 00 00 00 52 00 49 00 ....R.I.
0010: 56 00 45 00 52 00 53 00 V.E.R.S.
0018: 5c 00 4d 00 49 00 43 00 \.M.I.C.
0020: 52 00 4f 00 53 00 4f 00 R.O.S.O.
0028: 46 00 54 00 23 00 23 00 F.T.#.#.
0030: 53 00 53 00 45 00 45 00 S.S.E.E.
0038: 00 00 00 00 00 00 ......

Event Type: Information
Event Source: MSSQL$MICROSOFT##SSEE
Event Category: (2)
Event ID: 26048
Date: 8/27/2007
Time: 3:17:45 PM
User: N/A
Computer: RIVERS
Description:
Server local connection provider is ready to accept connection on [
\\.\pipe\MSSQL$MICROSOFT##SSEE\sql\query ].

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.
Data:
0000: c0 65 00 00 0a 00 00 00 Àe......
0008: 17 00 00 00 52 00 49 00 ....R.I.
0010: 56 00 45 00 52 00 53 00 V.E.R.S.
0018: 5c 00 4d 00 49 00 43 00 \.M.I.C.
0020: 52 00 4f 00 53 00 4f 00 R.O.S.O.
0028: 46 00 54 00 23 00 23 00 F.T.#.#.
0030: 53 00 53 00 45 00 45 00 S.S.E.E.
0038: 00 00 00 00 00 00 ......

Event Type: Failure Audit
Event Source: MSSQL$MICROSOFT##SSEE
Event Category: (4)
Event ID: 18456
Date: 8/27/2007
Time: 3:17:46 PM
User: NT AUTHORITY\NETWORK SERVICE
Computer: RIVERS
Description:
Login failed for user 'NT AUTHORITY\NETWORK SERVICE'. [CLIENT: <named pipe>]

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 18 48 00 00 0e 00 00 00 .H......
0008: 17 00 00 00 52 00 49 00 ....R.I.
0010: 56 00 45 00 52 00 53 00 V.E.R.S.
0018: 5c 00 4d 00 49 00 43 00 \.M.I.C.
0020: 52 00 4f 00 53 00 4f 00 R.O.S.O.
0028: 46 00 54 00 23 00 23 00 F.T.#.#.
0030: 53 00 53 00 45 00 45 00 S.S.E.E.
0038: 00 00 07 00 00 00 6d 00 ......m.
0040: 61 00 73 00 74 00 65 00 a.s.t.e.
0048: 72 00 00 00 r...

Content that mirrors the event log is in
C:\WINDOWS\SYSMSI\SSEE\MSSQL.2005\MSSQL\LOG\ERRORLOG:

2007-08-27 15:30:02.04 Server Server local connection provider is ready
to accept connection on [ \\.\pipe\MSSQL$MICROSOFT##SSEE\sql\query ].
2007-08-27 15:30:02.04 Server SQL Server is now ready for client
connections. This is an informational message; no user action is required.
2007-08-27 15:30:02.74 Logon Error: 18456, Severity: 14, State: 16.
2007-08-27 15:30:02.74 Logon Login failed for user 'NT
AUTHORITY\NETWORK SERVICE'. [CLIENT: <named pipe>]

I also have a bunch of errors in C:\Program Files\Update
Services\LogFiles\SoftwareDistribution.log. Here is a sampling:

2007-08-27 17:56:27.939
UTC Error WsusService.7 ExecutionContext.runTryCode SusEventDispatcher got
exception while polling database. Polling will continue: Excpetion details:
System.Data.SqlClient.SqlException: Cannot open database "SUSDB" requested by
the login. The login failed.
Login failed for user 'NT AUTHORITY\NETWORK SERVICE'.

2007-08-27 17:56:27.939
UTC Error WsusService.17 ExecutionContext.runTryCode System.Data.SqlClient.SqlException:
Cannot open database "SUSDB" requested by the login. The login failed.
Login failed for user 'NT AUTHORITY\NETWORK SERVICE'.

2007-08-27 17:56:28.377
UTC Error w3wp.15 ExecutionContext.runTryCode SusEventDispatcher got
exception while polling database. Polling will continue: Excpetion details:
System.Data.SqlClient.SqlException: Cannot open database "SUSDB" requested by
the login. The login failed.
Login failed for user 'NT AUTHORITY\NETWORK SERVICE'.

2007-08-27 17:56:28.408
UTC Error w3wp.11 ExecutionContext.runTryCode SusEventDispatcher got
exception while polling database. Polling will continue: Excpetion details:
System.Data.SqlClient.SqlException: Cannot open database "SUSDB" requested by
the login. The login failed.
Login failed for user 'NT AUTHORITY\NETWORK SERVICE'.

2007-08-27 17:56:28.424
UTC Error w3wp.7 ExecutionContext.runTryCode SusEventDispatcher got exception
while polling database. Polling will continue: Excpetion details:
System.Data.SqlClient.SqlException: Cannot open database "SUSDB" requested by
the login. The login failed.
Login failed for user 'NT AUTHORITY\NETWORK SERVICE'.

So the errors from all three places seem to point at the same problem --
failed permissions for the NETWORK SERVICE account. But I've granted Full
Control to this account everywhere I think it could possibly by necessary.
Any thoughts?

Paul

[Kurt Falde [MSFT]]

You can use the sql express management studio
http://www.microsoft.com/downloads/details.aspx?FamilyID=c243a5ae-4bd1-4e3d-94b8-5a0f62bf7796&DisplayLang=en
to look inside your db's use \\.\pipe\MSSQL$MICROSOFT##SSEE\sql\query as the
servername..

--
Kurt Falde [MSFT]

"psnarula" <psna...@discussions.microsoft.com> wrote in message

news:7C5B2AC5-FA01-4DCD...@microsoft.com...

[Lawrence Garvin [MVP]]

"psnarula" <psna...@discussions.microsoft.com> wrote in message

news:7C5B2AC5-FA01-4DCD...@microsoft.com...

> "Lawrence Garvin [MVP]" wrote:
>
>> did you activate these products using *MSDN* Product Keys?
>
> Yes.

Well... then... there ya have it. You're using an MSDN licensed product for
non-development use, and that's a violation of the MSDN licensing terms.

>> The only place the pathname to the content is stored is inside the
>> WSUS database on each individual deployment. It's why the 'wsusutil
>> movecontent' utility must be used -- because that contains the code to
>> execute the SQL UPDATE statement to define the new pathname.
>
> So if I wanted to peer inside the SUSDB.mdf and SUSDB_log.ldf files to
> view
> the internal structure of the database, how would I do that?

I know this is going to sound condescending, but I really don't intend it
that way, it's just that there's no other way I can concisely make my point
as strong as it needs to be made in response to your statement:

If you don't know how to "view the internal structure of [a] database", then
you simply ought not be trying to do that.

[psnarula]

"Lawrence Garvin [MVP]" wrote:

> "psnarula" <psna...@discussions.microsoft.com> wrote in message
> news:7C5B2AC5-FA01-4DCD...@microsoft.com...
> > "Lawrence Garvin [MVP]" wrote:
> >
> >> did you activate these products using *MSDN* Product Keys?
> >
> > Yes.
>
> Well... then... there ya have it. You're using an MSDN licensed product for
> non-development use, and that's a violation of the MSDN licensing terms.

I would argue that everything in the lab we have (including the WSUS
machine) is being used for development use. It's an isolated network with no
production-style applications or servers. The only reason I want WSUS in the
lab is because I want to test MOICE on a network that is not connected to the
internet. And MOICE can't be enabled unless a machine is patched. I have
tried installing patches manually but MOICE still is not functioning properly
so clearly I'm missing some patch(es). At the time, WSUS seemed like the
easiest solution. Now I'm not so sure.

> If you don't know how to "view the internal structure of [a] database", then
> you simply ought not be trying to do that.

I was more curious about how it would be done than actually planning on
making an attempt myself. If WSUS would stop thinking that he needed to
download more files, I woudln't be giving a second thought to the internal
structure of the database.

I think I'm going to blow it away and try again.

Paul

[psnarula]

"psnarula" wrote:

> I think I'm going to blow it away and try again.

I reinstalled Windows 2003 today. This time I used Windows Server 2003 R2
SP2 instead of the 64-bit edition. I'm just trying to mimic the
internet-connected machine as much as possible. And since it is not running
the 64-bit version of Windows 2003, neither is my lab machine (anymore). I
partitioned the hard drive so that now I have a D drive for storing the WSUS
content. Once Windows 2003 was installed, I joined the machine to my domain,
installed the necessary prerequisites, and installed WSUS 3.0 to D:\ with a
new instance of the Windows Internal Database. Then I used a tape backup
from the internet machine to restore update files to D:\WsusContent. Finally
I did "wsusutl import". The result was the same as always -- when I click on
an update, it tells me:

"This update cannot be approved for installation because its Microsoft
Software License Terms are still downloading"

-or-

"The files for this update have not yet been downloaded. The update can be
approved but will not be available to computers until the download is
complete".

This is becoming very frustrating. Seeing the exact same restore procedure
work on somebody else's offline lab network makes it even worse.

Paul

[psnarula]

I just wanted to post the solution to my problem (maybe somebody else will
benefit).

It turns out that all of my problems were related to the fact that the
internet-connected WSUS server didn't have the license files for Office XP.
All of our machines on the internet-connected network are running Office 2003
so I hadn't really noticed. If I had ever tried to approve an Office XP
update on the internet-connected network, I wouldn't have been able to do so
without first downloading the license files.

I poked around in the internal WSUS database and noticed that it has a queue
for BITS downloads which, by default, has size ten. On my offline WSUS
server, BITS was trying to download the missing license files and this was
filling the queue so WSUS was unable to "download" (ie, recongize the
presence of) the 24 gigs of updates that I had already imported from tape.
Before I realized that the missing license files were not on the
internet-connected server either, I just changed the size of this BITS queue
to 500 and then my WSUS server started "downloading" updates. Of course,
making this change on an internet-connected WSUS server could overwhelm the
network with download bandwidth. When the offline WSUS server was finished
seeing my updates, I was able to approve all updates except for the ones with
missing license files.

Doing "wsusutil reset" on the internet-connected WSUS server obtained the
missing files and everything works properly now (there is no need to edit the
internal database).

Paul