Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

pagefile.sys header

64 views
Skip to first unread message

Bill Cunningham

unread,
Oct 31, 2015, 2:41:40 PM10/31/15
to
I wondered if anyone knows anything about pagefile.sys's header. I
wanted to clear or zero out the contents of the pagefile and of course I
need to leave the header intact. I looked online and couldn't find anything.
I did learn though you can use the fsutil to encrypt pagefile.sys. I will
have to fiddle with it a bit.

Bill


VanguardLH

unread,
Oct 31, 2015, 5:13:37 PM10/31/15
to
Bill Cunningham wrote:

> I wondered if anyone knows anything about pagefile.sys's header. I
> wanted to clear or zero out the contents of the pagefile and of course I
> need to leave the header intact.

Why? If you are going to erase the pagefile, why do you need to leave
behind a skeleton file with a header? If the pagefile is missing,
Windows will recreate it on startup.

For example, you can configure Windows to zero out (clear); see
https://support.microsoft.com/en-us/kb/314834. If you delete the
pagefile.sys file, it gets recreated on Windows startup.

> I looked online and couldn't find anything. I did learn though you can
> use the fsutil to encrypt pagefile.sys. I will have to fiddle with it
> a bit.

The pagefile can only be cleared on shutdown to ensure there are no
processes that still have pages allocated to them. Clearing the
pagefile at any other time means removing data that current processes
may still need or destroying code that will cause the processes to
crash or worse misbehave. Similarly, the pagefile can only be defragged
during Windows startup before it starts to get used.

What would be the advantage of encrypting the pagefile? The instance of
Windows that is running would still have to read and write to the
pagefile so forensics would still be able to dig into that file looking
for nasty tidbits or passwords.

Bill Cunningham

unread,
Oct 31, 2015, 6:42:06 PM10/31/15
to

"VanguardLH" <V...@nguard.LH> wrote in message
news:d9kp7s...@mid.individual.net...
> Bill Cunningham wrote:
>
>> I wondered if anyone knows anything about pagefile.sys's header. I
>> wanted to clear or zero out the contents of the pagefile and of course I
>> need to leave the header intact.
>
> Why? If you are going to erase the pagefile, why do you need to leave
> behind a skeleton file with a header? If the pagefile is missing,
> Windows will recreate it on startup.

But not in the same place it was. It's suppose dto be at the beginning
of the disk.

> For example, you can configure Windows to zero out (clear); see
> https://support.microsoft.com/en-us/kb/314834. If you delete the
> pagefile.sys file, it gets recreated on Windows startup.
>
>> I looked online and couldn't find anything. I did learn though you can
>> use the fsutil to encrypt pagefile.sys. I will have to fiddle with it
>> a bit.
>
> The pagefile can only be cleared on shutdown to ensure there are no
> processes that still have pages allocated to them. Clearing the
> pagefile at any other time means removing data that current processes
> may still need or destroying code that will cause the processes to
> crash or worse misbehave. Similarly, the pagefile can only be defragged
> during Windows startup before it starts to get used.

Well I was meaning cleaning it from another OS so it wouldn't be being
used. Aotherwise no of course you couldn't touch it. Maybe it's not worth
it.

> What would be the advantage of encrypting the pagefile? The instance of
> Windows that is running would still have to read and write to the
> pagefile so forensics would still be able to dig into that file looking
> for nasty tidbits or passwords.

I read it was extra security to encrypt the pagefile. idk.

Bill


Bill Cunningham

unread,
Oct 31, 2015, 6:45:58 PM10/31/15
to

"VanguardLH" <V...@nguard.LH> wrote in message
news:d9kp7s...@mid.individual.net...

> For example, you can configure Windows to zero out (clear); see
> https://support.microsoft.com/en-us/kb/314834.

I didn't know that. Well I guess it's not needed then. This doesn't erase
the file does it?

Bill


Paul

unread,
Oct 31, 2015, 10:27:55 PM10/31/15
to
It would overwrite all the clusters of that file.

Deleting the pointer to the file doesn't clear it.
Overwriting all the clusters does clear it.

If the pagefile is large, this will extend the shutdown time.

Paul


VanguardLH

unread,
Oct 31, 2015, 10:36:33 PM10/31/15
to
Bill Cunningham wrote:

> VanguardLH wrote ...
>
>> For example, you can configure Windows to zero out (clear); see
>> https://support.microsoft.com/en-us/kb/314834.
>
> I didn't know that. Well I guess it's not needed then. This doesn't
> erase the file does it?

No, it zeroes it out (writes zeros into the file). The allocated
filespace remains.

Unless you know that the pagefile.sys file is at the beginning of the
disk (where is the fastest rotational speed of the disk), you should run
a defragger that includes the page file.

An old trick of trying to keep the pagefile from fragmenting is to set
it to a fixed size. Set the min and max size the same. Then Windows
allocates a fixed size to the file. Delete the current [fragmented]
pagefile.sys file (boot into the Recovery Console to delete or use a
deleter that can delete files on Windows startup) and reboot into
Windows. Then defrag to include the page file which moves it to the
beginning of the disk.

Also, if you have 2, or more, HDDs then you can leave a small segment
for the pagefile on the C: drive (on one HDD) and add a large segment
for the pagefile on the other drive (on the other HDD). The drives
cannot be on the same HDD. Each pagefile segment has to be on a
different physical hard disk in a partition there given a drive letter.
So, for example, if you have a C: drive on HDD 1 and a D: drive on HDD 2
then you would configure the pagefile to have a small segment on C: and
a big segment on D:. Windows will give priority to the pagefile segment
on an HDD other than where is Windows. This will eliminate some data
bus contention where Windows or apps are trying to access their HDD but
let Windows concurrently make writes to the pagefile on the other HDD.
If you have an SSD and that is where is Windows then just use the
pagefile on the much faster SSD.

VanguardLH

unread,
Oct 31, 2015, 10:38:24 PM10/31/15
to
Bill Cunningham wrote:

> VanguardLH wrote ...
>
>> Bill Cunningham wrote:
>>
>>> I wondered if anyone knows anything about pagefile.sys's header. I
>>> wanted to clear or zero out the contents of the pagefile and of course I
>>> need to leave the header intact.
>>
>> Why? If you are going to erase the pagefile, why do you need to leave
>> behind a skeleton file with a header? If the pagefile is missing,
>> Windows will recreate it on startup.
>
> But not in the same place it was. It's suppose dto be at the beginning
> of the disk.

Was not aware that was part of your unseen criteria. Get a defragmenter
that can include the pagefile; e.g., Piriform Defraggler and
SysInternals Page Defrag.

>> For example, you can configure Windows to zero out (clear); see
>> https://support.microsoft.com/en-us/kb/314834. If you delete the
>> pagefile.sys file, it gets recreated on Windows startup.
>>
>>> I looked online and couldn't find anything. I did learn though you can
>>> use the fsutil to encrypt pagefile.sys. I will have to fiddle with it
>>> a bit.
>>
>> The pagefile can only be cleared on shutdown to ensure there are no
>> processes that still have pages allocated to them. Clearing the
>> pagefile at any other time means removing data that current processes
>> may still need or destroying code that will cause the processes to
>> crash or worse misbehave. Similarly, the pagefile can only be defragged
>> during Windows startup before it starts to get used.
>
> Well I was meaning cleaning it from another OS so it wouldn't be being
> used. Aotherwise no of course you couldn't touch it. Maybe it's not worth
> it.

You couldn't be sure about what to delete, so deleting the file would be
the recommended method of "clearing" it while it was quiescent (not yet
in use).

Why is zeroing out the file not sufficient (for your unspecified needs)?

>> What would be the advantage of encrypting the pagefile? The instance of
>> Windows that is running would still have to read and write to the
>> pagefile so forensics would still be able to dig into that file looking
>> for nasty tidbits or passwords.
>
> I read it was extra security to encrypt the pagefile. idk.

Not from the instance of Windows where you chose to encrypt it. Perhaps
you should be looking at whole-disk encryption if you are concerned
about someone peeking in or using forensics on your computer. Any
encryption incurs overhead (for the decryption and re-encryption) so
your computer will be slower.

You don't get Bitlocker in any version of Windows XP but you can zero
the pagefile on shutdown (the clearing action will make shutdown take
longer) and you can use TrueCrypt for whole-disk encryption or just
create encrypted containers that load like drives.

TrueCrypt still works but its nebulous author(s) scattered when the FBI
sent them a "letter". Truecrypt's web page turned into a warrant canary
(https://en.wikipedia.org/wiki/Warrant_canary). The FBI's "national
security letter" legally bars the victim to reveal getting served with
such a letter, so those served may circumvent by issuing warrant
canaries to indicate what really happened. The TrueCrypt authors (who
were never known) crippled the latest version to read-only of existing
TrueCrypt containers. You need to get the prior version (7.1). You can
also do online searches on "whole disk encryption" if you want no one to
get at the contents of a drive. Bitlocker was Microsoft's corporate
oriented approach to protecting business computers when stolen.

Some whole-disk encryption programs are free, some are payware. PGP
Whole Disk Encryption got acquired by Symantec and renamed to Endpoint
Encryption. There's Sophos Safeguard and DriveCrypt. All of those are
payware. BestCrypt is not free except for its portable version
(http://www.jetico.com/products/free-security-tools/bestcrypt-traveller);
however, it does not have all the features of the non-crippled full
version plus it looks to only support containers, not whole-disk
encryption, and may only support "lite encryption" to create encrypted
containers.

If you don't like that the author(s) scattered and abandoned TrueCrypt
(for reasons that only be guessed), VeraCrypt is a fork of TrueCrypt.
Since TrueCrypt was open source, someone else decided to pick up the
program and continue supporting it. I haven't delved into VeraCrypt
since TrueCrypt is still usable. I only use TrueCrypt to use encrypted
containers (the file is encrypt, mounted as a drive, and the contents
decrypted to be accessible). As I recall, the usurpers of TrueCrypt
were based in a country outside the FBI's reach with their national
security letters and secret DOJ judges forcing a vendor to add a
backdoor to the encryption product. Although Veracrypt is hosted at
Codeplex.com (owned by Microsoft), they maintain a code repository
elsewhere to ensure the FBI and Microsoft don't interfere.

mike

unread,
Oct 31, 2015, 11:11:40 PM10/31/15
to
On 10/31/2015 7:36 PM, VanguardLH wrote:

> Unless you know that the pagefile.sys file is at the beginning of the
> disk (where is the fastest rotational speed of the disk), you should run
> a defragger that includes the page file.

What are the advantages of a pagefile.sys???
I've run win7 with 2GB of RAM and without a pagefile
for months at a time.
Only time I ever got an out of memory error was when I tried
to run two concurrent instances of virtualbox.

Since I sometimes wanted to do that,
I put the pagefile back on the secondary partition.
It's slower, but if it's never used, does it matter?
At least the big waste of space is not on the boot partition.
What am I missing?
Inquiring minds want to know???



VanguardLH

unread,
Nov 1, 2015, 1:31:08 AM11/1/15
to
mike wrote:

> VanguardLH wrote:
>
>> Unless you know that the pagefile.sys file is at the beginning of the
>> disk (where is the fastest rotational speed of the disk), you should
>> run a defragger that includes the page file.
>
> What are the advantages of a pagefile.sys??? I've run win7 with 2GB
> of RAM and without a pagefile for months at a time.

Depends on what you have loaded. I use Firefox (and have used Google
Chrome) and they can consume gobs of memory. I've used Sysinternals
Process Monitor to log events and it can use gobs of memory over time
(because it's recording all events despite I have a filter on just I
want to see). I have 8 GB of system RAM and just the other day I had
7.8 GB of it consumed (I don't recall how much pagefile was used). So I
could easily outstrip just 2 GB. Depends on how you use your computer.

> Only time I ever got an out of memory error was when I tried to run
> two concurrent instances of virtualbox.

As I recall, you can assign a fixed memory to a VM in VirtualBox so the
guest OS doesn't run out of memory. With just 2 GB, you have memory
consumed by Windows, your host OS processes, and each guest OS in a VM.
If I started using VMs again, just 2 would not suffice.

> Since I sometimes wanted to do that, I put the pagefile back on the
> secondary partition.

A segment of the pagefile that is in another partition on the SAME hard
disk as the OS partition where you also have a segment of pagefile will
not afford you any speed up. The conflict between read and writes to
the pagefile are device bound, so having multiple segments of the
pagefile on the SAME disk won't let you read and write to the pagefile
at the same time.

If you have multiple segments to the pagefile, each must be on a
different hard disk. Otherwise, you've wasted space in a partition on
the same hard disk as the partition for the OS where is a pagefile. If
you only have one hard disk, just use 1 segment for the pagefile and in
the OS partition.

> At least the big waste of space is not on the boot partition.

Partition 1 on HDD-1 with a pagefile and partition 2 on the same HDD-1
will afford you not disconnection between parallelizing reads and writes
to the pagefile segments. The contention is disk bound.

Do you have 2 hard disks or only 1? If 2 HDDs then you can split up the
pagefile. If you only have 1 HDD then don't bother slicing up the
pagefile.

mike

unread,
Nov 1, 2015, 7:33:47 AM11/1/15
to
I said nothing about slicing the pagefile. I took it off C: and put it
on D:. ONE drive.
The only reason it's there at all is in case I decide to use two
VMs again. It's not on the fastest part of the drive, but if it's
not used much, it shouldn't matter.
I put as little as possible on C: so I can image the partition
easily and quickly.

Bottom line is that if you don't have enough memory to do what you do,
more RAM is the solution. Speed of the pagefile is irrelevant if
it's not used.

Ken Blake, MVP

unread,
Nov 1, 2015, 9:27:13 AM11/1/15
to
On Sat, 31 Oct 2015 20:10:55 -0700, mike <ham...@netzero.net> wrote:


> What are the advantages of a pagefile.sys???
> I've run win7 with 2GB of RAM and without a pagefile
> for months at a time.
> Only time I ever got an out of memory error was when I tried
> to run two concurrent instances of virtualbox.
>
> Since I sometimes wanted to do that,
> I put the pagefile back on the secondary partition.
> It's slower, but if it's never used, does it matter?
> At least the big waste of space is not on the boot partition.
> What am I missing?
> Inquiring minds want to know???


1. If you don't have a page file, you can't use all the RAM you have.
That's because Windows preallocates virtual memory in anticipation of
a possible need for it, even though that allocated virtual memory may
never be used. Without a page file, that allocation has to be made in
real memory, thus tying up that memory and preventing it from being
used for any purpose.

2. There is never a benefit in not having a page file. If it isn't
needed, it won't be used. Don't confuse allocated memory with used
memory.
>
>

VanguardLH

unread,
Nov 1, 2015, 2:05:08 PM11/1/15
to
As mentioned, segmenting the pagefile is a per-device scheme. Moving it
around to any partition on the same disk won't provide any performance
boost.

Not having a pagefile is not recommended. Some programs require it.
Even Windows requires it. Not all processes must have all data in
memory at the same time. Apps may store data in the pagefile which is
not currently being accessed.

> The only reason it's there at all is in case I decide to use two
> VMs again. It's not on the fastest part of the drive, but if it's
> not used much, it shouldn't matter.
> I put as little as possible on C: so I can image the partition
> easily and quickly.

Most imaging products exclude the pagefile, hibernate file, temp
folders, and other files or folders that are not considered necessary to
include in an image. In fact, to include that fluff, you have to
configure the imaging program to perform sector-by-sector backups.

> Bottom line is that if you don't have enough memory to do what you do,
> more RAM is the solution. Speed of the pagefile is irrelevant if
> it's not used.

Not having a pagefile can slow Windows and apps. They expect the
availability of the pagefile to store data that is not currently inuse.
If the requests for pagefile space are rejected, they have to consume
more memory or run differently which usually means run slower. You
should have some pagefile space. Forcing what apps want to put in the
pagefile either into memory (to unnecessarily consume more memory than
it needs).

Windows is not the only controller paging out processes into the
pagefile. Apps can also request pagefile space. They expect it. Some
may not even run if there is no pagefile space available. Even if you
had 128 GB of system RAM, you still need a bit of pagefile space for
those processes that directly request it.

You only have 2GB of system RAM. Did you perform a minimal install of
Windows, disable unneeded services, eliminate startup programs, and run
only 1 application at a time? It only takes running Firefox with the
Adblock Plus add-on to end up consuming nearly 2 GB of memory on just
the firefox.exe process at some sites. If you run only 1 program at a
time (you don't multitask) then 2GB with no or little pagefile space
might work for you. For others running multiple programs, their
programs will start to crash usually with an initialization error (the
app couldn't get any of the pagefile space that it requested). Having
no pagefile or too small a pagefile is okay when you run just 1 or 2
apps and those are not memory hogs (immmediately or over time).

When you load Word or whatever editor you use capable of loading huge
documents, do you really want to consume all your system RAM by forcing
the loading of the huge document into system RAM? You aren't accessing
all that document at once so buffering allows viewing a huge document
without severely slowing down the application due to thrashing.

http://lifehacker.com/5426041/understanding-the-windows-pagefile-and-why-you-shouldnt-disable-it
http://www.howtogeek.com/126430/htg-explains-what-is-the-windows-page-file-and-should-you-disable-it/

If you are so tight on free disk space that eliminating or overly
undersizing the pagefile is your cure then you already know the real
cure is to get a bigger hard disk, or start removing all those data
files that you can put on removable media (optical discs, USB drives,
push to online storage if no sensitive data) along with uninstalling all
those nifty programs that you ended up never using.

Bill Cunningham

unread,
Nov 1, 2015, 4:16:18 PM11/1/15
to

"mike" <ham...@netzero.net> wrote in message
news:n13vp2$i0e$1...@dont-email.me...

> What are the advantages of a pagefile.sys???
> I've run win7 with 2GB of RAM and without a pagefile
> for months at a time.
> Only time I ever got an out of memory error was when I tried
> to run two concurrent instances of virtualbox.
>
> Since I sometimes wanted to do that,
> I put the pagefile back on the secondary partition.
> It's slower, but if it's never used, does it matter?
> At least the big waste of space is not on the boot partition.
> What am I missing?
> Inquiring minds want to know???

You know that's a good question. At one time a page or swap file or
partition was a good thing for swapping in and out memory pages. I don't use
a swap file or partition on my linux and it's fine. I am thinking about not
using one on windows. They're not used as much as they were at one time by
the OS. I don't know if it's more memory space or more efficancy in meory
management or want. Maybe some one else might be better able to answer your
question on that.

Bill


Bill Cunningham

unread,
Nov 1, 2015, 4:22:15 PM11/1/15
to

"Ken Blake, MVP" <kbl...@kb.invalid> wrote in message
news:l68c3bl8fobhrrmjb...@4ax.com...

> 1. If you don't have a page file, you can't use all the RAM you have.
> That's because Windows preallocates virtual memory in anticipation of
> a possible need for it, even though that allocated virtual memory may
> never be used. Without a page file, that allocation has to be made in
> real memory, thus tying up that memory and preventing it from being
> used for any purpose.
>
> 2. There is never a benefit in not having a page file. If it isn't
> needed, it won't be used. Don't confuse allocated memory with used
> memory.

Hum. Some apps require it. I didn't know that. You are right of course.

Bill


Bill Cunningham

unread,
Nov 1, 2015, 4:25:12 PM11/1/15
to

"Paul" <nos...@needed.com> wrote in message
news:n13t71$am0$1...@dont-email.me...
Well I see now that fsutil behavior doesn't allow pagefile encryption
with XP. I guess that's the newer windows OSes. Is there anyway t oencrypt
it another way on XP? Would I be worth it?
Bill


Micky

unread,
Nov 1, 2015, 4:45:35 PM11/1/15
to
[Default] On Sun, 1 Nov 2015 17:16:14 -0500, in
microsoft.public.windowsxp.general "Bill Cunningham"
<nos...@nspam.invalid> wrote:

>
>"mike" <ham...@netzero.net> wrote in message
>news:n13vp2$i0e$1...@dont-email.me...
>
>> What are the advantages of a pagefile.sys???
>> I've run win7 with 2GB of RAM and without a pagefile
>> for months at a time.
>> Only time I ever got an out of memory error was when I tried
>> to run two concurrent instances of virtualbox.
>>
>> Since I sometimes wanted to do that,
>> I put the pagefile back on the secondary partition.
>> It's slower, but if it's never used, does it matter?
>> At least the big waste of space is not on the boot partition.
>> What am I missing?
>> Inquiring minds want to know???
>
> You know that's a good question. At one time a page or swap file or
>partition was a good thing for swapping in and out memory pages. I don't use
>a swap file or partition on my linux and it's fine. I am thinking about not
>using one on windows. They're not used as much as they were at one time by
>the OS. I don't know if it's more memory space

That certainly makes a big difference. IIRC my first computer had
10megs of memory. My current one has 4000 times as much, but the
letters I write, the pictures I look at, and most other things haven't
gotten bigger. Videos might be 4000 times as big as anything I
looked at before, but iiuc streaming video files are marked for
deletion soon after I've watched that part of the video, and
non-streaming rely on the swapfile

I suppose many such non-streaming applications could keep going to the
original HDD location to bring into RAM the next few minutes of video,
but do any programs do that?

>or more efficancy in meory
>management

I'm in over my head here but doesn't more efficiency in memory
management equate to better use of the swapfile? For example, iiuc
now most programs read ahead, so that while you're reading page 21 to
25 of the file, the program foresees that you will soon need page 26
to 30, or maybe even 31 to 35, and in background it gets it from the
swapfile to RAM so that it's ready when you get there, and even if you
page ahead.

What kinds of recenty added efficiency in memory management would not
need the swapfile?

Micky

unread,
Nov 1, 2015, 4:50:34 PM11/1/15
to
[Default] On Sun, 1 Nov 2015 17:16:14 -0500, in
microsoft.public.windowsxp.general "Bill Cunningham"
<nos...@nspam.invalid> wrote:


Bill, are you unaffected by the return to Standard Time? Or maybe
you're affected but havent' changed your computer clock?

Because I just replied to you and yet it sorted in front of you, and I
notice your time here is 5:16, which is east of the eastern time zone.
Perhaps you're in Newfundland (sp?) or Prince Edward Island?

Although we both show -0500 for the time zone, and Agent is supposed
to convert times showing in the message list to my local time anyhow.
I'm confused.

Paul

unread,
Nov 1, 2015, 5:51:30 PM11/1/15
to
The only way to encrypt it, is with some other kind
of encryption solution.

Truecrypt for the entire partition.

A Seagate Momentus drive with full disk encryption.
(The tough part is not getting the disk, it's getting
some sort of software or BIOS code to work with it
properly. This kind of hardware solution is still
not being marketed to the public, only to
"system integrators" or larger OEM suppliers of
computers to government.)

That sort of idea.

And the thing is, those are comprehensive solutions, that
help cover all the other leakage mechanisms in the OS. So
they're desirable from that point of view.

Paul

Paul

unread,
Nov 1, 2015, 6:33:30 PM11/1/15
to
Micky wrote:

>
> I'm in over my head here but doesn't more efficiency in memory
> management equate to better use of the swapfile? For example, iiuc
> now most programs read ahead, so that while you're reading page 21 to
> 25 of the file, the program foresees that you will soon need page 26
> to 30, or maybe even 31 to 35, and in background it gets it from the
> swapfile to RAM so that it's ready when you get there, and even if you
> page ahead.

Then you haven't seen just how ridiculous this is getting :-)

On the Win10 machine, I have a copy of Macrium, It has
a conversion routine, to convert a .mrimg backup file,
to a .vhd file.

I open Task Manager and watch.

The operation starts with a lot of prefetch. Around
5GB of memory is actually booked by the operation.

The destination drive is slower than the source drive.
The output file will be about the same size (the .vhd
is about the same size as the .mrimg, if the .mrimg
wasn't compressed).

The booked memory continues to increase.

Soon, 10GB of memory is used, some for read prefetch,
some for write cache.

Eventually, the program is done. I click quit.

I look over at the hard drive status LED. It's
still lit and going full tilt. The Task Manager
memory thing indicates that 5GB of memory is
"draining" and that is what keeps the hard drive
light running. When this caching mechanism gets near
the end, it stops running the disk full tilt. It
"burps" out smaller write operations in pulses.
The write activity at the end, is a declining
write curve.

Eventually, the disk drive settles down, and it
looks like the caching mechanism is now drained.

So if a person was measuring the "time to complete"
the operation, it would be from clicking the button
to start the operation in Macrium, until the last
"write burp" to the drive.

Well, how much does that gain us ?

The operation cannot go faster than the destination
drive is willing to go (in this case). At some point,
either the source disk or the destination disk is
an issue.

*******

When the first desktop computers existed, there
wasn't any overlapping I/O. Certainly, on a dual
floppy drive machine, you could blame having only
one floppy controller and two drives on the cable for
it. But the software was also blocking the operations,
and only allowing one outstanding operation at a time.

+----------+ +----------+
| Read #1 | | Read #2 |
+----------+-----------+----------+-----------+
| Write #1 | | Write #2 |
+-----------+ +-----------+

Later, OSes like Windows acquired non-blocking
operations, intended to support overlapped I/O.
It was up to the application to make the right calls,
so many programs continued to do it the old way.
The first program I saw here, to do overlapped
I/O, was Robocopy.

x <------ Program running -------> X

+----------+-----------+
| Read #1 | Read #2 |
+----------+-----------+-----------+
| Write #1 | Write #2 |
+-----------+-----------+

With the large prefetch and large write buffer case
I've seen just recently, I'm not going to try to do
an ASCII Art diagram of that, but in essence, the
difference is like this. The program running portion
can appear shorter, but some hardware is still huffing
and puffing after the fact.

x <- Program running -> X

+----------+-----------+
| Read #1 | Read #2 |
+----------+-----------+-----------+
| Write #1 | Write #2 |
+-----------+-----------+
X <-Cache-> X
Drains

I'm having trouble seeing whether this new behavior
is a big win or not. This could be due to the
application using MapViewOfFile(), but I can't really
be sure of that.

So yes, there are instances of prefetch going on.
Even Explorer in Win10 attempts prefetch, as it
affects the appearance of the progress graph during
a file copy.

There are, in fact, a couple of RAM buffering options.
If you read a file, the contents are left in memory.

md5sum file.txt
md5sum file.txt

On the first run, the command gobbles data at 100MB/sec.
It is limited by the disk drive.

On the second run, it gobbles data at 300MB/sec. Why ?
The system file cache (which can use all unallocated
memory), holds a copy of the file. As long as the
file system is convinced the cached copy is the latest,
and nothing has purged the system file cache, you see
a performance speedup. The king at this, was Win2K, where
the system file cache was every bit as good as the
competing ones (SunOS or Solaris may have had this
well before any desktop OS, MacOSX has a good system
file cache too). The modern Windows ones, find more
excuses not to use it. It's still there though. For
example, if you defragment, the defragmenter will not
refer to any files contained in the system file
cache. It does read_uncached() instead, for "safety".

This is separate from the MapViewOfFile or similar concept.
The memory in that case is "charged to the system"
and you can see the activity in Task Manager. Whereas
the system file cache, there isn't a visual representation
for it. So in fact, some MapViewOfFile activity, as
it acquires RAM and is charged for it, that could be
purging a portion of the system file cache.

In short, there is lots going on behind the scenes.
More than I can keep track of. And some of it
is downright silly. It distorts progress bars (when
a file copy pre-fetches part of the copy from the
system file cache) and also makes dangerous situations
(from the user perspective), when 5GB of write cache memory
drains to disk and takes a whole minute to do it.
Any buffering on writes, should be short enough
so the bad battery on my UPS isn't an issue on
a power failure (power drops, before the 5GB of writes
are done).

Most of the time, on a modern OS, when I look
at the file transfer graph, my mouth is open
and I have that "WTF" look on my face. Because
the numbers in the graph are nonsense, and the
usage of RAM for stuff is a root cause. But many
times, things can't go any faster than the slowest
hard drive, so it's all a merry joke.

And if you ever see a drive deliver only half
of what you were expecting, check the "alignment".
I had a 4K sector hard drive, where I had to
realign it, to get the damn thing to run at
the proper speed.

Paul

Bill Cunningham

unread,
Nov 2, 2015, 3:47:28 PM11/2/15
to

"VanguardLH" <V...@nguard.LH> wrote in message
news:d9n62r...@mid.individual.net...
> mike wrote:
>
>> On 10/31/2015 10:30 PM, VanguardLH wrote:
>>> mike wrote:
>>>
>>>> VanguardLH wrote:

...

Well Those were nice links. I have my pagefile.sys set at 1341 and I guess
that is MB. It never seems to get any larger. So I set it at that size. I
will defrag it and I don't think it will need it again.

What about encrypting it with an outside encryption program? I don't
know if NTFS's encryption would encrypt it. Could XP still use it if it's
encrypted?

Bill


VanguardLH

unread,
Nov 2, 2015, 8:45:07 PM11/2/15
to
Bill Cunningham wrote:

> What about encrypting it with an outside encryption program? I don't
> know if NTFS's encryption would encrypt it. Could XP still use it if it's
> encrypted?

Already mentioned: whole-disk encryption. See my other reply that
mentions it.

VanguardLH

unread,
Nov 8, 2015, 7:48:17 AM11/8/15
to
VanguardLH wrote:

> For example, you can configure Windows to zero out (clear); see
> https://support.microsoft.com/en-us/kb/314834. If you delete the
> pagefile.sys file, it gets recreated on Windows startup.

I wasn't sure how you could delete the pagefile. I suspect it is
protected because it is always inuse by Windows. Rather than waiting
the long time to zero out all the pages in the pagefile, I read (but
have not tried) that you can tell Windows to delete the pagefile (and
have it create a new one on its next load) by running:

wmic pagefileset where name="C:\\pagefile.sys" delete

The double backslash is probably needed for parsing (i.e., escaping a
character which is done by prefixing it with a backslash, so you use a
backslash to escape a backslash). WMI is Windows Management Instruction
(https://en.wikipedia.org/wiki/Windows_Management_Instrumentation) and
wmic.exe is the console-mode command to interface with WMI.

Of course, per your concern, the new pagefile will be allocated to the
available free space in the file system on the next Windows load, so its
sector positioning could change (and why you might have to use a defrag
to move it back to the the beginning [outside] of the platter).
0 new messages