RAM Reg EWF with hibernation questions
Nightman
still had some questions yet.
1. Now I can boot sytem with one clear hibernatefl.sys everytime, but how
can I update my EWF system, etc. adding some folds and files, install drivers
for perpherals ? If I commit these changes to HD, I can't ensure I still boot
up with the original clear hibernatefl.sys.
2. Yes, I can do some changes on protected volume by using "xpepm
-hibernate" without disable EWF. But where these changes to be stored, as I
found there's no changes occur to hibernatefl.sys file, even with the size
and date.
Thanks!
Sean Liming (eMVP)
the EWF APIs say like EWFCommitFile and then re-hiberate.
Regards,
Sean Liming (eMVP)
Managing Director
SJJ Embedded Micro Solutions
www.sjjmicro.com / www.seanliming.com
Author: Windows XP Embedded Advanced and Windows NT Embedded Step-by-Step
"Nightman" <Nigh...@discussions.microsoft.com> wrote in message
news:6D102D59-4421-4EED...@microsoft.com...
KM
"committed state".
KM
Nightman
sure I can boot up system with the original fresh hibernate file. It againsts
the purpose of "Hibernate once/resume many". Can I commit only the pointed
change which I need, instead of all the accumulated changes?
Thanks.
KM
Committing the EWF volume commits the hibernation file. This zeroes out the first page of the hibernation file. This will cause the
system to boot normally and not from the hibernation file on the next boot even if it is a resume.
So you need to reboot after the EWF Commit.
KM
KM
See my response above. Basicaly, you have to reboot after you commit the changes and after the reboot to hibernate the image again
to use HORM.
Another option for you would be using resmany.dat
(http://msdn.microsoft.com/library/en-us/xpehelp/html/xetskEnablingHibernateOnceResumeManyEnvironmentWithEWF.asp). I never used or
tested it so I can't tell you how it works.
There is also a new feature of EWF on SP2 where you can commit some files (very restricted so you will unlikely use it). Read more
about it in XPe docs.
Another approach for you may be having an unprotected partition to store the files that needs to be changed often. But then please
keep in mind that you have to dismount that unprotected volume before you hibernate the image for HORM. Otherwise you can damage the
FS on that volume.
KM
KM
> about it in XPe docs.
I was referring here to this: http://msdn.microsoft.com/library/en-us/xpehelp/html/xelrfEwfMgrCommitFile.asp
KM
Nightman
Unfortunately, most of my possible changes on RAM Reg EWF system were
releated to windows system settings, etc static TCP/IP, computer name,
printers properties ... How can I commit one registry file then?
Slobodan Brcin (eMVP)
You have tough questions here, but let me give you basics then we can discuss what you must do depending on what you need.
1. EWF here is not your enemy it is your friend. You can't add new files to OS partition not because of EWF but because of content
that is stored in hiberfile and that content will be used during the boot. You can't change that content so as you can see it is a
problem.
2. All config info can be placed by you on unprotected and unmounted FS on second partition.
3. Regarding hibernation file read what Konstantin and Sean have said.
In theory you could update your system without the reboot, and then just do one reboot so your system apply this changes.
This would go something like this.
1. You have deployed few clients with same HORM content on disk.
2. You of course have a master XPe.
3. You do update of XPe image as suggested by Konstantin in you lab.
4. You access this disk offline with your application that will do following.
Prepare application:
- It should log all OS partition sector differences between new master disk and disk that is used on clients. (But you can exclude
unallocated sectors by FS to save some size).
- Additionally you should mark sectors that do not belong to hibrfile. (I mean you should save hiber file changes, also you should
figure out format of hibr file so you don't save full length of it just needed content)
Update application that will be executed on clients:
- Now we need to trick EWF and make a file system updates. First you would like to read content of each FS sector that we need to
update, and then write it back. This will force RAM EWF to cache this content in memory for successive OS usage.
- With previous step we cleared path for updating FS trough disk access with new sectors. (As long as OS is not rebooted we will use
virtual data from EWF overlay instead of changed data from disk so XPe will be ignorant of change)
- It should read hibr file data from your update file and overwrite the hibrfile on disk trough disk access. (This bypass EWF and
since data is not used during the XPe work it won't crash)
- Reboot computer without committing or hibernating.
Huh, this is it sound simple ?
Regards,
Slobodan
"Nightman" <Nigh...@discussions.microsoft.com> wrote in message news:6D102D59-4421-4EED...@microsoft.com...
KM
I have never tried that but you can give it a try.
--
Regards,
KM, BSquare Corp.
Slobodan Brcin (eMVP)
Even if he could commit one registry file or any other file he would then have a integrity problem after resume. (We are still
talking about hibernation in this thread right?)
Regards,
Slobodan
"KM" <konstmor@nospam_yahoo.com> wrote in message news:Ohzdw3C9...@TK2MSFTNGP09.phx.gbl...
KM
No, I think Nightman has asked about just committing his particular changes but not committing the entire overlay.
At least that was my assumption :-)
I certainly agree that this should not be used with hibernation feature unless the hiberfil.sys file is updated (either by
hibernating after a reboot, or using the procedure outlined in your other post in this thread).
Sean Liming (eMVP)
with HORM, but re-hibernation is the only solution that I can see.
Regards,
Sean Liming
Managing Director
SJJ Embedded Micro Solutions
www.sjjmicro.com / www.seanliming.com
Author: Windows XP Embedded Advanced and Windows NT Embedded Step-by-Step
"KM" <konstmor@nospam_yahoo.com> wrote in message
news:uuxzBdD9...@TK2MSFTNGP09.phx.gbl...
Slobodan Brcin (eMVP)
What do you think about the way that I have described that would allow update without rehibernation?
Regards,
Slobodan
"Sean Liming (eMVP)" <se...@seanliming.com.NOSPAM> wrote in message news:uy$fk4F9E...@tk2msftngp13.phx.gbl...
KM
Actually, there is so many interesting things that could be done in embedded space if we were having a way to change some
implementation or extend NT Loader, NT Kernel and EWF. However, none of them expose APIs (except the kernel but as we all know -
very minimum available).
Imagine ntldr extensions. Just pieces of code that ntldr calls in to (jumps to) on certain operations (before, during, after). That
could be some little programs (real/protected mode) under the root that, if found, are called from ntldr. E.g., instead of just
searching and checking resmany.dat (btw, has anybody confirmed this is working with XPe SP2?) the ntldr could load it up and jump to
a particular instruction by a specified offset. If this was well documented that could open many possibilities for extending XPe OS
features including the HORM.
The same store for EWF.sys, or some MS apps like SUS (WU Client), etc. If SUS client provided a way to call in a plug-in Dll, the
integration with EWF would be easier.
When it comes to considering XPe as a true embedded OS, it stumbles over the fact that XPe based on XP Pro binaries and XPe system
components cannot (or, better say, hard and time consuming to) be changed to meet some embedded needs.
--
Regards,
Konstantin
Sean Liming (eMVP)
It is interesting, and it sounds possible so long as the application being
updated doesn't go over the sectors of the older file. Or is there a method
to put extra sectors somewhere else. I not sure how you created the new
hiberfile to replace the old one. Step missing? Or did I read this wrong?
As far as empty space not used in the hiberfile, how is that recovered for
use?
Sounds like a nice low level sector app! When are you going to have it
finished ;)
Regards,
Sean
"Slobodan Brcin (eMVP)" <sbr...@ptt.yu> wrote in message
news:OJ5$NAG9EH...@TK2MSFTNGP12.phx.gbl...
Slobodan Brcin (eMVP)
Good suggestions, yup I agree that it would give us nice start point for making some beautiful features :-)
Merry Christmas,
Slobodan
"KM" <konstmor@nospam_yahoo.com> wrote in message news:Odp1pCG9...@TK2MSFTNGP14.phx.gbl...
Slobodan Brcin (eMVP)
I actually did not made such application, also I'm not working on making one. I just told this as a theoretical consideration to
what can be done.
> I not sure how you created the new
> hiberfile to replace the old one. Step missing? Or did I read this wrong?
You should do in lab complete process of updating and rehibernating XPe. After that you create update file that you will use for
updating deployed XPe systems.
> As far as empty space not used in the hiberfile, how is that recovered for
> use?
I mentioned this only for case of making smaller update image file. (It will be one huge file but it does not have to be size of
full memory, this is why I mentioned this.)
> It is interesting, and it sounds possible so long as the application being
> updated doesn't go over the sectors of the older file. Or is there a method
> to put extra sectors somewhere else.
Idea is to trick EWF to cache all sectors that we will update in RAM overlay and then we go around EWF and write update directly to
disk medium, so this way OS will be ignorant of what we are doing to it :-)
I hope that this clear idea a bit.
Regards,
Slobodan
"Sean Liming (eMVP)" <se...@seanliming.com.NOSPAM> wrote in message news:uTDrzQG9...@TK2MSFTNGP14.phx.gbl...
KM
Merry Christmas! :-)
Konstantin
Sean Liming (eMVP)
General Software has something called FirmBase that could be used to do
these low level tasks, but it doesn't match with your suggestion.
Regards,
Sean
"Slobodan Brcin (eMVP)" <sbr...@ptt.yu> wrote in message
news:OoDo5RG9...@TK2MSFTNGP15.phx.gbl...
Sean Liming (eMVP)
After thinking about this a bit...
Isn't the real problem here the FS information stored in the hiberfile?
Let me spin the idea a different way:
Commit the file to the disk using EWFMGRCommitFile - I would assume that
this updates the FS tables on the disk. Then it should be a matter of update
the FS table in the hiberfile with the update FS tables on the disk. Would
that work?
Regards,
Sean
"Slobodan Brcin (eMVP)" <sbr...@ptt.yu> wrote in message
news:ekSY0hG9...@TK2MSFTNGP12.phx.gbl...
Slobodan Brcin (eMVP)
> Isn't the real problem here the FS information stored in the hiberfile?
Of course it is, this was always a problem that prevent updates and that need from us to use stateless operations.
> Commit the file to the disk using EWFMGRCommitFile - I would assume that
> this updates the FS tables on the disk. Then it should be a matter of update
> the FS table in the hiberfile with the update FS tables on the disk. Would
> that work?
First problem here would be how can we update hibrfile. (We do not know anything about compression and data structure in it.)
Second problem would be that even if we could update FS cached in hiberfile we could not update information's about open files hold
by applications/driver that were running when we made an original image. So my best guess is that this approach is impossible even
in theory.
BTW: Let me simplify description of my approach so developers that read this in future can understand it: (I will abstract things so
that no optimizations are mentioned here.)
Field device update procedure.
Step 1. On running XPe we force RAM EWF driver to load whole protected partition data to memory. So all further read request will be
satisfied from memory instead from disk.
Step 2. We use image file of new/updated XPe and write it on disk level (bypassing EWF) over the current OS partition.
After the reboot we will have brand new OS that will do what ever it is made to do. So basically we replaced whole underlying OS.
Previous posts were explaining how can we minimize image size and RAM overlay usage during these operations.
Regards,
Slobodan
"Sean Liming (eMVP)" <se...@seanliming.com.NOSPAM> wrote in message news:uoIfz%23G9EH...@tk2msftngp13.phx.gbl...
KM
One question I have that may be easy for you to answer.
You are going to bypass EWF to update the hibernate file through the direct disk access.
So instaed of the scenerio you oulined below, wouldn't that be easier to update all the changed sectors the same way (direct disk
access) and then just reboot. The update will include all the sectors that were different between the master image and current
client image and the hiber.sys file sector that were changed.
Also, another question to understand your scenerio.. You basically use EWF overlay in RAM to cover all the changed files (actually
sectors) so that on next boot it won't matter the differences between real FS and the FS cache in hiber file, right?
Anyway, if you are going to make a few such update later on, aren't you going to end up with the whole FS redirected to EWF RAM
overlay? And, therefore, the EWF RAM usage would be huge and this all may not worth the efforts?
Interesting ideas, btw :-)
Konstantin
Slobodan Brcin (eMVP)
> You are going to bypass EWF to update the hibernate file through the direct disk access.
> So instaed of the scenerio you oulined below, wouldn't that be easier to update all the changed sectors the same way (direct disk
> access) and then just reboot. The update will include all the sectors that were different between the master image and current
> client image and the hiber.sys file sector that were changed.
Easier, of course. But I would like to keep image file size on minimum, and that require some file analysis.
We have three different types of regions on disk.
1. Data area occupied by hiberfile
2. FS area with valid data in it.
3. FS available clusters of data not assigned to any file in FS.
If we have 1GB of ram hiberfile would be 1GB. But internal hiberfile usage could be less than 50 MB. And we would need only to save
valid data to image file. (Or some other form of specific analysis of this file could be also used)
> Also, another question to understand your scenario.. You basically use EWF overlay in RAM to cover all the changed files (actually
> sectors) so that on next boot it won't matter the differences between real FS and the FS cache in hiber file, right?
After reboot both FS and hiber file would be updated. Also steps 2 and 3 would allow us not to overload RAM overlay with sectors
that are not used by FS.
> Anyway, if you are going to make a few such update later on, aren't you going to end up with the whole FS redirected to EWF RAM
> overlay? And, therefore, the EWF RAM usage would be huge and this all may not worth the efforts?
No, I would not :-)
I do not expect heavy usage of RAM overlay during this procedure at all. (Unless you update your image from SP1 to SP2)
Regards,
Slobodan
KM
> > You are going to bypass EWF to update the hibernate file through the direct disk access.
> > So instaed of the scenerio you oulined below, wouldn't that be easier to update all the changed sectors the same way (direct
disk
> > access) and then just reboot. The update will include all the sectors that were different between the master image and current
> > client image and the hiber.sys file sector that were changed.
>
> Easier, of course. But I would like to keep image file size on minimum, and that require some file analysis.
>
> We have three different types of regions on disk.
> 1. Data area occupied by hiberfile
> 2. FS area with valid data in it.
These two above I was actually referring to. But you are right, that would require some FS analysis.
> 3. FS available clusters of data not assigned to any file in FS.
>
> If we have 1GB of ram hiberfile would be 1GB. But internal hiberfile usage could be less than 50 MB. And we would need only to
save valid data to image file. (Or some other form of specific analysis of this file could be also used)
Because of compression it could be hard to capture only changed blocks in hiber file unless you know the hiber file format.
> > Also, another question to understand your scenario.. You basically use EWF overlay in RAM to cover all the changed files
(actually
> > sectors) so that on next boot it won't matter the differences between real FS and the FS cache in hiber file, right?
>
> After reboot both FS and hiber file would be updated. Also steps 2 and 3 would allow us not to overload RAM overlay with sectors
> that are not used by FS.
I probalby missed the step from your list where you update the files (to be commited) on the disk through direct access.
> > Anyway, if you are going to make a few such update later on, aren't you going to end up with the whole FS redirected to EWF RAM
> > overlay? And, therefore, the EWF RAM usage would be huge and this all may not worth the efforts?
>
> No, I would not :-)
> I do not expect heavy usage of RAM overlay during this procedure at all. (Unless you update your image from SP1 to SP2)
I have just re-read your previous message (with fresh head now :-) ) and finally found the missing peice because of which I could
not undertand why you are trying to "trick EWF".
I understand it now. You basically just want to create a situation where any OS file reads/writes to the sectors that we need to
update on the client will go to EWF overlay. Then any changes that go directly to the disk for those sectors won't break anything
from OS/FS point of view because FS will continue using EWF RAM OVerlay version of these sectors during this session anyway.
With that in mind, my question about the EWF RAM usage does not make much sense :-)
Interesting solution.. Although I still believe this would worth efforts if you could lower the number of sectors to be updated in
hiber.sys file. Who wants to dowload 1G file when you can just create it locally (and this will add one more reboot).
Btw, you coudl use your idea in SP1 where EWF does not support committting particular files (sectors).
Right now in the filed you have to do a reboot on the client to clean up the EWF overlay from all the unnecesary user changes. then
you do the admin required changes and commit them to the disk, do the reboot again.
Instead of all this, we can just "trick the EWF" following your method and then update the changed sectors directly on disk level
and do only one reboot.
Thanks,
Konstantin
Slobodan Brcin (eMVP)
Yup you understood it correctly :-)
> Interesting solution.. Although I still believe this would worth efforts if you could lower the number of sectors to be updated in
> hiber.sys file. Who wants to dowload 1G file when you can just create it locally (and this will add one more reboot).
Hey I just hope that someone from MS will do all this ;-) After all they have access to hiberfile specifications and DUA server
components already available, they could integrate all this.
If not I'm certain that with little effort we could figure out layout of hibernation file, but since I do not need this option at
the moment, I guess that this will stay only as an idea for a XPe improvement,
Regards,
Slobodan
"KM" <konstmor@nospam_yahoo.com> wrote in message news:eRSEKAQ9...@tk2msftngp13.phx.gbl...
KM
> > Interesting solution.. Although I still believe this would worth efforts if you could lower the number of sectors to be updated
in
> > hiber.sys file. Who wants to dowload 1G file when you can just create it locally (and this will add one more reboot).
>
> Hey I just hope that someone from MS will do all this ;-) After all they have access to hiberfile specifications and DUA server
> components already available, they could integrate all this.
Ah, that's your idea :-)
> If not I'm certain that with little effort we could figure out layout of hibernation file, but since I do not need this option at
> the moment, I guess that this will stay only as an idea for a XPe improvement,
For the moment all I remember is that kernel compressing very 64Kb chunk of the hiber file to have the overlap between the
compression and disk I/O operations. The same on resume.
Any compression algorithm coould be reverse engineering if you have a sample file (although, it is just a compression, not no
encryption) but then it will stumble over legal aspects on using the final solution.
So I guess you are right - this is MS call.
Regards,
Konstantin