hobocopy fails because a file is in use
Graeme Perrow
up my Thunderbird profile directory. I have set up my script to do a
full backup every Monday and an incremental backup on other days. When
doing an incremental backup while Thunderbird is running, I get this
error:
Unable to open file \\?\GLOBALROOT\Device
\HarddiskVolumeShadowCopy4\Documents and Settings\gperrow\Application
Data\Thunderbird\Profiles\yz85hgzm.default\abook.mab to retrieve file
size. Error was The process cannot access the file because it is being
used by another process.
(Error number 32).
I am using the following command:
c:
cd "\Documents and Settings\gperrow\Application Data\Thunderbird\"
hobocopy /r /skipdenied /incremental /statefile=G:\backups\current
\hobocopy-state Profiles G:\backups\current\c\Thunderbird\Profiles
If I shut down Thunderbird and then run it, it works fine, but the
point of using hobocopy is that I should be able to back it up without
shutting down Thunderbird. I know I had this working in the past, but
I think I was just doing full backups all the time. How can I solve
this?
Craig Andera
So, you're right: not having errors like this is sort of the point of
hobocopy. So we should fix this!
Looking at the code, it seems to me that the copy should proceed even
if it encounters this error - file sizes are used only to print out
statistics, to a failure to compute file size is not fatal. Are you
seeing the copy terminate when you get this error?
I also have a half-baked theory that version 1.0 does not properly
wait for the snapshot to complete before starting the backup, which
might result in this the problem. Is it feasible for you to try the
unstable version? The one that's on GitHub isn't the very latest code,
but I could probably find some time today to push a build of the very
latest stuff if you'd like to give it a try.
Graeme Perrow
the heck it is. I've searched all over sourceforge.net/projects/
hobocopy and I cannot find anything that I can download. The "files"
section says "This project has no files". I don't want the code, I
want the executable itself. I must have downloaded the "non-stable"
version from here a while ago, but right now I can't even find that.
Graeme
Craig Andera
> the heck it is. I've searched all over sourceforge.net/projects/
> hobocopy and I cannot find anything that I can download. The "files"
> section says "This project has no files". I don't want the code, I
> want the executable itself. I must have downloaded the "non-stable"
> version from here a while ago, but right now I can't even find that.
Sorry about that: the new home of hobocopy is
http://github.com/candera/hobocopy. You can go straight to the
downloads page at https://github.com/candera/hobocopy/downloads. I did
make an attempt to point people to the new site from the SF one, but
apparently didn't do so good a job.
Graeme Perrow
that same error on a number of files but keeps going. After some
number of those errors (looks like about 12) it gives up and just
stops.
On Apr 29, 2:17 pm, Craig Andera <cand...@wangdera.com> wrote:
> > I can give the unstable version a try, if you'll kindly tell me where
> > the heck it is. I've searched all over sourceforge.net/projects/
> > hobocopy and I cannot find anything that I can download. The "files"
> > section says "This project has no files". I don't want the code, I
> > want the executable itself. I must have downloaded the "non-stable"
> > version from here a while ago, but right now I can't even find that.
>
> downloads page athttps://github.com/candera/hobocopy/downloads. I did
Craig Andera
> that same error on a number of files but keeps going. After some
> number of those errors (looks like about 12) it gives up and just
> stops.
Can you add the /verbosity=4 switch and see if the additional info
tells us anything about where it's stopping?
Also, do you have TrueCrypt installed?
Graeme Perrow
information on the failure. It looks like some of the files it's
copying from \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy12\... are
being copied successfully.
I'm not using TrueCrypt or any form of encryption.
Craig Andera
> information on the failure. It looks like some of the files it's
> copying from \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy12\... are
> being copied successfully.
OK. Can you share the last few lines of the output at verbosity 4?
> I'm not using TrueCrypt or any form of encryption.
Sometimes even having it installed can cause issues with the Volume
Shadow Service. I'd also be interested to see the output of running
"vssadmin list providers", "vssadmin list shadows", "vssadmin list
shadowstorage", "vssadmin list volumes" and "vssadmin list writers".
Graeme Perrow
Skipping file yz85hgzm.default\News\news\news.grc.com.msf because it
doesn't meet filter criteria.
Copied directory yz85hgzm.default\News\news
Skipping file yz85hgzm.default\News\news.msf because it doesn't meet
filter criteria.
Skipping file yz85hgzm.default\News\news.rc because it doesn't meet
filter criteria.
Copied directory yz85hgzm.default\News
Copied file \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy12\Documents
\yz85hgzm.default\panacea.dat
Unable to calculate size of file. Size calculations may be incorrect.
Message was: Unable to open file \\?\GLOBALROOT\Device
\HarddiskVolumeShadowCopy12\Documents and Settings\gperrow\Application
Data\Thunderbird\Profiles\yz85hgzm.default\panacea.dat to retrieve
being used by another process.
(Error number 32).
Data\Thunderbird\Profiles\yz85hgzm.default\parent.lock exists. Error
process.
(Error number 32).
[C:\]vssadmin list providers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line
tool
(C) Copyright 2001-2005 Microsoft Corp.
Provider name: 'Microsoft Software Shadow Copy provider 1.0'
Provider type: System
Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
Version: 1.0.0.7
[C:\]vssadmin list shadows
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line
tool
(C) Copyright 2001-2005 Microsoft Corp.
No items found that satisfy the query.
[C:\]vssadmin list shadowstorage
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line
tool
(C) Copyright 2001-2005 Microsoft Corp.
No items found that satisfy the query.
[C:\]vssadmin list volumes
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line
tool
(C) Copyright 2001-2005 Microsoft Corp.
Volume path: C:\
Volume name: \\?\Volume{e6c8b174-6c3d-11e0-933d-806e6f6e6963}\
Volume path: D:\
Volume name: \\?\Volume{e6c8b175-6c3d-11e0-933d-806e6f6e6963}\
Volume path: E:\
Volume name: \\?\Volume{e6c8b176-6c3d-11e0-933d-806e6f6e6963}\
[C:\]vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line
tool
(C) Copyright 2001-2005 Microsoft Corp.
Writer name: 'Task Scheduler Writer'
Writer Id: {d61d61c8-d73a-4eee-8cdd-f6f9786b7124}
Writer Instance Id: {1bddd48e-5052-49db-9b07-b96f96727e6b}
State: [1] Stable
Last error: No error
Writer name: 'VSS Metadata Store Writer'
Writer Id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}
Writer Instance Id: {088e7a7d-09a8-4cc6-a609-ad90e75ddc93}
State: [1] Stable
Last error: No error
Writer name: 'Performance Counters Writer'
Writer Id: {0bada1de-01a9-4625-8278-69e735f39dd2}
Writer Instance Id: {f0086dda-9efc-47c5-8eb6-a944c3d09381}
State: [1] Stable
Last error: No error
Writer name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
Writer Instance Id: {2850eaae-45f7-4e9f-8365-02618a57239f}
State: [5] Waiting for completion
Last error: No error
Writer name: 'ASR Writer'
Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
Writer Instance Id: {3f044e8b-7f5f-47e4-bed8-d9f187cf35dd}
State: [1] Stable
Last error: No error
Writer name: 'COM+ REGDB Writer'
Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
Writer Instance Id: {d0af5063-35ff-44eb-b039-c07dd93a03db}
State: [1] Stable
Last error: No error
Writer name: 'Shadow Copy Optimization Writer'
Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
Writer Instance Id: {13770b51-8f15-49a1-b772-816dcbf11943}
State: [1] Stable
Last error: No error
Writer name: 'Registry Writer'
Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
Writer Instance Id: {c53d533d-f966-4050-9d6a-294cd974e894}
State: [1] Stable
Last error: No error
Writer name: 'MSSearch Service Writer'
Writer Id: {cd3f2362-8bef-46c7-9181-d62844cdc0b2}
Writer Instance Id: {8a85cdcd-dbda-4773-8f8f-e6705272767c}
State: [5] Waiting for completion
Last error: No error
Writer name: 'BITS Writer'
Writer Id: {4969d978-be47-48b0-b100-f328f07ac1e0}
Writer Instance Id: {3e26d57e-ecf7-4ac1-80b5-7df9fd9ee17f}
State: [1] Stable
Last error: No error
Writer name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
Writer Instance Id: {309ca2a6-64db-46f8-8eac-4f1a412034d1}
State: [5] Waiting for completion
Last error: No error
Craig Andera
Hmm. No real help there.
>
> Here's the other output you requested:
> [C:\]vssadmin list writers
> vssadmin 1.1 - Volume Shadow Copy Service administrative command-line
> tool
> (C) Copyright 2001-2005 Microsoft Corp.
So, these bits here look interesting:
> Writer name: 'System Writer'
> Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
> Writer Instance Id: {2850eaae-45f7-4e9f-8365-02618a57239f}
> State: [5] Waiting for completion
> Last error: No error
>
> Writer name: 'MSSearch Service Writer'
> Writer Id: {cd3f2362-8bef-46c7-9181-d62844cdc0b2}
> Writer Instance Id: {8a85cdcd-dbda-4773-8f8f-e6705272767c}
> State: [5] Waiting for completion
> Last error: No error
>
> Writer name: 'WMI Writer'
> Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
> Writer Instance Id: {309ca2a6-64db-46f8-8eac-4f1a412034d1}
> State: [5] Waiting for completion
> Last error: No error
The "waiting for completion" part is what catches my eye. Did you run
'vssadmin list writers' while hobocopy was running?
I'll admit I'm a bit puzzled by this one: VSS is pretty complicated
and I still don't understand it that well. But I'm happy to keep
working on it until we find you a solution; just bear with me.
One thing I'd like to know is what process has that file open. It's a
shadow copy, so nothing should. You can download the handle utility
from http://technet.microsoft.com/en-us/sysinternals/bb896655, then
fire up hobocopy and run something like this:
handle Thunderbird
That should give us a list of processes that are accessing files with
Thunderbird in their names. We're particularly interested in anything
that isn't hobocopy.exe.
If that doesn't turn up any smoking gun, I have a few other questions
and things to try:
* Does this only happen with incremental backups, or do you see it
with full backups, too?
* Do you have antivirus software installed?
* If so, does this still happen if you disable it?
* If you reboot, does the problem go away?
* If you run chkdsk /r c: and reboot, does the problem go away?
Graeme Perrow
> The "waiting for completion" part is what catches my eye. Did you run
> 'vssadmin list writers' while hobocopy was running?
calls.
> That should give us a list of processes that are accessing files with
> Thunderbird in their names. We're particularly interested in anything
> that isn't hobocopy.exe.
it's just Thunderbird.
> * Does this only happen with incremental backups, or do you see it
> with full backups, too?
> * Do you have antivirus software installed?
company IT policies. I can uninstall it, which I plan to do on Monday,
so if you think that's part of the problem, I can wait until then and
try again.
> * If you reboot, does the problem go away?
> * If you run chkdsk /r c: and reboot, does the problem go away?
Craig Andera
>> 'vssadmin list writers' while hobocopy was running?
>
> Nope, I waited until hobocopy was done before trying the vssadmin
> calls.
Huh. That's a little weird - with hobocopy not running, I don't see
why those writers would be doing anything. Do you know if there is any
other backup software running on the machine? Something else your
company mandates, perhaps?
>> That should give us a list of processes that are accessing files with
>> Thunderbird in their names. We're particularly interested in anything
>> that isn't hobocopy.exe.
>
> Just thunderbird.exe and hobocopy.exe. If hobocopy isn't running, then
> it's just Thunderbird.
And does Thunderbird have files open with paths that look like
\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy4\Documents and
Settings\gperrow\ApplicationData\Thunderbird\Profiles\yz85hgzm.default\abook.mab?
Or does Thunderbird have them open on C:\ ? I would assume the latter.
If it's the former, then that's rather puzzling - only hobocopy should
have files in the shadow copy open.
>> * Does this only happen with incremental backups, or do you see it
>> with full backups, too?
>
> Only incremental backups.
Hmm. I've had a sneaking suspicion for some time that I'm not handling
incremental backups correctly. I find the documentation around the API
almost totally impenetrable, though.
I've gone ahead and built two new versions of hobocopy. You can find them here:
http://dl.dropbox.com/u/451150/hobocopy-master.zip
http://dl.dropbox.com/u/451150/hobocopy-29.zip
If you don't mind, I'd love to have you try each of them. The master
one is the latest code, including all the bugfixes I've made to date.
So it's newer than the unstable one on github. The -29 one is one I
just created that changes the code to always use the VSS_BT_FULL flag,
even for incremental backups. You'll still get an incremental backup,
since I filter by file modification times, but VSS will act like it's
a full backup.
>> * Do you have antivirus software installed?
>
> Yes, McAfee VirusScan Enterprise, which I cannot disable due to stupid
> company IT policies. I can uninstall it, which I plan to do on Monday,
> so if you think that's part of the problem, I can wait until then and
> try again.
I always suspect antivirus when there are weird file problems. But I
don't have any specific reason to think that's it, no.
>> * If you reboot, does the problem go away?
>> * If you run chkdsk /r c: and reboot, does the problem go away?
>
> No and no. Actually, I just ran chkdsk /f C:, so I'll try /r.
/f versus /r shouldn't make any difference as far as I know. I'm
grasping at straws a bit, and was wondering whether it might not be a
disk corruption issue. The fact that it only fails during incremental
copies suggests to me that this is a hobocopy bug, though. We'll know
more if you get a chance to try those two versions I just posted.
Thanks for your patience!
Graeme Perrow
On Apr 30, 8:44 am, Craig Andera <cand...@wangdera.com> wrote:
> Huh. That's a little weird - with hobocopy not running, I don't see
> why those writers would be doing anything. Do you know if there is any
> other backup software running on the machine? Something else your
> company mandates, perhaps?
> And does Thunderbird have files open with paths that look like
>
> \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy4\Documents and
> Settings\gperrow\ApplicationData\Thunderbird\Profiles\yz85hgzm.default\aboo k.mab?
>
> Or does Thunderbird have them open on C:\ ? I would assume the latter.
> If it's the former, then that's rather puzzling - only hobocopy should
> have files in the shadow copy open.
> If you don't mind, I'd love to have you try each of them. The master
> one is the latest code, including all the bugfixes I've made to date.
> So it's newer than the unstable one on github. The -29 one is one I
> just created that changes the code to always use the VSS_BT_FULL flag,
> even for incremental backups. You'll still get an incremental backup,
> since I filter by file modification times, but VSS will act like it's
> a full backup.
If I get bored this afternoon or tomorrow, I may try downloading the
hobocopy code and see if I can build and debug it.
Craig Andera
OK. Long shot anyway.
> Nope, Thunderbird only has C:\ files.
OK. Would have been puzzling if it did.
> Tried 'em both, same result.
Wow - that's *really* confusing. Other than selecting which files to
copy, there shouldn't be any difference between full and incremental
in the 29 branch. But I wrote this code almost five years ago, and
have only recently started actively maintaining it, so it's entirely
possible I'm missing something.
> If I get bored this afternoon or tomorrow, I may try downloading the
> hobocopy code and see if I can build and debug it.
That would be really helpful, so I hope you get bored. :)
It should be as easy as cloning the repo and then running the
build.cmd from within a VS2010 command prompt. Then just add a /debug
switch to hobocopy and it will trigger a breakpoint at startup, which
should let you attach a VS2010 instance. The only caveat is that
build.cmd creates the release builds, so you might want to use a
VS2010 build instead.
The change I made to use VSS_BT_FULL is on the
29-always-use-full-backup branch if you want to check that out.
Let me know what you find: I'm not giving up on this one yet.
Craig Andera
Hopefully they'll forgive me.:)
Here's [1] the updated version. I pushed the code on the
29-always-use-full-backup branch, too. Again, I really appreciate you
taking the time to give this a try.
[1] http://dl.dropbox.com/u/451150/hobocopy-29-20110503-01.zip
On Tue, May 3, 2011 at 10:15 PM, Craig Andera <can...@wangdera.com> wrote:
>>> Right, that makes sense. Nice call. I can make that change.
>>
>> Does that mean I get a cut of all the hobocopy profits? :-)
>
> Haha. You can have 50% of the *revenue*, even. :)
>
>>> Huh. Again, very strange: the only process that should have a handle
>>> open is hobocopy. But AFAIK, processes don't block themselves. I'll
>>> take a look at that and see if I can figure anything out. Maybe we can
>>> add some sort of retry functionality.
>>
>> All I can think of is that Windows 7 does things slightly differently
>> than Vista, but I assume you've done testing on Windows 7 already.
>>
>> Let me know if I can help further.
>
> From your description (since I can't repro), it looks like the failure
> comes during a call to CreateFile. At a guess, *something* is grabbing
> that file for a brief time. Not sure what, though: it makes no sense
> that a piece of software would consistently lock a file in a
> newly-created shadow copy.
>
> I've got a few minutes to hack tonight. I'll try throwing together
> something on a branch that retries the CreateFile five times on some
> interval. If you can give it a try when I'm done, I'd appreciate it.
>
> Thanks again for your assistance in tracking this down. I'm not sure
> whether anyone else knows this, but hobocopy has been downloaded over
> 100,000 times over the last few years. And those are just the ones I
> know about - it's hosted on some websites that have higher Google
> juice than the project home. So your assistance has the potential to
> help a lot of people!
>
Craig Andera
last week, but I found some time today to work on hobocopy, so I
thought I'd see if there was any way I could help out.
Graeme Perrow
possibly next week.
Craig Andera
It's been a few months now, but as near as I can recall we never
resolved this issue.
I'm guessing that you're on a 64-bit machine that's not running
Windows XP: only one of those builds should actually work for you at
all. Have you tried the version that is at
http://dl.dropbox.com/u/451150/hobocopy-29-20110503-01.zip?
On Sat, Nov 12, 2011 at 4:49 AM, ahmetgns <ahme...@hotmail.com> wrote:
> I have the same problem with original poster of this thread with
> Windows 7 64-bit Home Premium version. I have tried three versions and
> for
> Release XP version > Hobocopy always stops running
> Release W2K3 , x86 > "There was a COM Failure" error appears.
> Release W2K3, x64 > "(Error number 32)." which makes me find this
> thread.
>
> If you have a new version I can give it a try also :) Thanks for the
> program.
>
> On 13 Mayıs, 18:25, Craig Andera <cand...@wangdera.com> wrote:
>> Have you had any time to follow up on this? It's been pretty busy the
>> last week, but I found some time today to work on hobocopy, so I
>> thought I'd see if there was any way I could help out.
>>
>>
>>
>> On Tue, May 3, 2011 at 10:32 PM, Craig Andera <cand...@wangdera.com> wrote:
>> > Whoops, realized I forgot to cc the group on the last few messages.
>> > Hopefully they'll forgive me.:)
>>
>> > Here's [1] the updated version. I pushed the code on the
>> > 29-always-use-full-backup branch, too. Again, I really appreciate you
>> > taking the time to give this a try.
>>
>> > [1]http://dl.dropbox.com/u/451150/hobocopy-29-20110503-01.zip
>>
ahmetgns
builds in a post of yours on 4th May in this thread as a reply to
Graeme Perrow and I have tried each of them. I have written what
errors I encountered for each of them in my previous post (my previous
post is only visible in your reply's quotation). I am using Windows 7
64-bit. Windows 7 is different. When I search my harddisk for the
youtube video file just after watching it (flaXXXX.tmp) the search
brings many results like
"c:\Documents and Settings\user_name\AppData\Local\Microsoft\Windows
\Temporary Internet Files\Low\"
"c:\Users\user_name\Local Settings\Application Data\Microsoft\Windows
\Temporary Internet Files\Low\"
...and many more just for the same file.
I suspect this problem causes this error (Error number 32) and I don't
know which path is the original path for the file and others are just
the mirror paths. I will be glad if I can help you to solve this
problem.
On 17 Kasım, 16:00, Craig Andera <cand...@wangdera.com> wrote:
> Sorry for the delay in responding. I was travelling and am only now catching up.
>
> It's been a few months now, but as near as I can recall we never
> resolved this issue.
>
> I'm guessing that you're on a 64-bit machine that's not running
> Windows XP: only one of those builds should actually work for you at
> all. Have you tried the version that is athttp://dl.dropbox.com/u/451150/hobocopy-29-20110503-01.zip?
ahmetgns
If there is a file with a name FILE.JPG, for instance, under
C:\Users\user\AppData\Local\Microsoft\Windows\Temporary Internet Files
\Low
then you can also find this file under
"C:\Users\user\AppData\Local\Application Data\Microsoft\Windows
\Temporary Internet Files\Low" or
"C:\Users\user\AppData\Local\Application Data\Application Data
\Microsoft\Windows\Temporary Internet Files\Low" or
"C:\Documents and Settings\user\AppData\Local\Microsoft\Windows
\Temporary Internet Files\Low" or
"C:\Documents and Settings\user\Local Settings\Microsoft\Windows
\Temporary Internet Files\Low" etc.
The real path for the file is, as I could determine" is the first one.
The others are shortcuts to those paths (and normally you cannot
access the contents of them by clicking on those shortcuts due to NTFS
permissions which Windows 7 sets the "List folder/Run file" permission
to "Not Allow" for Everyone by default) therefore HoboCopy cannot copy
content from those paths and gives the error number 32. See the
original poster's first post of this topic and notice that Graeme
Perrow wants to copy something in his personal account folder by
typing the path as "\Documents and Settings\" which is actually not a
path but just a shortcut for his "\Users\" folder.
For short, paths including "Documents and Settings", "Local Settings",
"Application Data" or similar are shortcuts to Windows 7's real paths
like "Users", "Appdata" etc. and HoboCopy can copy content only by
typing these real paths, if I could determine it correct.
Craig Andera
When (if - ugh) I get back to hobocopy work, I'll have to verify this
and see if there's anything I can do about somehow handling this.
sunth...@gmail.com
i Recently started using hobocopy and all required details where put in and it worked beautifully for the 1st week and stopped working. I mean nothing has changed? please can you help me.
Craig Andera
with /verbosity=4? Are you running as administrator?
> You received this message because you are subscribed to the Google Groups
> "hobocopy" group.
> To unsubscribe from this group, send email to
> hobocopy+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>