Vim creates files named "4913", why?

[Arve Knudsen]

Hi

I have a very annoying problem with the x64 build of Vim 7.2 on
Windows 7, for some reason it creates files named "4913" during
editing. Is this a bug, or maybe triggered by a problem with my
system?

Thanks,
Arve

[James Vega]

As a quick Google would tell you, Vim creates this file in an attempt to
verify it can create a file in the directory in which you see the file
and set the uid/gid. It's just a temporary file used during the process
of creating a backup file.

--
James
GPG Key: 1024D/61326D40 2003-09-02 James Vega <jame...@debian.org>

[Arve Knudsen]

On 21 Okt, 13:58, James Vega <james...@jamessan.com> wrote:
> On Wed, Oct 21, 2009 at 02:33:46AM -0700, Arve Knudsen wrote:
> > I have a very annoying problem with the x64 build of Vim 7.2 on
> > Windows 7, for some reason it creates files named "4913" during
> > editing. Is this a bug, or maybe triggered by a problem with my
> > system?
>
> As a quick Google would tell you, Vim creates this file in an attempt to
> verify it can create a file in the directory in which you see the file
> and set the uid/gid.  It's just a temporary file used during the process
> of creating a backup file.

I did Google it, and found that it could be a problem on network-
filesystems? The problem is that it *isn't* temporary, it persists.

Arve

[James Vega]

Here's the code:

3486 for (i = 4913; ; i += 123)
3487 {
3488 sprintf((char *)gettail(IObuff), "%d", i);
3489 if (mch_lstat((char *)IObuff, &st) < 0)
3490 break;
3491 }
3492 fd = mch_open((char *)IObuff,
3493
O_CREAT|O_WRONLY|O_EXCL|O_NOFOLLOW, perm);
3494 if (fd < 0) /* can't write in directory */
3495 backup_copy = TRUE;
3496 else
3497 {
3498 # ifdef UNIX
3499 # ifdef HAVE_FCHOWN
3500 ignored = fchown(fd, st_old.st_uid, st_old.st_gid);
3501 # endif
3502 if (mch_stat((char *)IObuff, &st) < 0
3503 || st.st_uid != st_old.st_uid
3504 || st.st_gid != st_old.st_gid
3505 || (long)st.st_mode != perm)
3506 backup_copy = TRUE;
3507 # endif
3508 /* Close the file before removing it, on MS-Windows we
3509 * can't delete an open file. */
3510 close(fd);
3511 mch_remove(IObuff);
3512 }

If mch_open (line 3492) returns a file descriptor, then we get into
the following else which always calls mch_remove on the file (line
3511). So, Vim is removing the file but your filesystem isn't
properly handling that.

[Arve Knudsen]

On 21 Okt, 17:13, James Vega <james...@jamessan.com> wrote:

Thanks for the explanation. It's just plain old NTFS, so I guess
there's something wrong in my system.

Arve

[Craig Barkhouse]

Arve:
To confirm, are you seeing this on a local filesystem (you say "plain old NTFS"), or is there a remote filesystem involved? Do you only see this with the 64-bit build of Vim, or 32-bit as well? Can you provide the simplest possible .vimrc and steps to repro the problem? I'm interested in investigating this. We can take this to private e-mail if that would be better, to not flood the mailing list.

Craig

[Ben Fritz]

On Oct 21, 3:07 pm, Craig Barkhouse <crai...@microsoft.com> wrote:
>  We can take this to private e-mail if that would be better, to not flood the mailing list.
>

I think it would be better to leave it ON the list, so that future
googlers will find it AND the solution.

[Raúl Núñez de Arenas Coronado]

Saluton Ben :)

Ben Fritz <f...@gmail.com> skribis:

I think exactly the same... Googling for the problem will find this
thread, so it's better if the solution is in the thread, too.

--
Raúl "DervishD" Núñez de Arenas Coronado
Linux Registered User 88736 | http://www.dervishd.net
It's my PC and I'll cry if I want to... RAmen!

[Arve Knudsen]

Hi Craig

This is indeed a local filesystem (my C: drive). So far I've only seen
this with an x64 build, but I have installed this package before
without any such problems. As to reproducing it, I just edit a file
for a while, and then "4913" appears (after a minute perhaps). Funny
though, when I try to reproduce it right now, nothing happens. I was
able to make it happen earlier with a _vimrc only containing "set
nocompatible".

Arve

Arve

[Arve Knudsen]

I have been having some driver issues causing Windows to bluescreen
(at least Windows reports that a driver bugcheck failed), so it might
be due to a spurious filesystem issue that Vim is unable to delete the
"1394" file (which I'm guessing is what goes wrong).

Arve

[Arve Knudsen]

Sorry, by "1394" I mean "4913" of course (must've been thinking of
Firewire).

Arve

[Charles Campbell]

Please bottom post on this list. Quote a small (relevant) part of the
message you are replying to, and put your text underneath.

The guidelines for the list may be found at:
http://groups.google.com/group/vim_use/web/vim-information

I have a 64-bit Vista system; haven't noticed any 4913s hanging around it.

* what o/s? (windows, but is it WInXP, Vista, 7?)
* using vim or gvim?
* compiled for native windows, cygwin, or what?
* vim version
* what compiler/linker did you use? or did you get a pre-compiled one
from somewhere?

Answers to these Qs will, I'd imagine, help someone with a similar setup
to attempt to duplicate your problem.

Regards,
Chip Campbell

[Arve Knudsen]

On 22 Okt, 16:05, Charles Campbell <Charles.E.Campb...@nasa.gov>
wrote:

> Please bottom post on this list. Quote a small (relevant) part of the
> message you are replying to, and put your text underneath.
>
> The guidelines for the list may be found at:http://groups.google.com/group/vim_use/web/vim-information
>
> I have a 64-bit Vista system; haven't noticed any 4913s hanging around it.
>
> * what o/s?  (windows, but is it WInXP, Vista, 7?)

Windows 7 x64 (as previously stated)

> * using vim or gvim?

gvim

> * compiled for native windows, cygwin, or what?

Precompiled for Windows x64 from http://code.google.com/p/vim-win3264.
I grabbed the 7.2.000 version.

Arve

[Bram Moolenaar]

Arve Knudsen wrote:

Isn't it a caching problem? I can imagine that when you look at the
file system at the right moment the file is there. If this view isn't
updated for a little while you see the file, even though Vim has already
deleted it.

It's hard to imagine the file is properly deleted one moment and hangs
around another time. Either something essential changed or it's flaky.

--
"It's so simple to be wise. Just think of something stupid to say
and then don't say it." -- Sam Levenson

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ download, build and distribute -- http://www.A-A-P.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Arve Knudsen]

On Oct 26, 2:12 pm, Bram Moolenaar <B...@Moolenaar.net> wrote:

> Arve Knudsen wrote:
> > This is indeed a local filesystem (my C: drive). So far I've only seen
> > this with an x64 build, but I have installed this package before
> > without any such problems. As to reproducing it, I just edit a file
> > for a while, and then "4913" appears (after a minute perhaps). Funny
> > though, when I try to reproduce it right now, nothing happens. I was
> > able to make it happen earlier with a _vimrc only containing "set
> > nocompatible".
>
> Isn't it a caching problem?  I can imagine that when you look at the
> file system at the right moment the file is there.  If this view isn't
> updated for a little while you see the file, even though Vim has already
> deleted it.
>
> It's hard to imagine the file is properly deleted one moment and hangs
> around another time.  Either something essential changed or it's flaky.

It appears that these files aren't deleted as they should, since they
hang around after Vim has been closed. I have reinstalled Windows now,
I'll see if the problem has gone away (current theory is my filesystem
was flaky).

Arve

[Arve Knudsen]

On 26 Okt, 14:41, Arve Knudsen <arve.knud...@gmail.com> wrote:
>
> It appears that these files aren't deleted as they should, since they
> hang around after Vim has been closed. I have reinstalled Windows now,
> I'll see if the problem has gone away (current theory is my filesystem
> was flaky).

Update: Even after reformatting the partition and reinstalling
Windows, the problem remains, Vim keeps littering my filesystem with
these files :/ Could Vim detect and report the error when it is unable
to delete such files?

Arve

[Yakov Lerner]

Arve, I wonder if changing line 3494 'if( fd < 0 )' to 'if( fd == -1)'  fixes the problem ?

Can you make the change on line 3494, rebuild vim, and see if there is change ?

Yakov

PS

I learned once that on win32, you *must* write

if(sock == -1) and never if(sock < 0).

Win32 sockets can be negative.  Yes they really can be.  

msdn specs _open()  as returning  -1 on error, not "negative number". 

[Yakov Lerner]

I was wrong. _open can't return negative fd.

Maybe moving mch_remove() 

after brace 3512 fixed it ? 

Maybe win32 _open() has

some race where  it creates the file but then returns -1 ?

[Arve Knudsen]

Hi Yakov

On 28 Okt, 07:47, Yakov Lerner <iler...@gmail.com> wrote:

> On Wed, Oct 28, 2009 at 00:52, Yakov Lerner <iler...@gmail.com> wrote:
> > PS
> > I learned once that on win32, you *must* write
> > if(sock == -1) and never if(sock < 0).
> > Win32 sockets can be negative.  Yes they really can be.
> > msdn specs _open()  as returning  -1 on error, not "negative number".
>
> I was wrong. _open can't return negative fd.
> Maybe moving mch_remove()
> after brace 3512 fixed it ?
> Maybe win32 _open() has
> some race where  it creates the file but then returns -1 ?

I was tipped that my antivirus is a possible explanation for this
behaviour, so I am going to look into that first. If that doesn't work
I might try to debug Vim in Visual Studio, if this doesn't present too
much of a challenge, to see for myself what is going on.

Arve

[Craig Barkhouse]

> -----Original Message-----
> From: vim...@googlegroups.com [mailto:vim...@googlegroups.com] On
> Behalf Of Arve Knudsen
> Sent: Wednesday, October 28, 2009 1:25 AM
> To: vim_dev
> Subject: Re: Vim creates files named "4913", why?
>
>

Although I have not been able to repro this myself, I have a theory, which is supported empirically by Arve. Consider:

3508 /* Close the file before removing it, on MS-Windows we
3509 * can't delete an open file. */
3510 close(fd);
3511 mch_remove(IObuff);

There's a race condition between lines 3510 and 3511, wherein another process could open a handle to the file and thus prevent the delete from succeeding. One class of software that is known to do this and cause problems is antivirus programs, which want to scan things for viruses as soon as they are created. Another class of software that sometimes causes problems is local search programs, which want to index things as soon as they are created. Such software can do their tasks unobtrusively if they are written carefully and using the mechanisms provided by the operating system, namely oplocks. Unfortunately there are lots of bad virus scanners and indexers out there (lots of bad software in general!).

One solution which is very commonly used in Windows programs is to detect when a delete or rename operation fails (can be narrowed to specific error codes -- sharing violation and access denied), and retry a couple times with short delays in between. The idea is to give the interfering process some time to finish whatever work it was doing with the file. I've had to write such code before. I realize it is ugly, but it works well in practice. It works best with small files. How many people out there have tried to delete a large hierarchy of files using "rd /s /q", and had it inexplicably fail to delete a few files? Yet if you repeat the command it will generally succeed. Most likely a virus scanner or search indexer got in the way and is now out of the way.

Another solution, which is cleaner but can only be used in cases where you know you want to delete a file immediately after closing its handle, is to set the delete disposition on the file before closing the handle. There are a couple of ways using Windows APIs to do this. One way is to pass FILE_FLAG_DELETE_ON_CLOSE to CreateFile() when opening the handle originally. Another way is to send down a FileDispositionInfo using SetFileInformationByHandle() or equivalent. (Essentially the DeleteFile() API just does the following: open a handle, set delete disposition, and close the handle; if we break apart these steps, we gain finer control.)

I suggest a TODO item around more reliable delete/rename on Windows. I can work on it at some point.

Craig

[Arve Knudsen]

I can confirm that Vim creates these files when my antivirus's
realtime filesystem protection is enabled. I've opted to turn it off,
since it's such an annoyance :/ For the record, I use ESET NOD32.

Arve