Error copying UNICODE Cyrillic filenames

[radulo...@gmail.com]

Hi,

I have source with non-Latin, Cyrillic letters (FastCopy v.3.54):

O:\crtani\Бен и Холи\

and program can not copy, with this error:

     FindFirstFileEx(No mapping for the Unicode character exists in the target multi-byte code page.1113) : O:\crtani\Бен и Холи

What this means?


Thank you in advance!

[Dan Alexandru]

Here on Windows 10 it works fine, both on NTFS and FAT.

Maybe try from the command prompt:

chcp 65001

fastcopy.exe "o:\crtani\Бен и Холи\" /to="o:\backup"

 

[radulo...@gmail.com]

Hi Dan,

I have Windows 7 Pro.

This is very strange. It (your example) works from command prompt, but it not works from the GUI.
But if try from GUI when it is launched from command prompt (fastcopy.exe "o:\crtani\Бен и Холи\" /to="k:\crtani"), it works again!

I have found what is the problem: when I do drag & drop from Windows Explorer to source and/or destination, something is copied wrong and it reports an error!
When I type it letter by letter (or Copy/Paste), it looks the same but it works?!

Solution?

[Dan Alexandru]

So if you start the command prompt, run the `chcp 65001` command to change the code page, then run `fastcopy` to start the GUI, then drag and drop from Windows Explorer, copying works fine?

[radulo...@gmail.com]

No, code page does not affect the behavior.

It seems that D&D from Windows Explorer fills some additional characters which are unvisible, but they force FastCopy to make an error when tries to copy.
That was not a case with Latin character set,

[Hiroaki SHIROUZU]

It seems strange...

What kind of filesystem are source and destination?

FastCopy.log is written the exact filename string even if it contains illegal characters.

So you can confirm that it contains illegal character by binary editor or etc.

[radulo...@gmail.com]

Hi Hiroaki,

Yes, this is very strange. I reviewed log and there is nothing illegal. There are three empty folders in source dir.
Both systems are FAT32.
Here is a fresh log:



=================================================
FastCopy(ver3.54) start at 2018/10/11 10:54:22

<Source>  "O:\crtani\Бен и Холи\"
<DestDir> k:\crtani
<Command> Diff (Size/Date)
-------------------------------------------------

TotalRead  = 0.0 MiB
TotalWrite = 0.0 MiB
TotalFiles = 0 (3)
TotalTime  = 0.1 sec
TransRate  = 0.00 MiB/s
FileRate   = 0.00 files/s

Result : (ErrFiles : 0 / ErrDirs : 0)

=================================================
FastCopy(ver3.54) start at 2018/10/11 10:54:33

<Source>  "O:\crtani\Бен и Холи\"
<DestDir> K:\crtani\
<Command> Diff (Size/Date)
-------------------------------------------------

FindFirstFileEx(No mapping for the Unicode character exists in the target multi-byte code page.1113) : O:\crtani\Бен и Холи

TotalRead  = 0.0 MiB
TotalWrite = 0.0 MiB
TotalFiles = 0 (0)
TotalTime  = 0.0 sec
TransRate  = 0.00 MiB/s
FileRate   = 0.00 files/s

Result : (ErrFiles : 0 / ErrDirs : 1)



The only difference is that, when copied (D&D) from Explorer, there is a "\" at the end.
However, if I put a Latin source, it works normaly regardless of "\".
If it is a Cyrillic, there is a problem:



=================================================
FastCopy(ver3.54) start at 2018/10/11 11:08:14

<Source>  "O:\LG Smart TV\"
<DestDir> K:\crtani\
<Command> Diff (Size/Date)
-------------------------------------------------

TotalRead  = 0.0 MiB
TotalWrite = 0.0 MiB
TotalFiles = 0 (2)
TotalTime  = 0.1 sec
TransRate  = 0.00 MiB/s
FileRate   = 0.00 files/s

Result : (ErrFiles : 0 / ErrDirs : 0)

=================================================
FastCopy(ver3.54) start at 2018/10/11 11:08:48

<Source>  "O:\crtani\Бен и Холи\"
<DestDir> K:\crtani\
<Command> Diff (Size/Date)
-------------------------------------------------

FindFirstFileEx(No mapping for the Unicode character exists in the target multi-byte code page.1113) : O:\crtani\Бен и Холи

TotalRead  = 0.0 MiB
TotalWrite = 0.0 MiB
TotalFiles = 0 (0)
TotalTime  = 0.0 sec
TransRate  = 0.00 MiB/s
FileRate   = 0.00 files/s

Result : (ErrFiles : 0 / ErrDirs : 1)



Is there any way to help you? To zip this files for trial?

[Hiroaki SHIROUZU]

I think it requires to examine the log by binary editor.

Can you send me a fresh(not modified) log?

[Hiroaki SHIROUZU]

I received the log file (thank you), but The file names of the OK log and the NG log were exactly the same.

FAT dir-entry(short filename) only supports MBCS code, but VFAT dir-entry(long filename) supports UNCODE.

So I can't understand why it requires UNICODE mapping to MBCS.

[radulo...@gmail.com]

Hi,

would you like to send you this files in a zip archive, so you can try to reproduce this behavior?

[Hiroaki SHIROUZU]

Ok, please send.

But, in my environment, I think that there is a high possibility not to reproduce.

[radulo...@gmail.com]

Yes, I understand that it can be unreproducible.

However, the only difference is when dest pathname has Cyrillic letters and ends with '\' (backslash).
Then the error occurs.

[Hiroaki SHIROUZU]

I received your zip archive.

But, unfortunately, I couldn't reproduce the problem by this archive.

[radulo...@gmail.com]

I 'l try to make this problem easier to reproduce, on local HD.

[Vitalii Dovgan]

For me it looks like the problem can be reproduced when all the following conditions are met:
1. the source drive is a flash drive (USB) with FAT32 file system;
2. the destination drive is HDD or SSD with NTFS file system;
3. the file name to be copied contains Latin and non-Latin characters (with character codes in the range of 0x80...0xFF of your ANSI codepage that corresponds to your locale).
I have an impression that for some reason an ANSI version of FindFirstFileEx is invoked for some reason.

[Vitalii Dovgan]

Try to give the following name to a file on a flash drive (USB) with FAT32 file system:

Гіпертермії.pdf

( this file name is the following sequence of the wide characters: L"\x0413\x0456\x043F\x0435\x0440\x0442\x0435\x0440\x043C\x0456\x0457\x002E\x0070\x0064\x0066" )

Then try to copy this file from the flash drive to HDD or SSD with NTFS file system.

[SHIROUZU Hiroaki]

Thank you for your information.

I confirmed it.

It seems Windows call ANSI version if target is FAT/FAT32.

But FastCopy uses only FindFirstFileExW, so it is no way to solve it except MS will fix it.

https://github.com/shirouzu/FastCopy/blob/72655443c775af09a69cc2fd2aa6a602b4d987c9/src/fastcopyr.cpp#L311

[SHIROUZU Hiroaki]

I found the problem is only FindFirstFileExW, not FindFirstFileW.

I will add fallback to FindFirstFileW, if FindFirstFileExW fails as ERROR_NO_UNICODE_TRANSLATION error.

[SHIROUZU Hiroaki]

It was fixed in v3.86.