UTF-8 nfD seems to cause problems

[Daniel Jozsef]

I imported a zpool that I've created using Linux, and later used in ZEVO. Under ZEVO, I have recursively changed all file names to be normal form D, as I couldn't set normalization on an existing file system.

When I imported it in openzfsonosx 1.2.7, first I noticed that I cannot access some German files. When running recursive commands, I'd get messages like "cannot access prüfung.doc: no such file or directory". This is mighty weird, so I went and tried to find out what's happened. The files in question all had certain accented characters, and were acting like zombies. Since zfs says it's fully healty, I had a hunch, and tried to create an artificial zombie...

I succeeded. See below.

Daniels-MacBook-Pro:teszt coralio$ sudo touch aüőúóí.txt

Password:
Daniels-MacBook-Pro:teszt coralio$ ls -l
ls: aüőúóí.txt: No such file or directory
total 9
dr-xr-xr-x+ 2 coralio staff 5 May 18 18:06 a
dr-xr-xr-x+ 2 coralio staff 2 May 18 18:06 b
dr-xr-xr-x+ 2 coralio staff 2 May 18 18:06 c
Daniels-MacBook-Pro:teszt coralio$

See? Apparently it gets created, but due to the special characters, it's now inaccessible!

[Bjoern Kahl]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hi Daniel,

Am 19.05.14 00:40, schrieb Daniel Jozsef:

unfortunately, this is a known issue. I was investigating it a couple
of weeks ago, but got distracted. I still have it on my ToDo list and
will try to find some time next week to propose a solution.

Unfortunately, the whole normalization handling is a mess, and since
it apparently is irrelevant on the other platforms (Solaris, Linux,
FreeBSD), it has not got much care in the past.


Best regards

Björn

- --
| Bjoern Kahl +++ Siegburg +++ Germany |
| "googlelogin@-my-domain-" +++ www.bjoern-kahl.de |
| Languages: German, English, Ancient Latin (a bit :-)) |
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQCVAgUBU3ucCFsDv2ib9OLFAQLwlwP/f1BnoE/NZSLWcTGOzF9Xn9syV3HvMP1C
coFwqoe8/Z7NEVC4HyIRdgw15lUQ6ztcv19HFCJ9oFjT6h7ybaXk9YW/FstBvwho
61xN6bBBFUZ5Wpoas6q80VkbmOlmKMpm3ZwmGxRTJ0WyWP8jEWc+hNxUtcRRIzNF
o0WiwDHD/z0=
=VgJm
-----END PGP SIGNATURE-----

[Jorgen Lundman]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I get slightly different results:

# touch aüőúóí.txt

# ls -l aüőúóí.txt
- -rw-r--r--+ 1 root wheel 0 May 21 09:05 aüőúóí.txt

# stat aüőúóí.txt
771751940 103009 -rw-r--r-- 1 root wheel 0 0 "May 21 09:05:18 2014" "May
21 09:05:18 2014" "May 21 09:05:18 2014" "May 21 09:05:18 2014" 131072 1
0 aüőúóí.txt


But my locale is set to:

LC_CTYPE=ja_JP.UTF-8

and my pool was created formD.

are we only speaking about the case where the pool isn't formD?

Jorgen Lundman | <lun...@lundman.net>
Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo | +81 (0)90-5578-8500 (cell)
Japan | +81 (0)3 -3375-1767 (home)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

Comment: Using GnuPG with SeaMonkey - http://www.enigmail.net/

iEYEARECAAYFAlN77W4ACgkQXZu5ATg1N8QylACcCtGJzlPWVQsDQZvIDWhJq9HO
tacAoONlJkmubpV4+MRfRKOUJVBSowzG
=x7X3
-----END PGP SIGNATURE-----

[Graham Perrin]

On Sunday, 18 May 2014 23:40:41 UTC+1, Daniel Jozsef wrote:

 

I imported a zpool that I've created using Linux, and later used in ZEVO. Under ZEVO, I have recursively changed all file names to be normal form D, as I couldn't set normalization on an existing file system. …

 For reference: 

<https://code.google.com/p/maczfs/issues/detail?id=53>

<http://zevo.getgreenbytes.com/forum/viewtopic.php?t=165>

At one time my understanding was that if the file system was not NFD from the outset, there was no useful way to deal with problem names in that system. 

Then there was more, but I can't find it – maybe one of the things that were discussed in IRC alone (neither in a forum nor in an issue tracker) – sorry. 

[Daniel Jozsef]

The file system has been created on Linux, with the filenames fixed by converting all file names to nfD. The file system itself is NOT set to any normalized form.

Is a fix in the works?

[Daniel Jozsef]

BTW the conversion was done under ZEVO, which handled nfC file names perfectly - though Finder choked on them, I could work with them fine from command line.

No matter what, there is absolutely zero excuse for reporting an existing file as missing. This should be a high priority bug.