utf8/unicode/spaces patch

[Jorge-León]

Hello!

Finally i got around to set up a development environment for ninefs
(WinDDK).

Find a patch to ninefs.c at

http://www.magma.com.ni/sw/ninefs/utf8.diff

where path conversion between Windows and Plan9 is consistently
handled
by Windows' MultiByteToWideChar() and WideCharToMultiByte() functions.

To adapt different filename conventions, winpath() and p9path()
functions have
been introduced in ninefs.c. By now, the only transformations done
are
path separator (\ to /) and spaces in filenames (' ' to ?).

Some testing has been done: copying, creating and deleting files and
directories
with Windows Explorer.

Copying a big file tree with xcopy failed upon creating the first
level of
subdirectories (propably unrelated to the patch?).

The patches ninefs.c is here:

http://www.magma.com.ni/sw/ninefs/ninefs.c

and an executable at:

http://www.magma.com.ni/sw/ninefs/ninefs.exe

Regards,

Jorge-León

[Tim Newsham]

thank you.  I havent integrated it yet, but will be pulling in your ideas.

--
Tim Newsham | www.thenewsh.com/~newsham | thenewsh.blogspot.com

[Tim Newsham]

Attached is my patch based on your patch.  Can you review and let me know if this works

for you.  One difference is that I added a "-t" flag to specify no path translation and

defaulted to enabling translation.  If translation is allowed, spaces map to '?' going

to plan9 and unicode translation errors are silently ignored.  If no translation is allowed,

no space to ? mapping happens and translation errors cause the filename translation

to fail.  Any failed translations cause the filename to silently be discarded from the

directory listing unless the debug flag is on, in which case they are just reported.

My intuition is that unicode translations should never occur for a well behaved 9p server.

I experimented briefly with using unicode 0x2423 character instead of '?', which is what

Acme-SAC uses for windows paths with spaces in them.  It's an underscore-like character

which looks like it was designed to represent spaces. However, the font I use in

plan9 doesnt seem to support that character anyway, so there was no win, and the code

was more complex (having to make a copy of the widestring before operating on it and

then translating it). So I backed that out and stuck with the ? character.

If these look good to you, I'll commit and update the documentation.

On Mon, Mar 8, 2010 at 9:09 AM, Jorge-León <jorge-...@magma.com.ni> wrote:

[Tim Newsham]

My intuition is that unicode translations should never occur for a well behaved 9p server.

"unicode translation errors" should never occur.

[Jorge-León]

Hello!

Will try out your version of the patch as soon as possible.

In a first try i shared your intuition. After some crashes i better
added the second
GetLastError() check, at some moment at was hit.

I had also a reproducible npfs assertion at work when copying a large
directory tree
from WinXP to Plan9.

Regrettably i had no time to investigate further. The errormessages
are safely
stored on the server at work - so i cannot access them right know :-|

Regards,

Jorge-León

[Tim Newsham]

let me know what the assertions are and I can take a look at it.  Even better if you

know a way that I can reproduce them.

[Georg Lehner]

Tim Newsham wrote:
> let me know what the assertions are and I can take a look at it. Even
> better if you
> know a way that I can reproduce them.

- Creating a "new" textfile (for the first time?) generates the
following error:

C:\Programme\Dokan\DokanLibrary>
C:\Programme\Dokan\DokanLibrary>ninefs -p password -u leg 192.168.97.113 n
ws2utf8: invalid input character encountered or other conversion error
ws2utf8: invalid input character encountered or other conversion error
ws2utf8: invalid input character encountered or other conversion error

- Copy session of whole disk E: (8.5GB) via Windows Explorer was
interrupted after about 5GB with a popup saying something like:

"there are not enough system ressources to complete this operation".
after this, checking the properties of the copied folder to find out
the size,
the following error occured

C:\Programme\Dokan\DokanLibrary>ninefs -p password -u leg 192.168.97.113 n
Assertion failed: x == 0, file
d:\ninefs\npfs\npfs\trunk\libnpfs\winthread.c, line 103

This application has requested the Runtime to terminate it in an unusual
way.
Please contact the application's support team for more information.
- - -

Regards,

Jorge-Le�n

[Tim Newsham]

On Mon, Mar 15, 2010 at 12:20 PM, Georg Lehner <jorge-...@magma.com.ni> wrote:

Tim Newsham wrote:

let me know what the assertions are and I can take a look at it.  Even better if you
know a way that I can reproduce them.

- Creating a "new" textfile (for the first time?) generates the following error:

C:\Programme\Dokan\DokanLibrary>
C:\Programme\Dokan\DokanLibrary>ninefs -p password -u leg 192.168.97.113 n
ws2utf8: invalid input character encountered or other conversion error
ws2utf8: invalid input character encountered or other conversion error
ws2utf8: invalid input character encountered or other conversion error

How did you create the file? 

 

- Copy session of whole disk E: (8.5GB) via Windows Explorer was
 interrupted after about 5GB with a popup saying something like:

  "there are not enough system ressources to complete this operation".
 after this, checking the properties of the copied folder to find out the size,
 the following error occured

There's a memory leak fixed by my latest patches.  It could have caused

this.  At any rate, it does sound like a memory leak of some kind.  I should

probably run some tests using some sort of heap tool.  I'll add that to my todo.

While you do any testing, you should keep track of the VM size and let me

know if you see it growing wildly...

 

C:\Programme\Dokan\DokanLibrary>ninefs -p password -u leg 192.168.97.113 n
Assertion failed: x == 0, file d:\ninefs\npfs\npfs\trunk\libnpfs\winthread.c, line 103

 Hrmm.. interesting.  And scary.  This might be a pain to figure out...

 

Regards,
    Jorge-León

Did you get a chance to look over or try my patches?  I'm curious if the memory

leak fix has helped...

[Georg Lehner]

Tim Newsham wrote:
...

>
> - Creating a "new" textfile (for the first time?) generates the
> following error:
>
> C:\Programme\Dokan\DokanLibrary>
> C:\Programme\Dokan\DokanLibrary>ninefs -p password -u leg
> 192.168.97.113 n
> ws2utf8: invalid input character encountered or other conversion error
> ws2utf8: invalid input character encountered or other conversion error
> ws2utf8: invalid input character encountered or other conversion error
>
>
> How did you create the file?

Browsing with Windows Explorer to N:\usr\leg, then selecting "New
Textdocument" from the
context menu.

...

> Did you get a chance to look over or try my patches? I'm curious if
> the memory
> leak fix has helped...

...
Sorry, busy at work here.

Today I tried to open a Visio file (copied from elsewhere) from a ninefs
drive. Visio stalled
a while, then told me, that it could not open the file the way it wanted
to. I will post a debug
log and precise error message some other day.

Regards,

Jorge-Le�n

[Georg Lehner]

Hello!

Tim Newsham wrote:
> Attached is my patch based on your patch. Can you review and let me
> know if this works

> for you. [...]
Works just as expected. Copying a large tree still gives troubles.
Testing is currently delayed
by a bycicle accident (i'm still alive :) and a workshop at work.
- - -

OpenOffice hit the notyet() message of SetAllocationSize. After some
googleing around i
dare to state, that it is safe to return true (0) in all cases. This
function is (optionally) called
when creating a file, to find out if the filesystem has sufficient space
left to create the file.
Interestingly OpenOffice calls SetAllocationSize with a size of zero...
Following is your patch, with a verbosely commented re-implementation of
SetAllocationSize().

http://www.magma.com.ni/sw/ninefs/utf8.patch
- - -

Feature request: a path prefix commandline option.

Since remotely the root directory is mounted, but "my" directory is at
'/usr/jorge/windows_store', the available path depth is reduced by 25
characters
(in my example). The prefix option (let it be -f) would be used as in:

ninefs -p password -u jorge -f /usr/jorge/windows_store acp n

and would prefix any windows path with the prefix string before sending
it to
dokan (Plan9), and strip it from any path receipt by dokan.

Best Regards,

Jorge-Le�n

[Tim Newsham]

Tim Newsham wrote:

Attached is my patch based on your patch.  Can you review and let me know if this works

for you.  [...]

Works just as expected.

I committed my patch and the accompanying manual page change. I'll take a look

at the errors you uncovered shortly.