Tilde expansion on (Windows)files?
Gerhard Reithofer
working on a storage optimization system (millions of files) I realized,
that a tilde in a filename makes problems:
Tcl 8.5.4, Windows XP - test:
(gerhardr) 4 % set fl [glob y:/WinXP/*]
{y:/WinXP/~$obkonzept_tech.doc} y:/WinXP/E1007029.exe y:/WinXP/ricohmemory_xp
(gerhardr) 5 % foreach f $fl {puts [file type $f]}
could not read "y:/WinXP/~$obkonzept_tech.doc": no such file or directory
What does this mean, which filename is searched?
If it works on Windows (see man page), why doesn't it find the file?
Another problem is, that the file system is served by a unix server
(Samba or NetApp with Unix security) and the file name on which the
program should run, is:
"/net/pub/WinXP/~$obkonzept_tech.doc"
It doesn't work either.
How can this file be handled in Tcl?
Does anyone know a solution for this problem?
--
Gerhard Reithofer
Tech-EDV Support Forum - http://support.tech-edv.co.at
Mark Janssen
wrote:
See Tcl bug 2511011 [http://sourceforge.net/tracker/index.php?
func=detail&aid=2511011&group_id=10894&atid=110894]
Mark
Mark Janssen
BTW I cannot reproduce your problem with 8.5.7 might be worth trying
that.
glob c:/test/*
{c:/test/~$ola} c:/test/~ola
% foreach f [glob c:/test/*] {puts [file type $f]}
file
file
Mark
igor.g
> that a tilde in a filename makes problems
If I understand correctly, the problem is in '$' character, not in
'~'.
For example,
set fl [glob D:/test/*]
{D:/test/~$file.txt} {D:/test/~[file].txt}
foreach f $fl {
puts [file type $f]
}
could not read "D:/test/~$file.txt": no such file or directory
foreach f $fl {
puts [file type [subst -novariables -nocommands $f]]
}
file
file
Gerhard Reithofer
> On 2 aug, 16:18, Mark Janssen <mpc.jans...@gmail.com> wrote:
> > On 2 aug, 14:37, Gerhard Reithofer <gerhard.reitho...@tech-edv.co.at>
> > wrote:
> >
> >
> >
> > > Hi,
> > > working on a storage optimization system (millions of files) I realized,
> > > that a tilde in a filename makes problems:
> >
> > > Tcl 8.5.4, Windows XP - test:
> > > (gerhardr) 4 % set fl [glob y:/WinXP/*]
> > > {y:/WinXP/~$obkonzept_tech.doc} y:/WinXP/E1007029.exe y:/WinXP/ricohmemory_xp
> > > (gerhardr) 5 % foreach f $fl {puts [file type $f]}
> > > could not read "y:/WinXP/~$obkonzept_tech.doc": no such file or directory
...
> > See Tcl bug 2511011 [http://sourceforge.net/tracker/index.php?
> > func=detail&aid=2511011&group_id=10894&atid=110894]
> >
> > Mark
>
> BTW I cannot reproduce your problem with 8.5.7 might be worth trying
> that.
Thanks for that hint, I'll give it a try.
But the bug above hasn't the state closed?!?!
Mark Janssen
wrote:
It's actually a different issue related to leading ~ expansion, which
on closer inspection was not the problem you were having.
Gerhard Reithofer
> On Sun, 2 Aug 2009, Mark Janssen wrote:
>
> > On 2 aug, 16:18, Mark Janssen <mpc.jans...@gmail.com> wrote:
> > > On 2 aug, 14:37, Gerhard Reithofer <gerhard.reitho...@tech-edv.co.at>
> > > wrote:
...
> > BTW I cannot reproduce your problem with 8.5.7 might be worth trying
> > that.
>
> Thanks for that hint, I'll give it a try.
> But the bug above hasn't the state closed?!?!
Cuurently installed 8.5. I have tried:
teacup install --dry-run ActiveState::ActiveTcl 8.5.7
got many messages until
Resolving struct::list -is package ... [package struct::list 1.7 tcl
... Already installed in repository, probing dependencies]
Resolving Tcl 8.0 -is package ... [package Tcl 8.5.5 _ ... Installed
outside repository, probing dependencies]
Aborting installation, was not able to locate all dependencies.
What to do?
Gerhard Reithofer
I've tested your solution and found it working in my cases.
But I cannot confirm, that it is ONLY a problem of the "$", also the
"[]" and "~" make problems and must be handled correspondingly.
Unfortunately the subst-options MUST be used dependent on the
appearance of "$, ~ or []", (I hope) I've tried all possibilities. Maybe
other "special" characters require also "special treatment".
=== File: /net/pub/WinXP/~[file].txt ===
[file exist $f]-> 0
[file type $f]-> could not read "/net/pub/WinXP/~[file].txt": no such file or directory
[file type [subst -novariables $f]]-> wrong # args: should be "file option ?arg ...?"
[file type [subst -nocommands $f]]-> file
[file type [subst -novariables -nocommands $f]]-> file
=== File: /net/pub/WinXP/~file.txt ===
[file exist $f]-> 0
[file type $f]-> could not read "/net/pub/WinXP/~file.txt": no such file or directory
[file type [subst -novariables $f]]-> file
[file type [subst -nocommands $f]]-> file
[file type [subst -novariables -nocommands $f]]-> file
=== File: /net/pub/WinXP/$[file].txt ===
[file exist $f]-> 1
[file type $f]-> file
[file type [subst -novariables $f]]-> wrong # args: should be "file option ?arg ...?"
[file type [subst -nocommands $f]]-> file
[file type [subst -novariables -nocommands $f]]-> file
=== File: /net/pub/WinXP/~$file.txt ===
[file exist $f]-> 0
[file type $f]-> could not read "/net/pub/WinXP/~$file.txt": no such file or directory
[file type [subst -novariables $f]]-> file
[file type [subst -nocommands $f]]-> can't read "file": no such variable
[file type [subst -novariables -nocommands $f]]-> file
=== File: /net/pub/WinXP/$file.txt ===
[file exist $f]-> 1
[file type $f]-> file
[file type [subst -novariables $f]]-> file
[file type [subst -nocommands $f]]-> can't read "file": no such variable
[file type [subst -novariables -nocommands $f]]-> file
=== File: /net/pub/WinXP/[file].txt ===
[file exist $f]-> 1
[file type $f]-> file
[file type [subst -novariables $f]]-> wrong # args: should be "file option ?arg ...?"
[file type [subst -nocommands $f]]-> file
[file type [subst -novariables -nocommands $f]]-> file
But the subst command circumvents the problem in any way.
I hope the this "bug" AND the "circumvention" is consistent over the
different 8.5.x versions.
PS: clt, the best and fasted newsgroup - THX :-)
Mark Janssen
wrote:
I find the subst workaround very odd, because in the absence of \
escapes, subst -novariables -nocommands is a no-op.
Also I don't think you can upgrade the Tcl version by using teacup, I
would suggest install 8.5.7 in a different directory and trying with
that.
Mark
Gerald W. Lester
> Hi,
> working on a storage optimization system (millions of files) I realized,
> that a tilde in a filename makes problems:
>
> Tcl 8.5.4, Windows XP - test:
> (gerhardr) 4 % set fl [glob y:/WinXP/*]
> {y:/WinXP/~$obkonzept_tech.doc} y:/WinXP/E1007029.exe y:/WinXP/ricohmemory_xp
> (gerhardr) 5 % foreach f $fl {puts [file type $f]}
> could not read "y:/WinXP/~$obkonzept_tech.doc": no such file or directory
>
> What does this mean, which filename is searched?
> If it works on Windows (see man page), why doesn't it find the file?
On Windows a ~$*.doc file is a temporary file that MS Word opens. This file
contains information about which user has the file open, to prevent multiple
users from attempting to change a document at the same time. Hence, it may
have rather strange interactions with normal file system queries.
--
+------------------------------------------------------------------------+
| Gerald W. Lester |
|"The man who fights for his ideals is the man who is alive." - Cervantes|
+------------------------------------------------------------------------+
Gerhard Reithofer
> On 2 aug, 23:30, Gerhard Reithofer <gerhard.reitho...@tech-edv.co.at>
> wrote:
> > On Sun, 2 Aug 2009, igor.g wrote:
> > > > working on a storage optimization system (millions of files) I realized,
> > > > that a tilde in a filename makes problems
...
I have not tested all cases, another result/problem (I missed the
\-substitution):
=== File: /net/pub/WinXP/test/#\!home\!alina\!name# ===
[file exist $f]-> 1
[file type $f]-> file
[file type [subst -novariables $f]]-> could not read "/net/pub/WinXP/test/#!home!alina!name#": no such file or directory
[file type [subst -nocommands $f]]-> could not read "/net/pub/WinXP/test/#!home!alina!name#": no such file or directory
[file type [subst -novariables -nocommands $f]]-> could not read "/net/pub/WinXP/test/#!home!alina!name#": no such file or directory
[file type [subst -nobackslashes -novariables -nocommands $f]]-> file
...
> I find the subst workaround very odd, because in the absence of \
> escapes, subst -novariables -nocommands is a no-op.
Yes, but it is current the ONLY possibilty for me.
> Also I don't think you can upgrade the Tcl version by using teacup, I
> would suggest install 8.5.7 in a different directory and trying with
> that.
Installation is still open.
If I can't upgrade to higher subversion (8.5.x) it's one more argument
against teacup :-(
The main problem on all prerequisties is, that my customer has to do the
upgrade also, which ist often hardly or not possible.
We already had this dicussion here.
Gerhard Reithofer
> Gerhard Reithofer wrote:
> > Hi,
> > working on a storage optimization system (millions of files) I realized,
> > that a tilde in a filename makes problems:
> >
> > Tcl 8.5.4, Windows XP - test:
> > (gerhardr) 4 % set fl [glob y:/WinXP/*]
> > {y:/WinXP/~$obkonzept_tech.doc} y:/WinXP/E1007029.exe
> > y:/WinXP/ricohmemory_xp (gerhardr) 5 % foreach f $fl {puts [file type $f]}
> > could not read "y:/WinXP/~$obkonzept_tech.doc": no such file or directory
> >
> > What does this mean, which filename is searched?
> > If it works on Windows (see man page), why doesn't it find the file?
>
> On Windows a ~$*.doc file is a temporary file that MS Word opens. This file
> contains information about which user has the file open, to prevent multiple
> users from attempting to change a document at the same time. Hence, it may
> have rather strange interactions with normal file system queries.
My question was regarding the man page description
"This works on Unix, MacOS X and Windows (except very old releases)."
but I could not reproduce it on Windows.
All files in my examples are from server backups and are real world
examples for my intended project.
The current situation is, that there are many cases, where Tcl fails (in
sense of runtime errors) unless you catch all these situations.
The reason is an "interpretation" of special characters in filenames,
but in my project I have to handle the files "as they are" and therefore
Tcl may not the right tool if I don't find a reliable solution.
E.g. the tilde expansion only makes sense in a user specific environment
but not on a server where the files of thousands of users must be
handled independently.
In the meantime I've investigated the problem more deeply and igor.g
pointed me to the [subst] solution, that works in all my cases until
now.
The possibility to disable any "interpretation" by an option (e.g.
-nosubst) could solve this problem and would not break any
compatibility.
IMHO: In future versions (>=9) this feature should be removed or
disabled by default.
THX
Mark Janssen
My earlier suprise at the subst solution was caused by the fact that
subst -novariables -nocommands will lead to the exact same string. So
if this solves your issues with file type, it means that 'Everything
Is A String' is violated. glob or any of the file commands should not
do any substitution and even if they do, passing a subst-ed string
cannot supress this in any way. Very odd behaviour indeed.
Also examples like:
[file type [subst -novariables $f]]-> wrong # args: should be "file
option ?arg ...?"
Look very suspicious. Are you sure you are using the core file command
and not some wrapper to file which is doing strange stuff?
Regards,
Mark
Gerhard Reithofer
On Mon, 3 Aug 2009, Mark Janssen wrote:
> On Aug 3, 12:24 pm, Gerhard Reithofer <gerhard.reitho...@tech-
> edv.co.at> wrote:
> > On Sun, 2 Aug 2009, Gerald W. Lester wrote:
> > > Gerhard Reithofer wrote:
> > > > Hi,
> > > > working on a storage optimization system (millions of files) I realized,
> > > > that a tilde in a filename makes problems:
...
> > My question was regarding the man page description
> > "This works on Unix, MacOS X and Windows (except very old releases)."
> > but I could not reproduce it on Windows.
> >
> > All files in my examples are from server backups and are real world
> > examples for my intended project.
> >
> > The current situation is, that there are many cases, where Tcl fails (in
> > sense of runtime errors) unless you catch all these situations.
> > The reason is an "interpretation" of special characters in filenames,
> > but in my project I have to handle the files "as they are" and therefore
> > Tcl may not the right tool if I don't find a reliable solution.
> > E.g. the tilde expansion only makes sense in a user specific environment
> > but not on a server where the files of thousands of users must be
> > handled independently.
> >
> > In the meantime I've investigated the problem more deeply and igor.g
> > pointed me to the [subst] solution, that works in all my cases until
> > now.
> >
> > The possibility to disable any "interpretation" by an option (e.g.
> > -nosubst) could solve this problem and would not break any
> > compatibility.
> > IMHO: In future versions (>=9) this feature should be removed or
> > disabled by default.
...
> My earlier suprise at the subst solution was caused by the fact that
> subst -novariables -nocommands will lead to the exact same string. So
> if this solves your issues with file type, it means that 'Everything
> Is A String' is violated. glob or any of the file commands should not
> do any substitution and even if they do, passing a subst-ed string
> cannot supress this in any way. Very odd behaviour indeed.
I think all file commands are affected.
The file commands DO change the strings (at least the documented tilde
expansion). The "string" is reached to the file command, and it becomes
"preprocessed" inside of the file function.
> Also examples like:
>
> [file type [subst -novariables $f]]-> wrong # args: should be "file
> option ?arg ...?"
>
> Look very suspicious. Are you sure you are using the core file command
> and not some wrapper to file which is doing strange stuff?
The complete test script:
#!/bin/sh
#
# let's start the TCL-shell\
exec tclsh "$0" "{1+$@}"
if {$tcl_platform(platform) eq "windows"} {
set dir {y:\WinXP\test}
} else {
set dir "/net/pub/WinXP/test"
}
foreach f [lsort [glob [file join $dir * ]]] {
puts "=== File: $f ==="
puts -nonewline {[file exist $f]-> }
set res [catch {puts [file exist $f]} err]
if {$res} {puts $err}
puts -nonewline {[file type $f]-> }
set res [catch {puts [file type $f]} err]
if {$res} {puts $err}
puts -nonewline {[file type [subst -novariables $f]]-> }
set res [catch {puts [file type [subst -novariables $f]]} err]
if {$res} {puts $err}
puts -nonewline {[file type [subst -nocommands $f]]-> }
set res [catch {puts [file type [subst -nocommands $f]]} err]
if {$res} {puts $err}
puts -nonewline {[file type [subst -novariables -nocommands $f]]-> }
set res [catch {puts [file type [subst -novariables -nocommands $f]]} err]
if {$res} {puts $err}
puts -nonewline {[file type [subst -nobackslashes -novariables -nocommands $f]]-> }
set res [catch {puts [file type [subst -nobackslashes -novariables -nocommands $f]]} err]
if {$res} {puts $err}
}
#
# end script
#################
The directory contents:
ls -l /net/pub/WinXP/test
total 28
-rw-r--r-- 1 505 users 1536 Feb 21 2002 #\!home\!alina\!name#
-rw-r--r-- 1 gerhardr users 6 Aug 2 23:11 $[file].txt
-rw-r--r-- 1 gerhardr users 3 Aug 2 22:37 $file.txt
-rw-r--r-- 1 gerhardr users 7 Aug 2 23:11 [file].txt
-rw-r--r-- 1 root root 5 Aug 2 22:42 ~$file.txt
-rw-r--r-- 1 gerhardr users 18 Aug 2 23:11 ~[file].txt
-rw-r--r-- 1 root root 6 Aug 2 22:43 ~file.txt
% info patchlevel
8.5.4
Andreas Kupries
> On Sun, 2 Aug 2009, Gerhard Reithofer wrote:
>
>> On Sun, 2 Aug 2009, Mark Janssen wrote:
>>
>> > On 2 aug, 16:18, Mark Janssen <mpc.jans...@gmail.com> wrote:
>> > > On 2 aug, 14:37, Gerhard Reithofer <gerhard.reitho...@tech-edv.co.at>
>> > > wrote:
>
> ...
>
>> > BTW I cannot reproduce your problem with 8.5.7 might be worth trying
>> > that.
>>
>> Thanks for that hint, I'll give it a try.
>> But the bug above hasn't the state closed?!?!
>
> Cuurently installed 8.5. I have tried:
> teacup install --dry-run ActiveState::ActiveTcl 8.5.7
My understanding of 'Mark Janssen's '8.5.7 might be worth trying' is
that he asks you to get a 8.5.7 based tclsh.
This cannot be done through teacup/teapot. We (ActiveState) distribute
packages and basekits through this system, but not the regular
tclsh/wish. For that you have to go to
http://www.activestate.com/activetcl/
and download the distribution.
> got many messages until
> Resolving struct::list -is package ... [package struct::list 1.7 tcl
> ... Already installed in repository, probing dependencies]
> Resolving Tcl 8.0 -is package ... [package Tcl 8.5.5 _ ... Installed
> outside repository, probing dependencies]
>
> Aborting installation, was not able to locate all dependencies.
>
> What to do?
To investigate this it would be useful to capture the output of teacup
in a file which we can then look through to find the package which it
missed, and from where then who claimed it as dependency.
--
So long,
Andreas Kupries <akup...@shaw.ca>
<http://www.purl.org/NET/akupries/>
Developer @ <http://www.activestate.com/>
-------------------------------------------------------------------------------
Larry W. Virden
> > Cuurently installed 8.5. I have tried:
> > teacup install --dry-run ActiveState::ActiveTcl 8.5.7
>
So, here's a teacup question.
I have at least 3 vesions of ActiveTcl installed in my environment -
8.5.4 (production) 8.5.7 and 8.6.0 beta.
If I do a standard install, into different directories, do these
different installations share information between the install
directories?
What I was expecting was that if I set my directory so that the teacup
from the 8.6.0 install was first, then updates done by that teacup
only applied to the tclsh8.6 .
I don't know if that is the case though.
The reason I am saying that is that as far as I was aware, I was
working against the 8.5.7 and 8.6.0 version all summer. However, today
I tried a
teacup install --dry-run ActiveState::ActiveTcl 8.5.7
against my production copy - to see what all would be updated.
After the process finished, I ran
teacup list --only newer
and was told there were 0 entities - even though I know that it has
been quite some time since I updated that older version.
When I looked through the directory, I found many files with
modification dates over the past few months.
So I am confused.
Gerhard Reithofer
On Wed, 5 Aug 2009, Andreas Kupries wrote:
> Gerhard Reithofer <gerhard....@tech-edv.co.at> writes:
>
> > On Sun, 2 Aug 2009, Gerhard Reithofer wrote:
> >
> >> On Sun, 2 Aug 2009, Mark Janssen wrote:
> >>
> >> > On 2 aug, 16:18, Mark Janssen <mpc.jans...@gmail.com> wrote:
> >> > > On 2 aug, 14:37, Gerhard Reithofer <gerhard.reitho...@tech-edv.co.at>
> >> > > wrote:
> >
> > ...
> >
> >> > BTW I cannot reproduce your problem with 8.5.7 might be worth trying
> >> > that.
> >>
> >> Thanks for that hint, I'll give it a try.
> >> But the bug above hasn't the state closed?!?!
> >
> > Cuurently installed 8.5. I have tried:
> > teacup install --dry-run ActiveState::ActiveTcl 8.5.7
>
> My understanding of 'Mark Janssen's '8.5.7 might be worth trying' is
> that he asks you to get a 8.5.7 based tclsh.
>
> This cannot be done through teacup/teapot. We (ActiveState) distribute
> packages and basekits through this system, but not the regular
> tclsh/wish. For that you have to go to
>
> http://www.activestate.com/activetcl/
>
> and download the distribution.
OK, I will do that.
> > got many messages until
> > Resolving struct::list -is package ... [package struct::list 1.7 tcl
> > ... Already installed in repository, probing dependencies]
> > Resolving Tcl 8.0 -is package ... [package Tcl 8.5.5 _ ... Installed
> > outside repository, probing dependencies]
> >
> > Aborting installation, was not able to locate all dependencies.
> >
> > What to do?
>
> To investigate this it would be useful to capture the output of teacup
> in a file which we can then look through to find the package which it
> missed, and from where then who claimed it as dependency.
I found the following suspicous entries:
Resolving doctools::idx::export::plugin -is package ... Not found in the archives.
Resolving doctools::idx::import::plugin -is package ... Not found in the archives.
Resolving doctools::toc::export::plugin -is package ... Not found in the archives.
Resolving doctools::toc::import::plugin -is package ... Not found in the archives.
THX
Andreas Kupries
>> To investigate this it would be useful to capture the output of teacup
>> in a file which we can then look through to find the package which it
>> missed, and from where then who claimed it as dependency.
>
> I found the following suspicous entries:
> Resolving doctools::idx::export::plugin -is package ... Not found in the archives.
> Resolving doctools::idx::import::plugin -is package ... Not found in the archives.
> Resolving doctools::toc::export::plugin -is package ... Not found in the archives.
> Resolving doctools::toc::import::plugin -is package ... Not found in the archives.
Ok. That gives me something to look in the build process. Thanks.
Andreas Kupries
> On Aug 6, 12:08 am, Andreas Kupries <akupr...@shaw.ca> wrote:
>
>> > Cuurently installed 8.5. I have tried:
>> > teacup install --dry-run ActiveState::ActiveTcl 8.5.7
>>
>
> So, here's a teacup question.
>
> I have at least 3 vesions of ActiveTcl installed in my environment -
> 8.5.4 (production) 8.5.7 and 8.6.0 beta.
>
> If I do a standard install, into different directories, do these
> different installations share information between the install
> directories?
8.6 and 8.5.7 should not. Regarding 8.5.4 I am not sure if the teacup
in that distribution was modern enough to have full segregation
yet. Of course, it might have been update-self'd too, complicating the
picture.
One way to investigate is to run all the different teacup's for
teacup cache status
and teacup default
The output from these should tell us if any of them share either
repository or index cache information.
> What I was expecting was that if I set my directory so that the teacup
> from the 8.6.0 install was first, then updates done by that teacup
> only applied to the tclsh8.6 .
Updates run by the 8.6 teacup should apply only to the 8.6
installation, regardless of its position in the PATH.
> I don't know if that is the case though.
>
> The reason I am saying that is that as far as I was aware, I was
> working against the 8.5.7 and 8.6.0 version all summer. However, today
> I tried a
> teacup install --dry-run ActiveState::ActiveTcl 8.5.7
This is the AT 8.6 teacup ?
> against my production copy - to see what all would be updated.
>
> After the process finished, I ran
> teacup list --only newer
> and was told there were 0 entities - even though I know that it has
> been quite some time since I updated that older version.
> When I looked through the directory, I found many files with
> modification dates over the past few months.
>
> So I am confused.
I am as well.
JMN
Adding a 'file normalize' seems to fix it:
foreach f $fl {set f [file normalize $f];puts [file type $f]}
I've reported this previously as bug id: 2806250 - glob, 'file
exists', 'file normalize' interaction with tilde
This is probably more or less the same as bug 2511011 already
mentioned.
I also made some detailed comments about this whole situation in this
newsgroup recently:
http://groups.google.com/group/comp.lang.tcl/browse_thread/thread/54f4cd8c47eaa406/b3602d39c21d7d81?lnk=gst&q=fiasco#
I was hoping my suggestions in that thread may have got a bit more
response from others - in particular Tcl Core team members. Perhaps
my title of ''file join' behaviour and ye olde tilde fiasco' was too
inflammatory - but frankly.. I still think this area is a mess.
Julian
Gerhard Reithofer
> On Aug 2, 12:37 pm, Gerhard Reithofer <gerhard.reitho...@tech-
> edv.co.at> wrote:
> > Hi,
> > working on a storage optimization system (millions of files) I realized,
> > that a tilde in a filename makes problems:
...
> Adding a 'file normalize' seems to fix it:
> foreach f $fl {set f [file normalize $f];puts [file type $f]}
>
> I've reported this previously as bug id: 2806250 - glob, 'file
> exists', 'file normalize' interaction with tilde
>
> This is probably more or less the same as bug 2511011 already
> mentioned.
>
> I also made some detailed comments about this whole situation in this
> newsgroup recently:
> http://groups.google.com/group/comp.lang.tcl/browse_thread/thread/54f4cd8c47eaa406/b3602d39c21d7d81?lnk=gst&q=fiasco#
>
> I was hoping my suggestions in that thread may have got a bit more
> response from others - in particular Tcl Core team members. Perhaps
> my title of ''file join' behaviour and ye olde tilde fiasco' was too
> inflammatory - but frankly.. I still think this area is a mess.
New situation: tcl version=8.5.7
=== File: /net/pub/WinXP/test/#\!home\!alina\!name# ===
[file exist $f]-> 1
[file type $f]-> file
[file type [subst -novariables $f]]-> ERR
[file type [subst -nocommands $f]]-> ERR
[file type [subst -nobackslashes $f]]-> file
=== File: /net/pub/WinXP/test/$[file].txt ===
[file exist $f]-> 1
[file type $f]-> file
[file type [subst -novariables $f]]-> ERR
[file type [subst -nocommands $f]]-> file
[file type [subst -nobackslashes $f]]-> ERR
=== File: /net/pub/WinXP/test/$file.txt ===
[file exist $f]-> 1
[file type $f]-> file
[file type [subst -novariables $f]]-> file
[file type [subst -nocommands $f]]-> ERR
[file type [subst -nobackslashes $f]]-> ERR
=== File: /net/pub/WinXP/test/[file].txt ===
[file exist $f]-> 1
[file type $f]-> file
[file type [subst -novariables $f]]-> ERR
[file type [subst -nocommands $f]]-> file
[file type [subst -nobackslashes $f]]-> ERR
=== File: /net/pub/WinXP/test/~$file.txt ===
[file exist $f]-> 0
[file type $f]-> ERR
[file type [subst -novariables $f]]-> file
[file type [subst -nocommands $f]]-> ERR
[file type [subst -nobackslashes $f]]-> ERR
=== File: /net/pub/WinXP/test/~[file].txt ===
[file exist $f]-> 0
[file type $f]-> ERR
[file type [subst -novariables $f]]-> ERR
[file type [subst -nocommands $f]]-> file
[file type [subst -nobackslashes $f]]-> ERR
=== File: /net/pub/WinXP/test/~file.txt ===
[file exist $f]-> 0
[file type $f]-> ERR
[file type [subst -novariables $f]]-> file
[file type [subst -nocommands $f]]-> file
[file type [subst -nobackslashes $f]]-> file
Other situation but IMHO not consistent.
All leading-tilde files require "subst" but the "neccessary" options are
not clear to me.
[file normalize] did not change the situation, all my paths are
absolute.
JMN
edv.co.at> wrote:
> On Fri, 7 Aug 2009, JMN wrote:
> > On Aug 2, 12:37 pm, Gerhard Reithofer <gerhard.reitho...@tech-
> > edv.co.at> wrote:
> > > Hi,
> > > working on a storage optimization system (millions of files) I realized,
> > > that a tilde in a filename makes problems:
>
> ...
>
>
>
>
>
> > Adding a 'file normalize' seems to fix it:
> > foreach f $fl {set f [file normalize $f];puts [file type $f]}
>
> > I've reported this previously as bug id: 2806250 - glob, 'file
> > exists', 'file normalize' interaction with tilde
>
> > This is probably more or less the same as bug 2511011 already
> > mentioned.
>
> > I also made some detailed comments about this whole situation in this
> > newsgroup recently:
Sorry - but I need to ask this. Did you actually try it as shown in my
modification of your example?
Yes - the paths are absolute - but apparently 'file normalize' affects
the internal representation.
e.g
set list [glob c:/Users/Administrator/ttest/*]
c:/Users/Administrator/ttest/~test.doc
%set f [lindex $list 0]
c:/Users/Administrator/ttest/~test.doc
%file exists $f
0
%set f [file normalize $f]
c:/Users/Administrator/ttest/~test.doc
%file exists $f
1
as you can see - 'set f [file normalize $f]' appears to be setting f
to the same value as before - but it 'repairs' things somehow.
JN
Mark Janssen
edv.co.at> wrote:
> On Fri, 7 Aug 2009, JMN wrote:
> > On Aug 2, 12:37 pm, Gerhard Reithofer <gerhard.reitho...@tech-
> > edv.co.at> wrote:
> > > Hi,
> > > working on a storage optimization system (millions of files) I realized,
> > > that a tilde in a filename makes problems:
>
> ...
>
>
>
> > Adding a 'file normalize' seems to fix it:
> > foreach f $fl {set f [file normalize $f];puts [file type $f]}
>
> > I've reported this previously as bug id: 2806250 - glob, 'file
> > exists', 'file normalize' interaction with tilde
>
> > This is probably more or less the same as bug 2511011 already
> > mentioned.
>
> > I also made some detailed comments about this whole situation in this
> > newsgroup recently:
And because in Tcl everything is a string, any behaviour where the
internal representation of the string makes a difference is a bug. The
is true for a [subst] or [file normalize] which doesn't change the
string, but does prevent the error. Please log a bug on sourceforge,
because that will have a higher chance of the appropriate people
looking at it and probably fixing the issue.
As a sidenote, looking at the examples I think it will help to isolate
the ones that are giving an issue to reduce the clutter:
> === File: /net/pub/WinXP/test/#\!home\!alina\!name# ===
> [file exist $f]-> 1
> [file type $f]-> file
> [file type [subst -novariables $f]]-> ERR
> [file type [subst -nocommands $f]]-> ERR
> [file type [subst -nobackslashes $f]]-> file
Without knowing exactly what the errors are, this is correct
behaviour, becayse subst will substitute the backslashes if called
without -nobackslashes.
> === File: /net/pub/WinXP/test/$[file].txt ===
> [file exist $f]-> 1
> [file type $f]-> file
> [file type [subst -novariables $f]]-> ERR
> [file type [subst -nocommands $f]]-> file
> [file type [subst -nobackslashes $f]]-> ERR
Correct behaviour, because [file] requires additional arguments so the
[subst] fails.
> === File: /net/pub/WinXP/test/$file.txt ===
> [file exist $f]-> 1
> [file type $f]-> file
> [file type [subst -novariables $f]]-> file
> [file type [subst -nocommands $f]]-> ERR
> [file type [subst -nobackslashes $f]]-> ERR
Correct behaviour if $file is not set.
> === File: /net/pub/WinXP/test/[file].txt ===
> [file exist $f]-> 1
> [file type $f]-> file
> [file type [subst -novariables $f]]-> ERR
> [file type [subst -nocommands $f]]-> file
> [file type [subst -nobackslashes $f]]-> ERR
Correct behaviour, [file] requires additional arguments.
> === File: /net/pub/WinXP/test/~$file.txt ===
> [file exist $f]-> 0
> [file type $f]-> ERR
> [file type [subst -novariables $f]]-> file
> [file type [subst -nocommands $f]]-> ERR
> [file type [subst -nobackslashes $f]]-> ERR
This seems to be a case where file exists is definitely wrong, based
on the fact that the subst with -novariables does work. The fact that
the other forms of subst fail is expected.
> === File: /net/pub/WinXP/test/~[file].txt ===
> [file exist $f]-> 0
> [file type $f]-> ERR
> [file type [subst -novariables $f]]-> ERR
> [file type [subst -nocommands $f]]-> file
> [file type [subst -nobackslashes $f]]-> ERR
This seems to be a case where file exists is definitely wrong, based
on the fact that the subst with -nocommands does work. The fact that
the other forms of subst fail is expected.
> === File: /net/pub/WinXP/test/~file.txt ===
> [file exist $f]-> 0
> [file type $f]-> ERR
> [file type [subst -novariables $f]]-> file
> [file type [subst -nocommands $f]]-> file
> [file type [subst -nobackslashes $f]]-> file
Again demonstration of the fact that subst seems to change the
internal representation of $f in such a way that [file type] works. So
again a demonstration of the bug.
> === File: /net/pub/WinXP/test/~file.txt ===
> [file exist $f]-> 0
> [file type $f]-> ERR
> [file type [subst -novariables $f]]-> file
> [file type [subst -nocommands $f]]-> file
> [file type [subst -nobackslashes $f]]-> file
Again demonstration of the fact that subst seems to change the
internal representation of $f in such a way that [file type] works. So
again a demonstration of the bug.
Mark
Gerhard Reithofer
> On Aug 7, 12:49 pm, Gerhard Reithofer <gerhard.reitho...@tech-
> edv.co.at> wrote:
> > On Fri, 7 Aug 2009, JMN wrote:
> > > On Aug 2, 12:37 pm, Gerhard Reithofer <gerhard.reitho...@tech-
> > > edv.co.at> wrote:
> > > > Hi,
> > > > working on a storage optimization system (millions of files) I realized,
> > > > that a tilde in a filename makes problems:
...
> > [file normalize] did not change the situation, all my paths are
> > absolute.
>
> Sorry - but I need to ask this. Did you actually try it as shown in my
> modification of your example?
>
> Yes - the paths are absolute - but apparently 'file normalize' affects
> the internal representation.
>
> e.g
> set list [glob c:/Users/Administrator/ttest/*]
> c:/Users/Administrator/ttest/~test.doc
> %set f [lindex $list 0]
> c:/Users/Administrator/ttest/~test.doc
> %file exists $f
> 0
> %set f [file normalize $f]
> c:/Users/Administrator/ttest/~test.doc
> %file exists $f
> 1
>
> as you can see - 'set f [file normalize $f]' appears to be setting f
> to the same value as before - but it 'repairs' things somehow.
I've checked this once more and you are right!
[file normalize] leads to the situation that it handles all 7 cases
correct.
I've tested it only on Linux, because I don't have much time currently
and I must go to others computers for for Windows tests, transfer the
test programs ...
Gerhard Reithofer
> On Aug 7, 2:49 pm, Gerhard Reithofer <gerhard.reitho...@tech-
> edv.co.at> wrote:
> > On Fri, 7 Aug 2009, JMN wrote:
> > > On Aug 2, 12:37 pm, Gerhard Reithofer <gerhard.reitho...@tech-
> > > edv.co.at> wrote:
> > > > Hi,
> > > > working on a storage optimization system (millions of files) I realized,
> > > > that a tilde in a filename makes problems:
...
> And because in Tcl everything is a string, any behaviour where the
> internal representation of the string makes a difference is a bug. The
> is true for a [subst] or [file normalize] which doesn't change the
> string, but does prevent the error. Please log a bug on sourceforge,
> because that will have a higher chance of the appropriate people
> looking at it and probably fixing the issue.
>
> As a sidenote, looking at the examples I think it will help to isolate
> the ones that are giving an issue to reduce the clutter:
Reported as bug #2834453
http://sourceforge.net/tracker/?func=detail&aid=2834453&group_id=10894&atid=110894
THX
JMN
edv.co.at> wrote:
> On Sat, 8 Aug 2009, Mark Janssen wrote:
> > On Aug 7, 2:49 pm, Gerhard Reithofer <gerhard.reitho...@tech-
> > edv.co.at> wrote:
> > > On Fri, 7 Aug 2009, JMN wrote:
> > > > On Aug 2, 12:37 pm, Gerhard Reithofer <gerhard.reitho...@tech-
> > > > edv.co.at> wrote:
> > > > > Hi,
> > > > > working on a storage optimization system (millions of files) I realized,
> > > > > that a tilde in a filename makes problems:
>
> ...
>
> > And because in Tcl everything is a string, any behaviour where the
> > internal representation of the string makes a difference is a bug. The
> > is true for a [subst] or [file normalize] which doesn't change the
> > string, but does prevent the error. Please log a bug on sourceforge,
> > because that will have a higher chance of the appropriate people
> > looking at it and probably fixing the issue.
>
> > As a sidenote, looking at the examples I think it will help to isolate
> > the ones that are giving an issue to reduce the clutter:
>
> Reported as bug #2834453
>
Unfortunately I think things have gone a little off track here.
Good on you for making the effort to submit a report - but It seems to
me the whole 'subst' thing is a bit of a diversion - and you've
perhaps tried to cover too many points at once in the bug report.
Leaving aside the issue of the backslash for now (I'm not sure how to
even create a file named #\!home\!alina\!name# on a windows system)..
I see no substitution problem regarding the characters $ [ or ].
It works fine as follows - and I don't see any reason to use 'subst'
on the filenames.
%set files [glob c:/tmp/test/*]
{c:/tmp/test/$aaa.txt} {c:/tmp/test/[aaa].txt}
%foreach f $files {
puts "$f [file type $f]"
}
c:/tmp/test/$aaa.txt file
c:/tmp/test/[aaa].txt file
What exactly is the perceived issue with $,[,] ?
tilde is another issue altogether. As noted in my bug report 2806250 -
it is also dependent on the current working directory of your script.
JN
JMN
No - the '$' sign here causes no problems.
See my other example in this thread showing this.
The above error is due to the tilde.
It just happens that certain string operations such as subst & file
normalize can hack around the bug.
It seems that even the following works:
foreach f $fl {
set f [string range $f 0 end]
puts [file type $f]
}
J
Gerhard Reithofer
It is my basic problem, that I have to handle Unix AND Windows files on
a server - which serves to both worlds.
The file was created under Linux as temporary backup from an editor
AFAIR.
> I see no substitution problem regarding the characters $ [ or ].
> It works fine as follows - and I don't see any reason to use 'subst'
> on the filenames.
>
> %set files [glob c:/tmp/test/*]
> {c:/tmp/test/$aaa.txt} {c:/tmp/test/[aaa].txt}
> %foreach f $files {
> puts "$f [file type $f]"
> }
> c:/tmp/test/$aaa.txt file
> c:/tmp/test/[aaa].txt file
>
> What exactly is the perceived issue with $,[,] ?
In 8.5.7 $,[,] is interpreted correct as my examples show, in 8.4 also
file which start with a tilde are handled correct. But the interaction
between tilde and the special characters is not clear to me.
=== 1: File: /net/pub/WinXP/test/~[filename].txt ===
[file exist $f]-> 0 [file type $f]-> could not read
"/net/pub/WinXP/test/~[filename].txt": no such file or directory
[file type [file normalize $f]]-> file
[file type [subst -nocommands $f]]-> file
You see that [subst -nocommands $f] also circumvents the error like
[file normalize $f].
Same with the file which begins with "~$":
=== 0: File: /net/pub/WinXP/test/~$filename.txt ===
[file exist $f]-> 0 [file type $f]-> could not read
"/net/pub/WinXP/test/~$filename.txt": no such file or directory
[file type [file normalize $f]]-> file
[file type [subst -novariables $f]]-> file
Also here [subst -novariables $f] circumvents the error.
But I also confess that it may be another problem.
> tilde is another issue altogether. As noted in my bug report 2806250 -
> it is also dependent on the current working directory of your script.
Tilde expansion only is for "expanding" the beginning of the file and
not inside of a file path and in csh also "quoting" is possibe (but
not in tclsh).
gerhardr@nserver:/net/pub/WinXP/test>csh
nserver:/net/pub/WinXP/test> ls ~filename.txt
Unknown user: filename.txt.
nserver:/net/pub/WinXP/test> ls '~filename.txt'
~filename.txt
nserver:/net/pub/WinXP/test> cd ..
nserver:/net/pub/WinXP> ls /net/pub/WinXP/test/~filename.txt
/net/pub/WinXP/test/~filename.txt