Why does ioutil/TempFile take a prefix instead of a suffix?

[meta keule]

Hi,

can somebody enlighten me why ioutil/TempFile takes a prefix for a filename instead of a suffix?

Everytime I need it, I also need to make sure to keep the file extension. 

That would be easily achieved with a suffix. 

The way it works now, I can't use TempFile or need to move the generated file 

(what may result in a conflict).

Regards,

Benny

[Jan Mercl]

File name extensions are merely hints/tags for humans. Most (well
designed) programs can/should cope with a proper file regardless of
its extension, in my experience.

I've heard some rumors about an "OS" where simply changing the
extension gave the file execution rights, but I don't believe that.
It's way to silly, isn't it? ;-)

-j

[meta keule]

Yes, but a browser needs a proper file extension for a download and a
email client to.
There are countless non windows tools that uses file extensions to
determine the mime type.
Often you don't even have another chance, for example if your cli
program accepts only
a path to a file and does require to get the type by extension
guessing.

So what is your argument? What is the advantage of having a prefix
here?
It looks to me that a suffix could offer the same, but the additional
quality of allowing to
define the file extension.

Prove me wrong!

On 27 Mai, 09:28, Jan Mercl <0xj...@gmail.com> wrote:

[Dave Cheney]

I think the reason, more than anything else is: that is how mktemp(1) works.

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[chris dollin]

On 27 May 2013 09:34, meta keule <marcre...@googlemail.com> wrote:

Yes, but a browser needs a proper file extension for a download and a
email client to.

Should a browser be using the negotiated media type, not an accident

of the filename [however handy that is to humans]?

Chris

--
Chris "allusive" Dollin

[David Symonds]

On Mon, May 27, 2013 at 5:28 PM, Jan Mercl <0xj...@gmail.com> wrote:

> File name extensions are merely hints/tags for humans. Most (well
> designed) programs can/should cope with a proper file regardless of
> its extension, in my experience.

FYI, our beloved go tool insists on a .go extension.

[Kyle Lemons]

One is always free to rename the file afterward, or to copy/paste the implementation of TempFile.  It's in ioutil because it is frequently sufficient, with the full understanding that it may not always be so.

--

[meta keule]

You don't always have a mime type.

Also there have been some bugs in browsers, if you deliver images with correct mime type, but

without proper extension.

What about unpacking a zip file? Maybe you get it from windows?

I don't know, but I can't imagine that a zip file would preserve file type info, such as

the ones encoded in mime types.

Please you file extension agnostics, can you tell me, in which world do you live and which drugs did you take?

Do you really want to tell me, that there is no need to keep a file extension?

I can't believe it.

[Kyle Lemons]

On Mon, May 27, 2013 at 1:55 AM, meta keule <marcre...@googlemail.com> wrote:

You don't always have a mime type.

Also there have been some bugs in browsers, if you deliver images with correct mime type, but

without proper extension.

This seems tangential to anything about filenames from TempFile.

 

What about unpacking a zip file? Maybe you get it from windows?

I don't know, but I can't imagine that a zip file would preserve file type info, such as

the ones encoded in mime types.

Files on disk and in archives are, at least in the *nix world, often identified by magic numbers, not file extensions.

 

Please you file extension agnostics, can you tell me, in which world do you live and which drugs did you take?

This statement is unhelpful, and isn't really appropriate for a mailing list through which you wish to receive help and guidance on technical matters.  Consider the number of man-hours that will be spent reading your email by the Gophers around the world before you click "Send."

 

Do you really want to tell me, that there is no need to keep a file extension?

When creating temporary files, I cannot recall a time where I have been required to adhere to any file naming standards at all.  They're just that: temporary.  If they need to exist for a longer period of time, at that point I move them where they are required, with an appropriate name.

Downloading images and/or attachments might be done to a temporary file, but again you can (and probably should) rename the temporary file and/or move it according to the name the server or URI suggests.

 

I can't believe it.

Am Montag, 27. Mai 2013 10:42:18 UTC+2 schrieb chris dollin:

On 27 May 2013 09:34, meta keule <marcre...@googlemail.com> wrote:

Yes, but a browser needs a proper file extension for a download and a
email client to.

Should a browser be using the negotiated media type, not an accident

of the filename [however handy that is to humans]?

Chris

--
Chris "allusive" Dollin

--

[meta keule]

You're telling me something new.

Did you read my post?

While renaming is an additional effort, that take access to the file system and therefor is slow it is also

unsafe, because there might be another file with this name already 

(TempFile would not check it, since it checks for the existence of the name it is going to use not the name + some extension).

But really, I am yet to hear any reasonable argument.

I will make a ticket and see, if the maintainer are going to change it sometime.

Their should not be much code that would break.

If we are never going to improve things, the things won't get better.

[Jan Mercl]

On Mon, May 27, 2013 at 10:55 AM, meta keule
<marcre...@googlemail.com> wrote:
> Please you file extension agnostics, can you tell me, in which world do you
> live and which drugs did you take?

(I don't like the quoted sentence.) Extensions are useful. Attaching
semantics to an extension is not a good idea, see eg. file(1). Abusing
extension to be the only source of file's semantics, as done by some
(poorly designed) OSs, is not a mistake. That's called "defective by
design", "bug" and "attack vector".

> Do you really want to tell me, that there is no need to keep a file extension?

No, no one said that. It's as good as e.g a file name is. Humans are
good in recalling names. Computers can handle inodes much better.
Actually using names just slows them down.

> I can't believe it.

Proves what?

-j

[meta keule]

Am Montag, 27. Mai 2013 11:08:51 UTC+2 schrieb Jan Mercl:

On Mon, May 27, 2013 at 10:55 AM, meta keule
<marcre...@googlemail.com> wrote:
> Please you file extension agnostics, can you tell me, in which world do you
> live and which drugs did you take?

(I don't like the quoted sentence.) Extensions are useful. Attaching
semantics to an extension is not a good idea, see eg. file(1). Abusing
extension to be the only source of file's semantics, as done by some
(poorly designed) OSs, is not a mistake. That's called "defective by
design", "bug" and "attack vector".

> Do you really want to tell me, that there is no need to keep a file extension?

No, no one said that.

 

But that was all my question was about: Why not allow suffix in order to keep file extensions.

Then you and others argue are implying that the current behavior is perfect, that we can rename,

that one should not rely on file extension and so on. That is not the point. 

In this world there is some computational reliance on file extensions whether you like it or not 

and sometimes we have to deal with it, if we want to make usable real world applications.

Why not make a simple change to allow to keep an information that is - while not reliable and perfect -

usable in lots of scenarios instead of throwing it away?

You look narrow minded to me.

[Jan Mercl]

On Mon, May 27, 2013 at 11:07 AM, meta keule
<marcre...@googlemail.com> wrote:
> I will make a ticket and see, if the maintainer are going to change it
> sometime.

No need for that. There's a Go compatibility promise. The suffix to
prefix change is not possible.

-j

[meta keule]

BTW

Isn't go supposed to be multi plattform?

Now you already have mentioned a plattform where file extensions are significant.

The tempfile would not be very useful on windows without renaming. 

At least not for other programs. So on windows TempFile is nearly useless.

[meta keule]

BINGO.

I guess, there will never be a go 2.

Hurd would be stable before that happens.

Until then we don't need tickets for go 2 nor language improvements.

Right?

[Michael Jones]

If the tmpfile basenames are unique, then would not making a new file from BASENAME+EXT also be unique? Just create a file with the name you're given plus the extension you like appended...

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Michael T. Jones | Chief Technology Advocate  | m...@google.com |  +1 650-335-5765

[meta keule]

Am Montag, 27. Mai 2013 11:04:50 UTC+2 schrieb Kyle Lemons:

On Mon, May 27, 2013 at 1:55 AM, meta keule <marcre...@googlemail.com> wrote:

You don't always have a mime type.

Also there have been some bugs in browsers, if you deliver images with correct mime type, but

without proper extension.

This seems tangential to anything about filenames from TempFile.

For example: uploaded file that should be served only for some time (temporary).

 

What about unpacking a zip file? Maybe you get it from windows?

I don't know, but I can't imagine that a zip file would preserve file type info, such as

the ones encoded in mime types.

Files on disk and in archives are, at least in the *nix world, often identified by magic numbers, not file extensions.

But we live in a mixed world.

 

 

Please you file extension agnostics, can you tell me, in which world do you live and which drugs did you take?

This statement is unhelpful, and isn't really appropriate for a mailing list through which you wish to receive help and guidance on technical matters.  Consider the number of man-hours that will be spent reading your email by the Gophers around the world before you click "Send."

Well frankly your answers weren't helpful, but noise.

I simple

"No, there is no reason other than, 'it is how mktemp works',

but make a ticket for go 2 if you like, since the change would break

compatibility promise".

or

"Yes there is the following reason [some real reason]"

would have been sufficient.

But instead one gets lots of argument why one should not rely on file extensions

and why you don't need this and why tools are wrong that rely on extensions.

I did not want to get a lecture, but a simple answer.

The only helpful answer was given by Dave Cheney.

[Jan Mercl]

On Mon, May 27, 2013 at 12:12 PM, meta keule
<marcre...@googlemail.com> wrote:
> BINGO.
>
> I guess, there will never be a go 2.
>
> Hurd would be stable before that happens.
>
> Until then we don't need tickets for go 2 nor language improvements.
>
> Right?

http://godoc.org/github.com/cznic/fileutil#TempFile

-j

[meta keule]

Am Montag, 27. Mai 2013 11:04:50 UTC+2 schrieb Kyle Lemons:

On Mon, May 27, 2013 at 1:55 AM, meta keule <marcre...@googlemail.com> wrote:

You don't always have a mime type.

Also there have been some bugs in browsers, if you deliver images with correct mime type, but

without proper extension.

This seems tangential to anything about filenames from TempFile.

For example: uploaded file that should be served only for some time (temporary).

 

What about unpacking a zip file? Maybe you get it from windows?

I don't know, but I can't imagine that a zip file would preserve file type info, such as

the ones encoded in mime types.

Files on disk and in archives are, at least in the *nix world, often identified by magic numbers, not file extensions.

But we live in a mixed world.

 

 

Please you file extension agnostics, can you tell me, in which world do you live and which drugs did you take?

This statement is unhelpful, and isn't really appropriate for a mailing list through which you wish to receive help and guidance on technical matters.  Consider the number of man-hours that will be spent reading your email by the Gophers around the world before you click "Send."

Well frankly your answers weren't helpful, but noise.

A simple

"No, there is no reason other than, 'it is how mktemp works',

but make a ticket for go 2 if you like, since the change would break

compatibility promise".

or

"Yes there is the following reason [some real reason]"

would have been sufficient.

But instead one gets lots of argument why one should not rely on file extensions

and why you don't need this and why tools are wrong that rely on extensions.

I did not want to get a lecture, but a simple answer.

The only reasonable answer was given by Dave Cheney

[meta keule]

The tmpfile name is unique only for the time of the creation of the file, not for the time of the renaming.

Although it might be no problem, if only tmpfile writes into the directory.

But theoretically the following might happen if you create  tempfiles in a loop

- tempfile '453acd' created

- move to '453acd.jpg'

- tempfile '453acd' created (since '453acd' is no longer there)

- move to '453acd.jpg' /// overwriting the original

But I guess the implementation of TempFile will have some timestamp based information

(didn't check it), so that won't be a problem, as long as you are not creating them in

different goroutine at the same time.

[David Symonds]

For the couple of times I've needed control over the full filename
(including an extension), I've just used io/ioutil's TempDir function.
It's especially handy if you're making multiple files, or you are
going to run a subprocess that might use filenames in error messages.

[Steve McCoy]

On Monday, May 27, 2013 5:04:50 AM UTC-4, Kyle Lemons wrote:

Files on disk and in archives are, at least in the *nix world, often identified by magic numbers, not file extensions.

 

Not text files.

[meta keule]

Thanks, the 2nd useful answer appeared.

On 27 Mai, 12:44, Jan Mercl <0xj...@gmail.com> wrote:
> On Mon, May 27, 2013 at 12:12 PM, meta keule
>

[Steve McCoy]

Seconded. The dir acts as a fine prefix. 

[Michael Jones]

I was thinking to leave the first file alone, then (as I said) *create* a new one of the same name but with your desired extension. That should work fine, but I like the tmpdir approach better.

[Jan Mercl]

(13:06) jnml@fsc-r550:~/tmp$ echo blah > foo
(13:06) jnml@fsc-r550:~/tmp$ echo Mūhldorf > bar
(13:07) jnml@fsc-r550:~/tmp$ file foo bar
foo: ASCII text
bar: UTF-8 Unicode text
(13:07) jnml@fsc-r550:~/tmp$

-j

[meta keule]

Good idea, will try.

[Jan Mercl]

On Mon, May 27, 2013 at 12:59 PM, meta keule
<marcre...@googlemail.com> wrote:
> Thanks, the 2nd useful answer appeared.

My intention was purely to show that instead of keeping bitching into
the mailing box of several thousand list subscribers is less
productive and uses way more more time than spending the required _5_
minutes to modify the open sourced function, make it pass a test and
publish it on some hosting service.

-j

[Dan Kortschak]

On Mon, 2013-05-27 at 13:07 +0200, Jan Mercl wrote:
> (13:06) jnml@fsc-r550:~/tmp$ echo blah > foo
> (13:06) jnml@fsc-r550:~/tmp$ echo Mūhldorf > bar
> (13:07) jnml@fsc-r550:~/tmp$ file foo bar
> foo: ASCII text
> bar: UTF-8 Unicode text
> (13:07) jnml@fsc-r550:~/tmp$

It's not always that nice. Sure, you know their encoding, but not what
they encode. Trust me, while I agree much of the use of extensions is
pointless, there are times when they are needed - my field is an
unfortunate case of this [1]. BioPerl has a package that implements
format guessing, and often gets it correct, but extensions are safer.

[1] http://www.biostars.org/p/7126/#7136

[meta keule]

On 27 Mai, 13:07, Michael Jones <m...@google.com> wrote:
> I was thinking to leave the first file alone, then (as I said) *create* a
> new one of the same name but with your desired extension. That should work
> fine, but I like the tmpdir approach better.

Me too.
You approach would produce more overhead (access to fs).

[Steve McCoy]

On Monday, May 27, 2013 7:07:47 AM UTC-4, Jan Mercl wrote:

(13:06) jnml@fsc-r550:~/tmp$ echo blah > foo
(13:06) jnml@fsc-r550:~/tmp$ echo Mūhldorf > bar
(13:07) jnml@fsc-r550:~/tmp$ file foo bar
foo: ASCII text
bar: UTF-8 Unicode text
(13:07) jnml@fsc-r550:~/tmp$

-j

Those files do not contain any magic numbers to identify the content; "file" is using other heuristics in their absence and is often inaccurate or wrong. E.g.

% echo '{"frogcount": 123, "frogsound": "ribbit"}' > frob

% file frob

frob: ASCII text

or

% echo GIF89a is great > frob

% file frob

frob: GIF image data, version 89a, 26912 x 8307

[Jan Mercl]

On Mon, May 27, 2013 at 1:18 PM, Steve McCoy <mcc...@gmail.com> wrote:
> % echo '{"frogcount": 123, "frogsound": "ribbit"}' > frob
> % file frob
> frob: ASCII text
>
> or
>
> % echo GIF89a is great > frob
> % file frob
> frob: GIF image data, version 89a, 26912 x 8307

Yep, using ASCII for magic numbers is not a good idea. I know 'file'
is and cannot be 100% accurate nor deterministic. Yet I haven't met a
case where it failed for me.

-j

[meta keule]

[ ] you did read my post
[ ] you did answer my post
[x] you initialized a thread about the unimportance of file
extensions,
been proven wrong by me and others
[x] you signaled that go would not improve the current implementation
[x] I responded to signal that I did not want to bitch, but a useful
answer, and to because, I couldn't have done it by myself.
Do you want the core library to be useful or not?

Definition of a useful answer
[x] your concern is valid, but we are not able to fix it now / will
fix it later
[x] your concern is invalid, because of ....
[x] look, that's how we typically solve this problem [link]
[x] look, we have this nice TempDir, use this instead
[ ] why do you rely on file extensions
[ ] software should not rely on file extensions
[ ] we won't change that, because we keep everything as is
[ ] why do you bitch around, do it yourself

Ok, I'm done with this thread.
Thanks for the kind of answers that were helpful.

On 27 Mai, 13:10, Jan Mercl <0xj...@gmail.com> wrote:
> On Mon, May 27, 2013 at 12:59 PM, meta keule
>

[Jan Mercl]

On Mon, May 27, 2013 at 1:35 PM, meta keule <marcre...@googlemail.com> wrote:

Everyone can be as much ad hominem as she wants - or expect the
community to be helpful. Pick one.

-j

[meta keule]

And sometimes no answer is better than questioning
the legitimacy.

Unfortunately there is some tendency on this list to react on
reasonable questions / proposals by questioning
the legitimacy of the proposal and holding lectures about
how to code (like in "You should avoid this and that, you
should rely on this and that")

You may say, that this is not ad hominem and on the surface
it might not look like that, but you suppose the poster is not
experienced or a moron. Such a reaction is called
"passive aggression" and is it often accepted in very homogenous
groups.

That is called "group thinking" and it reliable prevents the
development
of a group.

There should be some effort to stay open minded, even if hurts.
Otherwise you will get nowhere.

[/Lecture closed]

On 27 Mai, 13:39, Jan Mercl <0xj...@gmail.com> wrote:

[jens....@tink.se]

In case anyone stumbles across this thread, here's a tempfile library that supports file suffix: https://github.com/tink-ab/tempfile

Also, if you'd like to track this upstream in Golang standard library, you can do that here: https://github.com/golang/go/issues/4896

Cheers,

Jens