Getting Mailcap to work with paths that include spaces

61 views
Skip to first unread message

mik13ST

unread,
Jul 26, 2020, 5:10:57 PM7/26/20
to lf-fm
Hello,

I know this isn't related to lf, but I figured this might be a suitable place to solve these problems because lf wiki suggests using Mailcap so there might be some user base here.

When I run
run-mailcap --debug "spaces in name/file.txt"  
the command hangs after
 - checking mailcap entry "text/*; less '%s'; needsterminal;"
 - program to execute: less '%s'
 - needsterminal is satisfied by stdout

Whereas if I run
run-mailcap --debug "no-spaces-in-name/file.txt"
less opens in the same terminal. Then I close less and see
 - checking mailcap entry "text/*; less '%s'; needsterminal;"
 - program to execute: less '%s'
 - needsterminal is satisfied by stdout
 - executing: less '/home/mik/mailcap-test/no-spaces-in-name/file.txt'

My .mailcap look like this
text/*; less '%s'; needsterminal;
which is syntactically identical to the example on lf wiki page.

I have tried googling around and found people with the same problem. The threads were usually old (2014) and mentioned it being fixed in some patch in 2015. This patch surely is on ArchLinux (mailcap version 2.1.49-1). Other threads suggested fiddling with the quotes in the config file and stuff so I did and it didn't improve anything.

Is this the correct place to ask for help? Should I go through the hassle of creating an account on Pagure (Fedora's GitHub or something) just to post this issue?

My intention is to use run-mailcap as a file opener for lf.  I like Mailcap because it's a mature project and has a single config file with simple syntax (especially compared to XDG Desktop Entry files).

Gokcehan Kara

unread,
Jul 26, 2020, 8:10:55 PM7/26/20
to mik13ST, lf-fm
Mailcap seems like a nice alternative.

Just to be clear, on Arch, `mailcap` package does not seem to provide `run-mailcap`, instead it comes from `run-mailcap` package in AUR, and that package seems to install a package from ubuntu repositories. I tried that and it seems to be failing as you described. I'm not sure if the bug report you linked is related to this problem. It seems that it is getting stuck in an infinite loop in one of the conditionals when there are spaces in the filename. Commenting out the conditional as below seems to make it work:

    480,491c480,491
    <             if (decode(langinfo(CODESET()), $file) =~ m![^[:alnum:],.:/@%^+=_-]!i) {
    <                 $match =~ m/nametemplate=(.*?)\s*($|;)/;
    <                 my $prefix = $1;
    <                 my $linked = 0;
    <                 while (!$linked) {
    <                     $tmplink = TempFile($prefix);
    <                     unlink($tmplink);
    <                     $linked = symlink($file,$tmplink);
    <                 }
    <                 $file = $tmplink;
    <                 print STDERR " - filename contains shell meta-characters; aliased to '$tmplink'\n" if $debug;
    <             }
    ---
    >             # if (decode(langinfo(CODESET()), $file) =~ m![^[:alnum:],.:/@%^+=_-]!i) {
    >             #     $match =~ m/nametemplate=(.*?)\s*($|;)/;
    >             #     my $prefix = $1;
    >             #     my $linked = 0;
    >             #     while (!$linked) {
    >             #         $tmplink = TempFile($prefix);
    >             #         unlink($tmplink);
    >             #         $linked = symlink($file,$tmplink);
    >             #     }
    >             #     $file = $tmplink;
    >             #     print STDERR " - filename contains shell meta-characters; aliased to '$tmplink'\n" if $debug;
    >             # }


I'm not good at Perl so I'm not sure what's being checked here. Our wiki page links to the man page of FreeBSD, so I suspect it was written by a FreeBSD user. They might have a different version of `run-mailcap` in FreeBSD which may not suffer from this problem.

Pagure page you linked seems to be related to Fedora `mailcap` package, which only seems to have the mailcap database but not the `run-mailcap` script, so I doubt they will be helpful. In `run-mailcap` man page, there seems to be an author address as below, which you might have a better luck:

run-mailcap (and its aliases) was written by Brian White <bcw...@pobox.com>.

Hope this helps.

Gokcehan

--
You received this message because you are subscribed to the Google Groups "lf-fm" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lf-fm+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lf-fm/eec0d571-f06d-448e-8f0b-761bcf4622afo%40googlegroups.com.

mojmir...@gmail.com

unread,
Jul 27, 2020, 4:45:34 PM7/27/20
to lf-fm
Thanks a lot for your research. I thought `run-mailcap` is just a simple wrapper/invoker for mailcap and that's why I didn't look there.

I attempted to run FreeBSD just to test this (using a Vagrant box) and it appears the problem is currently present there too (FreeBSD 11.4-STABLE; perl5.32.0). In that case, does it still make sense to keep it in the wiki? It isn't a major flaw and the wiki page might still help with using other file openers.

The patch you supplied seem to solve the issue. I am not sure whether I want to maintain the altered script on both of my computers though.

I will consider reporting this as a bug to the maintainers of `run-mailcap`, but I doubt anybody will do something about it.
To unsubscribe from this group and stop receiving emails from it, send an email to lf...@googlegroups.com.

Gokcehan Kara

unread,
Jul 27, 2020, 6:43:18 PM7/27/20
to mik13ST, lf-fm
If someone found it useful and put that page in the wiki, I'm not sure if it is a good idea to get rid of it altogether. There might be users who wouldn't mind the issue with filename spaces. Although it would be nice to add a note about it so people wouldn't waste time with it. If you decide to report the bug somewhere public, you can put a link to it as well so it would be possible to keep track of it. To be honest, it doesn't seem to be a difficult problem to fix if there are people willing to spend some time on it.

Gokcehan

To unsubscribe from this group and stop receiving emails from it, send an email to lf-fm+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lf-fm/f21835c4-62e6-4a06-82cf-8b46562e459do%40googlegroups.com.

mojmir...@gmail.com

unread,
Jul 28, 2020, 4:24:27 PM7/28/20
to lf-fm
I managed to find an alternative: `run-mailcap-rs`. It's a Rust rewrite of `run-mailcap` and doesn't suffer from that issue. I will give myself some time to test this alternative implementation, after that I will consider noting my experience on the wiki.

Even though the author of `run-mailcap-rs` states `run-mailcap` isn't being maintained, I have sent an email to the author of `run-mailcap` with the bug report. I will post updates here (if any).

whit...@gmail.com

unread,
Aug 5, 2020, 2:22:29 PM8/5/20
to lf-fm
It's been a long time since I wrote run-mailcap for Debian.  Like other shell meta-characters, spaces can be problematic.  While the possibilities aren't as obvious as allowing characters like quotes and semi-colons, there's no guarantee that multiple parameters won't be interpreted in special ways by whatever binary receives them.  Remember: There's nothing forcing a mailcap entry to quote the "%s" field; it's just best-practice.

IMPORTANT: If you comment out the block that handles files with special characters, you're opening every mailcap entry to a remote-code-execution attack by any user that can create an arbitrary filename and get the user to "see" it.

It sounds like there may be a problem with the way the temporary symlink (a name with no special characters) is generated.  It uses /bin/tempfile under the hood to create a unique file which is then removed and replaced with a symlink to the filename with those special characters.  Perhaps there are better ways to accomplish that today.

-- Brian
To unsubscribe from this group and stop receiving emails from it, send an email to lf...@googlegroups.com.

mojmir...@gmail.com

unread,
Aug 6, 2020, 2:04:32 PM8/6/20
to lf-fm
In order to verify which tempfile it uses, I decided to get an `strace` of the invocation. I ran `strace run-mailcap file.txt` inside the `/home/mik/mailcap-test/spaces in name/` directory. This is the interesting portion of the result. The "no such file or directory" error from the end of this output continues until the end of the strace (until I aborted the command).

openat(AT_FDCWD, "/home/mik/.mime.types", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/local/etc/mime.types", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/etc/mime.types", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/mime.types", O_RDONLY|O_CLOEXEC) = 3
ioctl(3, TCGETS, 0x7ffe260828a0)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=67454, ...}) = 0
read(3, "# This is a comment. I love comm"..., 8192) = 8192
read(3, "ion/pkcs8\t\t\t\tp8\napplication/pkcs"..., 8192) = 8192
read(3, "artisan\napplication/vnd.artsquar"..., 8192) = 8192
read(3, "on/vnd.geogebra.tool\t\t\tggt\nappli"..., 8192) = 8192
brk(0x565269384000)                     = 0x565269384000
read(3, "e.macroEnabled.12\tdotm\napplicati"..., 8192) = 8192
read(3, "nd.openxmlformats-officedocument"..., 8192) = 8192
read(3, "tion/vnd.uoml+xml\t\t\tuoml uo\nappl"..., 8192) = 8192
read(3, "\t\t\t\tjxs\nimage/jxsc\t\t\t\t\tjxsc\nimag"..., 8192) = 8192
read(3, "lash\t\t\tspl\napplication/x-gtar\t\t\t"..., 8192) = 1918
read(3, "", 8192)                       = 0
close(3)                                = 0
openat(AT_FDCWD, "/home/mik/.mailcap", O_RDONLY|O_CLOEXEC) = 3
ioctl(3, TCGETS, 0x7ffe260828a0)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=288, ...}) = 0
read(3, "text/*; nvim '%s'; needsterminal"..., 8192) = 288
read(3, "", 8192)                       = 0
close(3)                                = 0
openat(AT_FDCWD, "/etc/mailcap", O_RDONLY|O_CLOEXEC) = 3
ioctl(3, TCGETS, 0x7ffe260828a0)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=272, ...}) = 0
read(3, "### \n### Begin Red Hat Mailcap\n#"..., 8192) = 272
read(3, "", 8192)                       = 0
close(3)                                = 0
openat(AT_FDCWD, "/usr/local/etc/mailcap", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/etc/mailcap", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/etc/mailcap", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("./file.txt", {st_mode=S_IFREG|0644, st_size=10, ...}) = 0
stat("./file.txt", {st_mode=S_IFREG|0644, st_size=10, ...}) = 0
geteuid()                               = 1000
geteuid()                               = 1000
ioctl(1, TCGETS, 0x7ffe26082b20)        = -1 ENOTTY (Inappropriate ioctl for device)
getcwd("/home/mik/mailcap-test/spaces in name", 4095) = 38
pipe2([3, 4], O_CLOEXEC)                = 0
fcntl(3, F_GETFD)                       = 0x1 (flags FD_CLOEXEC)
pipe2([5, 6], O_CLOEXEC)                = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f89e6612a10) = 1045289
close(6)                                = 0
close(4)                                = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=1045289, si_uid=1000, si_status=1, si_utime=0, si_stime=0} ---
read(5, "\2\0\0\0", 4)                  = 4
close(5)                                = 0
close(3)                                = 0
wait4(1045289, [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 1045289
lstat("", 0x7ffe26082b00)               = -1 ENOENT (No such file or directory)
symlink("/home/mik/mailcap-test/spaces in name/file.txt", "") = -1 ENOENT (No such file or directory)
pipe2([3, 4], O_CLOEXEC)                = 0
pipe2([5, 6], O_CLOEXEC)                = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f89e6612a10) = 1045290
close(6)                                = 0
close(4)                                = 0
read(5, "\2\0\0\0", 4)                  = 4
close(5)                                = 0
close(3)                                = 0
wait4(1045290, [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 1045290
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=1045290, si_uid=1000, si_status=1, si_utime=0, si_stime=0} ---
lstat("", 0x7ffe26082b00)               = -1 ENOENT (No such file or directory)
symlink("/home/mik/mailcap-test/spaces in name/file.txt", "") = -1 ENOENT (No such file or directory)
pipe2([3, 4], O_CLOEXEC)                = 0
pipe2([5, 6], O_CLOEXEC)                = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f89e6612a10) = 1045291
close(6)                                = 0
close(4)                                = 0
read(5, "\2\0\0\0", 4)                  = 4
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=1045291, si_uid=1000, si_status=1, si_utime=0, si_stime=0} ---
close(5)                                = 0
close(3)                                = 0
wait4(1045291, [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 1045291
lstat("", 0x7ffe26082b00)               = -1 ENOENT (No such file or directory)
symlink("/home/mik/mailcap-test/spaces in name/file.txt", "") = -1 ENOENT (No such file or directory)
pipe2([3, 4], O_CLOEXEC)                = 0
pipe2([5, 6], O_CLOEXEC)                = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f89e6612a10) = 1045292
close(6)                                = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=1045292, si_uid=1000, si_status=1, si_utime=0, si_stime=0} ---

whit...@gmail.com

unread,
Aug 6, 2020, 5:38:54 PM8/6/20
to lf-fm
I love strace.

symlink("/home/mik/mailcap-test/spaces in name/file.txt", "")

I can tell you right now that the call to /bin/tempfile is failing, hence the empty second string.  Does the binary exist on your system?

-- Brian

mojmir...@gmail.com

unread,
Aug 7, 2020, 3:00:52 PM8/7/20
to lf-fm
No, there is no `/bin/tempfile` binary on my system. There is no such thing in the official nor user repositories of my distribution either. I also tried searching for a package that provides the `/bin/tempfile` (or only `tempfile`) file using my distributions package manager, but it still yielded no results (it cannot perform this search in the user repository though).

Then I tried searching the internet for `linux "tempfile" bin` which lead me to Ubuntu man page of the tempfile tool which mentions it being a part of the debianutils package. Debianutils package is available in the user repositories and after installing that the tempfile tool is available. This solves the problems with run-mailcap and it is now capable of opening files with spaces in path. Thank you, Brian.

I will note this on the `lf` wiki.

whit...@gmail.com

unread,
Aug 7, 2020, 3:13:45 PM8/7/20
to lf-fm
You could also rewrite the TempFile function to do it another way.  It essential to have that function working or many things will break, including reading from stdin, compose, etc.

Glad it's working for you now.

-- Brian

Gokcehan Kara

unread,
Aug 22, 2020, 3:41:07 PM8/22/20
to lf-fm
Thank you both for identifying the issue. I tried installing `debianutils` and it seems to be working fine. I think it is worth notifying `run-mailcap` AUR maintainers to add `debianutils` as a dependency to not have this issue in the future, though I have had some keyring issues installing `debianutils`.

Quotes in wiki pages seem to be redundant, since filenames with spaces already work without quotes. I don't know why they were added by the original author. Also `OPENER=run-mailcap` does not seem to work well since the default `open` command is a `&` command, so this would not work to open a text file in a terminal editor. This is why we have the custom `open` example in our example configuration file. I think that example can be adapted `run-mailcap`.

Brian, I think it's worth changing TempFile to use `mktemp` instead of `tempfile` to be more posix compliant, but I understand if you're not interested in working on it.

Rust rewrite is good but it might be difficult to install if you don't have rust installed.

In the meantime, I realized Python standard library has both mimetypes and mailcap modules, so it's trivial to write a similar script since heavy lifting is done by these modules. I tried to create such a script myself and uploaded it to pypi if you're interested:


I hope this might be easier to install and maintain. I might add a link to the wiki page later on.

Gokcehan

--
You received this message because you are subscribed to the Google Groups "lf-fm" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lf-fm+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lf-fm/3aadd0f0-a208-4a12-81c6-69c367b36448o%40googlegroups.com.

Brian White

unread,
Aug 22, 2020, 5:57:19 PM8/22/20
to Gokcehan Kara, lf-fm
Thank you both for identifying the issue. I tried installing `debianutils` and it seems to be working fine. I think it is worth notifying `run-mailcap` AUR maintainers to add `debianutils` as a dependency to not have this issue in the future, though I have had some keyring issues installing `debianutils`.

Quotes in wiki pages seem to be redundant, since filenames with spaces already work without quotes. I don't know why they were added by the original author. Also `OPENER=run-mailcap` does not seem to work well since the default `open` command is a `&` command, so this would not work to open a text file in a terminal editor. This is why we have the custom `open` example in our example configuration file. I think that example can be adapted `run-mailcap`.

Quotes around the %s in mailcap entries are important because not all applications that launch programs via mailcap are necessarily smart enough to create aliases for filenames with spaces or even other shell metacharacters.  Run-mailcap plays it completely safe because it's possible that some malicap entry might not have the quotes, notably in a user-created ~/.mailcap file.
 

Brian, I think it's worth changing TempFile to use `mktemp` instead of `tempfile` to be more posix compliant, but I understand if you're not interested in working on it.

It's been too long for me to remember all (or even any) of the details but I did try several "make temp file" mechanisms before settling on "tempfile" to do the heavy lifting.  Of course, programs have evolved since I wrote run-mailcap so that might not be what I'd choose today.
 

Rust rewrite is good but it might be difficult to install if you don't have rust installed.

In the meantime, I realized Python standard library has both mimetypes and mailcap modules, so it's trivial to write a similar script since heavy lifting is done by these modules. I tried to create such a script myself and uploaded it to pypi if you're interested:


I hope this might be easier to install and maintain. I might add a link to the wiki page later on.

I don't work on Debian any longer.  I don't even have a build system.  Just be careful if you're attempting something new: Remember that you're creating an interface that allows untrusted people to run arbitrary commands on the user's machine.  A mistake can be a serious problem!  Mailcap was created in the days where such things were expected to be used only as intended; it is not inherently secure.

  Brian
------------------------------------------------------------------------------------------
Treat someone as they are and they will remain that way.
Treat someone as they can be and they will become that way.

 
You received this message because you are subscribed to a topic in the Google Groups "lf-fm" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lf-fm/wkef5VJdynw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lf-fm+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lf-fm/CAEWroOhS7X4C7meiVp5iQHbFiSY%3DYxV68rQKvY8WP%2BMoDNYTMA%40mail.gmail.com.

Gokcehan Kara

unread,
Aug 22, 2020, 7:04:04 PM8/22/20
to Brian White, lf-fm
Thanks for the tips Brian.

Quotes around the %s in mailcap entries are important because not all applications that launch programs via mailcap are necessarily smart enough to create aliases for filenames with spaces or even other shell metacharacters.  Run-mailcap plays it completely safe because it's possible that some malicap entry might not have the quotes, notably in a user-created ~/.mailcap file.

Are you aware of any other programs using the mailcap system. On my machine, firefox seems to be the only package depending on mailcap and I'm not sure if they are even using it or not. There is no mention of quoting in the mailcap standard and none of the examples have quotes so I have implemented mopen to automatically quote filenames. It doesn't even work when there are quotes in the mailcap file.

It's been too long for me to remember all (or even any) of the details but I did try several "make temp file" mechanisms before settling on "tempfile" to do the heavy lifting.  Of course, programs have evolved since I wrote run-mailcap so that might not be what I'd choose today.
I don't work on Debian any longer.  I don't even have a build system.

It would have been nice if `run-mailcap` code was hosted somewhere, if not by you then maybe by someone else interested. AUR package in Arch tries to extract the code by pulling a package from an Ubuntu repository. I don't know Perl much but I think there might be people who are willing to work on such a patch. It seems that many systems still have `run-mailcap` installed, and I have even seen it used as a fallback in my `xdg-open` script, so I think it would be a shame to let it rot.

Just be careful if you're attempting something new: Remember that you're creating an interface that allows untrusted people to run arbitrary commands on the user's machine.  A mistake can be a serious problem!  Mailcap was created in the days where such things were expected to be used only as intended; it is not inherently secure.

This is specifically mentioned in shlex.quote function doc:


I used the same logic in mopen to implement quoting.

Gokcehan

Brian White

unread,
Sep 7, 2020, 6:10:21 PM9/7/20
to Gokcehan Kara, lf-fm

Quotes around the %s in mailcap entries are important because not all applications that launch programs via mailcap are necessarily smart enough to create aliases for filenames with spaces or even other shell metacharacters.  Run-mailcap plays it completely safe because it's possible that some malicap entry might not have the quotes, notably in a user-created ~/.mailcap file.

Are you aware of any other programs using the mailcap system. On my machine, firefox seems to be the only package depending on mailcap and I'm not sure if they are even using it or not. There is no mention of quoting in the mailcap standard and none of the examples have quotes so I have implemented mopen to automatically quote filenames. It doesn't even work when there are quotes in the mailcap file.

To my knowledge, it was never widely used.  It's not applicable to much other than things like web browsers or file navigators.  Even on my work system, which is based on Debian, there's another system used to figure out what program to run when I ctrl-click on a link in a console window.
 

It's been too long for me to remember all (or even any) of the details but I did try several "make temp file" mechanisms before settling on "tempfile" to do the heavy lifting.  Of course, programs have evolved since I wrote run-mailcap so that might not be what I'd choose today.
I don't work on Debian any longer.  I don't even have a build system.

It would have been nice if `run-mailcap` code was hosted somewhere, if not by you then maybe by someone else interested. AUR package in Arch tries to extract the code by pulling a package from an Ubuntu repository. I don't know Perl much but I think there might be people who are willing to work on such a patch. It seems that many systems still have `run-mailcap` installed, and I have even seen it used as a fallback in my `xdg-open` script, so I think it would be a shame to let it rot.

It was written for Debian rather than as a general purpose tool.  It's the kind of thing that needs to be integrated, not something you'd use as a stand-alone product.  As such, it's "hosted" in the Debian source package.  Note that I haven't touched it in 15 years or so; I'm not sure it needs much maintenance.
 
-- Brian
Reply all
Reply to author
Forward
0 new messages