How to use Pub in production?

154 views
Skip to first unread message

Mark Haase

unread,
Feb 23, 2015, 11:39:29 AM2/23/15
to mi...@dartlang.org
I really like Dart -- a lot -- but the Dart ecosystem is... meh. Pub is especially confounding. I find myself fighting against it and trying to work around it more often than not.

How is Pub envisioned for use on production servers? I've run into a few problems while trying to migrate Dart applications from my development server to a production server. (The application uses a Dart client and a Python application server, if that matters.)

The main issue is the pub cache. By default, it is stored in ~/.pub-cache, which quickly gets annoying if you want to install as one user but run as a different user. E.g. I'm used to installing most of my application files as root but running the application as a less privileged user. With default pub settings, my packages will get cached in /root/.pub-cache but the application will look in /home/app-user/.pub-cache.

I see there is a PUB_CACHE environment variable, so I tried setting that to a global location and then running `sudo pub get`. This still doesn't work, because (for some unknown reason), the package directories are set with 700 permissions: 

root@localhost:/opt/pub-cache/hosted/pub.dartlang.org# l
total 112K
drwxr-xr-x 28 root root 4.0K Feb 23 16:20 .
drwxr-xr-x  3 root root 4.0K Feb 23 16:19 ..
drwx------  6 root root 4.0K Feb 23 16:19 analyzer-0.15.7
drwx------ 12 root root 4.0K Feb 23 16:19 angular-1.0.0
drwx------  5 root root 4.0K Feb 23 16:19 args-0.10.0+2
drwx------  5 root root 4.0K Feb 23 16:19 barback-0.14.2
drwx------  5 root root 4.0K Feb 23 16:19 bootjack-0.6.5+2
drwx------  3 root root 4.0K Feb 23 16:19 browser-0.10.0+2
drwx------  4 root root 4.0K Feb 23 16:19 code_transformers-0.1.6
drwx------  4 root root 4.0K Feb 23 16:19 collection-0.9.4

Root's umask is 0022, which I confirmed by starting a root shell and touching a new file and mkdir'ing a new directory. The files and folders inside these packages have reasonable permissions, but for some reason this top level does not.

I found a thread here from Feb 2013 with a similar topic, but it ended without any resolution. It's now 2 years later, so I'm thinking the resolution might have changed anyway.

I've noticed that the Dart SDK .zip file also has obnoxious privileges: it also has 700 for its top level directory. (I'm not too swift with .zip; I didn't even realize that it could contain unix file permissions. Why isn't Dart distributed as a tarball?)

Any advice? Who's running Dart in production? How do you work around these issues?

Is there any way to disable pub caching completely? I don't see a significant benefit. I'm not downloading gigs of Dart packages every day, and the time saved by caching has been outweighed 10000-to-1 by time spent fiddling with Pub.

Bob Nystrom

unread,
Feb 23, 2015, 7:47:04 PM2/23/15
to General Dart Discussion
On Mon, Feb 23, 2015 at 8:39 AM, Mark Haase <meh...@gmail.com> wrote:
I really like Dart -- a lot -- but the Dart ecosystem is... meh. Pub is especially confounding. I find myself fighting against it and trying to work around it more often than not.

How is Pub envisioned for use on production servers? I've run into a few problems while trying to migrate Dart applications from my development server to a production server. (The application uses a Dart client and a Python application server, if that matters.)

Most users run pub build and deploy the output from that. Your build output physically contains all of the dependencies, so there are no more symlinks or usage of the cache.

The main issue is the pub cache. By default, it is stored in ~/.pub-cache, which quickly gets annoying if you want to install as one user but run as a different user. E.g. I'm used to installing most of my application files as root but running the application as a less privileged user. With default pub settings, my packages will get cached in /root/.pub-cache but the application will look in /home/app-user/.pub-cache.

I see there is a PUB_CACHE environment variable, so I tried setting that to a global location and then running `sudo pub get`. This still doesn't work, because (for some unknown reason), the package directories are set with 700 permissions: 

root@localhost:/opt/pub-cache/hosted/pub.dartlang.org# l
total 112K
drwxr-xr-x 28 root root 4.0K Feb 23 16:20 .
drwxr-xr-x  3 root root 4.0K Feb 23 16:19 ..
drwx------  6 root root 4.0K Feb 23 16:19 analyzer-0.15.7
drwx------ 12 root root 4.0K Feb 23 16:19 angular-1.0.0
drwx------  5 root root 4.0K Feb 23 16:19 args-0.10.0+2
drwx------  5 root root 4.0K Feb 23 16:19 barback-0.14.2
drwx------  5 root root 4.0K Feb 23 16:19 bootjack-0.6.5+2
drwx------  3 root root 4.0K Feb 23 16:19 browser-0.10.0+2
drwx------  4 root root 4.0K Feb 23 16:19 code_transformers-0.1.6
drwx------  4 root root 4.0K Feb 23 16:19 collection-0.9.4

Root's umask is 0022, which I confirmed by starting a root shell and touching a new file and mkdir'ing a new directory. The files and folders inside these packages have reasonable permissions, but for some reason this top level does not.

I found a thread here from Feb 2013 with a similar topic, but it ended without any resolution. It's now 2 years later, so I'm thinking the resolution might have changed anyway.

Weird. I recall that issue, but I'm not at all an expert on how permissions work on Unix. We're just using the normal dart:io APIs to create directories, symlinks, etc.

This may be because we create a new directory in there by creating a temp directory and then rename it after the download completes. (That way we don't leave a broken directory if something fails.) Perhaps that affects the permissions?

Is there any way to disable pub caching completely? I don't see a significant benefit. I'm not downloading gigs of Dart packages every day, and the time saved by caching has been outweighed 10000-to-1 by time spent fiddling with Pub.

No, there isn't. The packages you use have to be downloaded somewhere. Since they are the same for each application that uses them, it makes sense to have a shared cache.

We could definitely support installing them locally in your application's package too, but never had a chance to do that. If you think it would be helpful, can you file a bug?

Thanks!

- bob

Günter Zöchbauer

unread,
Feb 24, 2015, 1:04:40 AM2/24/15
to mi...@dartlang.org
Pub build is basically for client side codo. I saw there was some work done on dart2dart but I guess it's still not officially supported.

Rico Wind

unread,
Feb 24, 2015, 2:55:17 AM2/24/15
to General Dart Discussion, Søren Gjesse

+sgjesse
On Tue, Feb 24, 2015 at 1:46 AM, 'Bob Nystrom' via Dart Misc <mi...@dartlang.org> wrote:

On Mon, Feb 23, 2015 at 8:39 AM, Mark Haase <meh...@gmail.com> wrote:
I really like Dart -- a lot -- but the Dart ecosystem is... meh. Pub is especially confounding. I find myself fighting against it and trying to work around it more often than not.

How is Pub envisioned for use on production servers? I've run into a few problems while trying to migrate Dart applications from my development server to a production server. (The application uses a Dart client and a Python application server, if that matters.)

Most users run pub build and deploy the output from that. Your build output physically contains all of the dependencies, so there are no more symlinks or usage of the cache.

The main issue is the pub cache. By default, it is stored in ~/.pub-cache, which quickly gets annoying if you want to install as one user but run as a different user. E.g. I'm used to installing most of my application files as root but running the application as a less privileged user. With default pub settings, my packages will get cached in /root/.pub-cache but the application will look in /home/app-user/.pub-cache.

I see there is a PUB_CACHE environment variable, so I tried setting that to a global location and then running `sudo pub get`. This still doesn't work, because (for some unknown reason), the package directories are set with 700 permissions: 

root@localhost:/opt/pub-cache/hosted/pub.dartlang.org# l
total 112K
drwxr-xr-x 28 root root 4.0K Feb 23 16:20 .
drwxr-xr-x  3 root root 4.0K Feb 23 16:19 ..
drwx------  6 root root 4.0K Feb 23 16:19 analyzer-0.15.7
drwx------ 12 root root 4.0K Feb 23 16:19 angular-1.0.0
drwx------  5 root root 4.0K Feb 23 16:19 args-0.10.0+2
drwx------  5 root root 4.0K Feb 23 16:19 barback-0.14.2
drwx------  5 root root 4.0K Feb 23 16:19 bootjack-0.6.5+2
drwx------  3 root root 4.0K Feb 23 16:19 browser-0.10.0+2
drwx------  4 root root 4.0K Feb 23 16:19 code_transformers-0.1.6
drwx------  4 root root 4.0K Feb 23 16:19 collection-0.9.4

Root's umask is 0022, which I confirmed by starting a root shell and touching a new file and mkdir'ing a new directory. The files and folders inside these packages have reasonable permissions, but for some reason this top level does not.

I found a thread here from Feb 2013 with a similar topic, but it ended without any resolution. It's now 2 years later, so I'm thinking the resolution might have changed anyway.

Weird. I recall that issue, but I'm not at all an expert on how permissions work on Unix. We're just using the normal dart:io APIs to create directories, symlinks, etc.

This may be because we create a new directory in there by creating a temp directory and then rename it after the download completes. (That way we don't leave a broken directory if something fails.) Perhaps that affects the permissions?
I think it is pretty standard to make temporary directories readable only by the creating user (for security reasons), this dart code shows that is happening:
import 'dart:io';

void main() {

  Directory dir = new Directory('/tmp/testing');
  print('Created: ${dir.createTempSync("foobar")}');
}

ricow@nobbo:/tmp$ rm -rf testing/*
ricow@nobbo:/tmp$ sudo /usr/local/google/home/ricow/src/dart/dart/out/ReleaseIA32/dart-sdk/bin/dart main.dart
Created: Directory: '/tmp/testing/foobarZliQcO'
ricow@nobbo:/tmp$ ls -l testing/
total 4
drwx------ 2 root root 4096 Feb 24 08:52 foobarZliQcO

 
Python agrees:
ricow@nobbo:/tmp$ sudo python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> tempfile.mkdtemp(dir='/tmp/python')
'/tmp/python/tmpLokCVP'
>>> 
ricow@nobbo:/tmp$ lsl python/
total 4
drwx------ 2 root root 4096 Feb 24 08:53 tmpLokCVP

Cheers,
Rico

Is there any way to disable pub caching completely? I don't see a significant benefit. I'm not downloading gigs of Dart packages every day, and the time saved by caching has been outweighed 10000-to-1 by time spent fiddling with Pub.

No, there isn't. The packages you use have to be downloaded somewhere. Since they are the same for each application that uses them, it makes sense to have a shared cache.

We could definitely support installing them locally in your application's package too, but never had a chance to do that. If you think it would be helpful, can you file a bug?

Thanks!

- bob

--
For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 
To file a bug report or feature request, go to http://www.dartbug.com/new

To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

Søren Gjesse

unread,
Feb 24, 2015, 3:08:19 AM2/24/15
to Rico Wind, General Dart Discussion
dart:io uses mkdtemp to create temporary directories on Linux. It is documented to set the permission to 700.

Regards,
Søren

---
Søren Gjesse
Software Engineer, Google Denmark
CVR nr. 28 86 69 84

George Moschovitis

unread,
Feb 24, 2015, 5:03:18 AM2/24/15
to mi...@dartlang.org

Pub build is basically for client side codo. I saw there was some work done on dart2dart but I guess it's still not officially supported.

We are waiting for this for a long time. Any ETA on pub build with support of dart2dart for server-side deployments?

-g. 

Günter Zöchbauer

unread,
Feb 24, 2015, 5:47:37 AM2/24/15
to mi...@dartlang.org
I guess a snapshot could work well but it has to be the same Dart build where the snapshot was created where it's executed (never tried though).

Bob Nystrom

unread,
Feb 24, 2015, 1:20:14 PM2/24/15
to General Dart Discussion

On Tue, Feb 24, 2015 at 2:03 AM, George Moschovitis <george.mo...@gmail.com> wrote:
We are waiting for this for a long time. Any ETA on pub build with support of dart2dart for server-side deployments?

No, sorry, no ETA. I think our expectation is that most users will lean on snapshots for this.

Cheers,

- bob

George Moschovitis

unread,
Feb 24, 2015, 1:50:17 PM2/24/15
to mi...@dartlang.org
Can you please explain how one can use the snapshot feature for deployment?  
AFAIK, it's not documented.

-g.

Günter Zöchbauer

unread,
Feb 24, 2015, 1:54:33 PM2/24/15
to mi...@dartlang.org

Natalie Weizenbaum

unread,
Feb 24, 2015, 2:00:12 PM2/24/15
to General Dart Discussion, Rico Wind
The permissions issue is particularly tricky because Dart doesn't provide a way of setting them manually (issue 15078).

Greg Lowe

unread,
Feb 24, 2015, 3:04:12 PM2/24/15
to mi...@dartlang.org
Not sure. But I believe that for production you don't need the whole SDK. All of the core libs are compiled into the dart executable file. So in production you could just deploy the correct dart executable along with the snapshot file.

Mark Haase

unread,
Feb 25, 2015, 1:40:50 PM2/25/15
to mi...@dartlang.org, ri...@google.com
Søren and Rico, thanks to both of you for the detailed explanations.

I now understand why 700 permissions are the default, but that seems impractical for many real world situations involving non-temp directories. Once Pub renames the directory, it should probably set sensible permissions using the current umask, no? (On platforms where umask exists.) I made a 'fixpub' alias just for this purpose, but I'm contemplating writing a wrapper for pub that fixes permissions every time I run pub, because just yesterday I repeated that mistake several times. 

It feels like Pub is booby trapped: you can't pub get as one user and then pub serve (or any other method of serving) as a different user.

Do you agree?

Mark Haase

unread,
Feb 25, 2015, 2:54:09 PM2/25/15
to mi...@dartlang.org
Most users run pub build and deploy the output from that. Your build output physically contains all of the dependencies, so there are no more symlinks or usage of the cache.

Bob, thanks for the reply. I didn't realize that the build directory is self-contained, but that makes a lot more sense now. In the future, I will do as you suggest and just deploy the build directory.

No, there isn't. The packages you use have to be downloaded somewhere. Since they are the same for each application that uses them, it makes sense to have a shared cache.
We could definitely support installing them locally in your application's package too, but never had a chance to do that. If you think it would be helpful, can you file a bug?
 
It's definitely an unusual design. Neither Pip nor NPM (which have similar purposes, no?) uses symlinks to a per-user cache. It would make more sense to me to have a search path and a standard location for installing system packages, but it's not a big deal to work with it now that I know what it's doing.

Still, the funky permissions are a problem. As I mentioned in another e-mail, I have a 'fixpub' alias that I run any time I 'pub get' or 'pub upgrade' (if I can remember), however I'm thinking about just writing a wrapper for pub that fixes permissions automatically, since I'm constantly forgetting to run fixpub at the right times and it continues to give me grief.

I'm trying to figure out the source of the funky permissions. As an experiment, I cleared my Pub cache (rm -fr /opt/pub-cache/) and then ran `sudo -E pub get` from my project directory. This brings 32 packages into the cache:

mhaase@ubuntu:~$ ls -1 /opt/pub-cache/hosted/pub.dartlang.org/
analyzer-0.22.4
angular-1.1.0
args-0.12.2+6
barback-0.15.2+4
bootjack-0.6.5+2
browser-0.10.0+2
cli_util-0.0.1+1
code_transformers-0.2.5
collection-0.9.4
csslib-0.11.0+4
di-3.3.3
dnd-0.1.4
dquery-0.7.1
html5lib-0.12.0
initialize-0.2.0
intl-0.11.12
logging-0.9.3
observe-0.12.2+1
path-1.3.3
perf_api-0.0.9
petitparser-1.3.6
pool-1.0.1
route_hierarchical-0.6.1+1
smoke-0.3.1
source_maps-0.10.0+1
source_span-1.0.3
stack_trace-1.2.1
utf-0.9.0+2
watcher-0.9.4
web_components-0.10.1
when-0.2.0
which-0.1.3
mhaase@ubuntu:~$ ls -1 /opt/pub-cache/hosted/pub.dartlang.org/ | wc -l
32

Now I expect all these files to be owned by root, so I check that:

mhaase@ubuntu:~$ sudo find /opt/pub-cache/ ! -user root | cut -d'/' -f 6 | uniq | sort
analyzer-0.22.4
angular-1.1.0
args-0.12.2+6
barback-0.15.2+4
browser-0.10.0+2
cli_util-0.0.1+1
code_transformers-0.2.5
collection-0.9.4
csslib-0.11.0+4
di-3.3.3
html5lib-0.12.0
initialize-0.2.0
intl-0.11.12
logging-0.9.3
observe-0.12.2+1
path-1.3.3
perf_api-0.0.9
petitparser-1.3.6
pool-1.0.1
route_hierarchical-0.6.1+1
smoke-0.3.1
source_maps-0.10.0+1
source_span-1.0.3
stack_trace-1.2.1
utf-0.9.0+2
watcher-0.9.4
web_components-0.10.1
mhaase@ubuntu:~$ sudo find /opt/pub-cache/ ! -user root | cut -d'/' -f 6 | uniq | sort | wc -l
27

I'm using 'cut | uniq | sort' to count the number of packages that contain files owned by somebody other than root. The number is 27, which means that there are 5 packages that correctly unpack all of the files with root as the owner.

This leads me to think that the issue has to do with the build process. If people are building tarballs with files owned by UIDs that don't exist on my system and tar heeds those UIDs when it extracts, then I would end up with this:

mhaase@ubuntu:~$ sudo ls -l /opt/pub-cache/hosted/pub.dartlang.org/analyzer-0.22.4
[sudo] password for mhaase: 
total 32
drwxr-xr-x 2 root   root 4096 Feb 25 11:14 bin
-rw-r--r-- 1 117706 5000  231 Aug 20  2014 CHANGELOG.md
drwxr-xr-x 2 root   root 4096 Feb 25 11:14 example
drwxr-xr-x 5 root   root 4096 Feb 25 11:14 lib
-rw-r--r-- 1 117706 5000 1524 Jun 16  2014 LICENSE
-rw-r--r-- 1 117706 5000  385 Aug 28 07:54 pubspec.yaml
-rw-r--r-- 1 117706 5000 1275 Jun 16  2014 README.md
drwxr-xr-x 6 root   root 4096 Feb 25 11:14 test

I guess you're probably exec'ing the system's `tar`, right? (I can't find a source code repo for Pub on GitHub.) So I looked at the GNU tar man page (http://www.gnu.org/software/tar/manual/html_section/tar_70.html) and see this:

`--same-owner'
Create extracted files with the same ownership they have in the archive.

This is the default behavior for the superuser, so this option is meaningful only for non-root users, when tar is executed on those systems able to give files away. This is considered as a security flaw by many people, at least because it makes quite difficult to correctly account users for the disk space they occupy. Also, the suid or sgid attributes of files are easily and silently lost when files are given away.

When writing an archive, tar writes the user ID and user name separately. If it can't find a user name (because the user ID is not in `/etc/passwd'), then it does not write one. When restoring, it tries to look the name (if one was written) up in `/etc/passwd'. If it fails, then it uses the user ID stored in the archive instead.

`--no-same-owner'
`-o'
Do not attempt to restore ownership when extracting. This is the default behavior for ordinary users, so this option has an effect only for the superuser.

So dollars to donuts, Pub is exec'ing tar without '--no-same-owner', which would work fine when running as a normal user but has undesirable behavior when running as root. I'm guessing few people are running Pub as root because they are doing a build on their dev machine and then deploying the whole build directory to production.

Nonetheless, if my hypothesis is correct, I'd like to submit a bug report to add the '--no-same-owner' flag when running tar. Does that sound reasonable?

Greg Lowe

unread,
Feb 25, 2015, 3:16:58 PM2/25/15
to mi...@dartlang.org
My system administrator foo is weak - however I thought from a security perspective it is not got practice to run software as the root user - i.e. pub in the case.

If you want the files to be owned by root, would it be better practice to run pub as a normal user and then copy/chown the package files?

Jim Trainor

unread,
Feb 25, 2015, 3:25:51 PM2/25/15
to mi...@dartlang.org
I deploy the dart2js output rather than invoking pub build.  Maybe I'm "old school"... if that's even possible with dart ?!?!

Bob Nystrom

unread,
Feb 25, 2015, 3:59:31 PM2/25/15
to General Dart Discussion
On Wed, Feb 25, 2015 at 11:54 AM, Mark Haase <meh...@gmail.com> wrote:
 
It's definitely an unusual design. Neither Pip nor NPM (which have similar purposes, no?) uses symlinks to a per-user cache.

Bundler and RubyGems, which are pub's closest spiritual inspirations do. Pip also installs to a global cache by default as far as I know. I think Cabal and LuaRocks do too. It's pretty typical.

It's only relatively recently that the idea of a per-application set of packages has come to fore. That's where you get things like virtualenv, rvm, etc. Those exist to compensate for the fact that the globally-installed packages in those languages aren't versioned. If you install a new version of some package, it replaces the old version for all applications on your machine. Pub's cache is explicitly versioned, and doesn't have that problem.
 
It would make more sense to me to have a search path and a standard location for installing system packages, but it's not a big deal to work with it now that I know what it's doing.

It's more complex than a search path, since you have to take versioning into account. But, yes, pub basically does have a single cache of "system packages". It's just that each application controls which versions of those it uses.
 

This leads me to think that the issue has to do with the build process. If people are building tarballs with files owned by UIDs that don't exist on my system and tar heeds those UIDs when it extracts, then I would end up with this:

mhaase@ubuntu:~$ sudo ls -l /opt/pub-cache/hosted/pub.dartlang.org/analyzer-0.22.4
[sudo] password for mhaase: 
total 32
drwxr-xr-x 2 root   root 4096 Feb 25 11:14 bin
-rw-r--r-- 1 117706 5000  231 Aug 20  2014 CHANGELOG.md
drwxr-xr-x 2 root   root 4096 Feb 25 11:14 example
drwxr-xr-x 5 root   root 4096 Feb 25 11:14 lib
-rw-r--r-- 1 117706 5000 1524 Jun 16  2014 LICENSE
-rw-r--r-- 1 117706 5000  385 Aug 28 07:54 pubspec.yaml
-rw-r--r-- 1 117706 5000 1275 Jun 16  2014 README.md
drwxr-xr-x 6 root   root 4096 Feb 25 11:14 test

I'm not much of a permissions expert, so a lot of this is beyond me. :-/
 

I guess you're probably exec'ing the system's `tar`, right?

Yup.
 
(I can't find a source code repo for Pub on GitHub.)

 
So dollars to donuts, Pub is exec'ing tar without '--no-same-owner', which would work fine when running as a normal user but has undesirable behavior when running as root. I'm guessing few people are running Pub as root because they are doing a build on their dev machine and then deploying the whole build directory to production.

I think that's right. At the very least, I'm not aware of any reason why you would want to run pub as root. Pub itself certainly doesn't require it. That's why we put the cache in your home directory—that will be accessible to you without sudo. Running pub as root for kicks should be safe (pub doesn't execute any user code on installation), but just seems like asking for trouble.
 
Nonetheless, if my hypothesis is correct, I'd like to submit a bug report to add the '--no-same-owner' flag when running tar. Does that sound reasonable?

I'd have to do some more research to get a better understanding of what this means. Yes, definitely file a bug if you think it's worth us looking into it. But it may be that doing that isn't the best solution. I don't think we want to encourage users to run pub as root, so I don't know if it makes much sense to add features that are specifically aimed at that use case.

Cheers!

- bob

Mark Haase

unread,
Feb 25, 2015, 4:02:17 PM2/25/15
to mi...@dartlang.org
The idea is you don't want important files (e.g. server code or client code) to be writable by the user who is running the server. If the server is exploited, then it might be able to overwrite code. For example, if somebody could overwrite your main.dart, what kind of havoc could they wreak? (Or in my case, what if they could overwrite my Python application server script?)

One way to achieve this is to have all files owned by root and permissions like 644, then run the server as a different user. That user has read access to those files but not write access. Then, if the server is exploited, they at least won't be able to overwrite important files. 

As Bob and Jim pointed out, the build product is self-contained, so I can deploy that to a server (owned by root) and won't need a `pub build` step on the server. That renders my point moot, to some extent... but I still think its erroneous behavior to set privileges to 700 and set the UID to the UID of whoever built and published that package.

Mark Haase

unread,
Feb 25, 2015, 4:17:37 PM2/25/15
to mi...@dartlang.org
Pip also installs to a global cache by default as far as I know.

Pip does install to a system-wide directory, but it's not a "cache". It places packages in a directory that is in Python's search path. This means a sysadmin can install packages that other users can use in their Python scripts. I know this probably doesn't resonate with people running single-user systems, but I don't think its preposterous that Pub might be used on a multi-user system. (I typically configure several users in a production environment for least-privilege reasons.)

This scenario is annoying with Pub because the sysadmin needs to repair permissions each time after running Pub. (E.g. user ID 117706 belongs to rkirov, who I think works at Google? It's weird that I install a package on my machine and its owned by rkirov.) I think this scenario could be fixed pretty easily with a few lines of code, but it would be blocked by #15078.

 I don't think we want to encourage users to run pub as root, so I don't know if it makes much sense to add features that are specifically aimed at that use case.

I can live with this, I just think its weird. I now see that the right way is to build on my dev box and then deploy that to production, but my application is so young that I don't even have an automated build, yet. So I tried doing it by hand and was frustrated that Pub makes it harder than it needs to be.

But if I'm alone on this, I will just learn to use it differently. No doubt I have some bias coming from a background where I'm used to `sudo apt-get` and `sudo pip`, etc...

Anyway, I appreciate your time explaining it. The need to offer multiple versions of a dependency does explain a lot about why the pub cache needs to be designed this way.

Greg Lowe

unread,
Feb 25, 2015, 5:04:52 PM2/25/15
to mi...@dartlang.org
> you don't want important files ...  to be writable by the user who is running the server.

Sure - and you can use chmod/chown etc to achieve this. My point is that running pub as root is not necessary and better avoided. Pub interacts with the network, and in some cases also executes remote code loaded of the network. Running this as root increases your security risk.

But this is all rather moot. Since in practice it's probably better to use pub build. And then as part of your deploy process use a script to chmod/chown the files. For security reasons it may also be worth considering using a separate user other than root. This means that you won't need to run your deploy script as the root user.

Rico Wind

unread,
Feb 26, 2015, 10:03:37 AM2/26/15
to Mark Haase, Bob Nystrom, General Dart Discussion
On Wed, Feb 25, 2015 at 7:40 PM, Mark Haase <meh...@gmail.com> wrote:
Søren and Rico, thanks to both of you for the detailed explanations.

I now understand why 700 permissions are the default, but that seems impractical for many real world situations involving non-temp directories. Once Pub renames the directory, it should probably set sensible permissions using the current umask, no? (On platforms where umask exists.) I made a 'fixpub' alias just for this purpose, but I'm contemplating writing a wrapper for pub that fixes permissions every time I run pub, because just yesterday I repeated that mistake several times. 

It feels like Pub is booby trapped: you can't pub get as one user and then pub serve (or any other method of serving) as a different user.

Do you agree?
Bob: could we simply fix this by not relying on system generated temporary dirs for this? You are already creating these in a directory you control, how about temp/temp_package_name_millisecondsSinceEpoch for the temporary name? Creating that with the normal Directory.create function? It should be a pretty local change if I remember correctly on how pub has the io library wrapped.

Cheers,
Rico

Bob Nystrom

unread,
Feb 26, 2015, 12:54:58 PM2/26/15
to Rico Wind, Mark Haase, General Dart Discussion

On Thu, Feb 26, 2015 at 7:03 AM, Rico Wind <ri...@google.com> wrote:
Bob: could we simply fix this by not relying on system generated temporary dirs for this? You are already creating these in a directory you control, how about temp/temp_package_name_millisecondsSinceEpoch for the temporary name? Creating that with the normal Directory.create function? It should be a pretty local change if I remember correctly on how pub has the io library wrapped.

I considered that, but I felt like I was kind of asking for name collisions if I tried to roll my own "random non-colliding name" function. But maybe it's worth doing? In practice, the amount of concurrency going on is pretty small.

I'd like to hear Natalie's thoughts, but I'm up for doing this. Want to file a bug?

Thanks!

- bob

Rico Wind

unread,
Feb 26, 2015, 1:05:22 PM2/26/15
to Bob Nystrom, Mark Haase, General Dart Discussion
Reply all
Reply to author
Forward
0 new messages