I really like Dart -- a lot -- but the Dart ecosystem is... meh. Pub is especially confounding. I find myself fighting against it and trying to work around it more often than not.How is Pub envisioned for use on production servers? I've run into a few problems while trying to migrate Dart applications from my development server to a production server. (The application uses a Dart client and a Python application server, if that matters.)
The main issue is the pub cache. By default, it is stored in ~/.pub-cache, which quickly gets annoying if you want to install as one user but run as a different user. E.g. I'm used to installing most of my application files as root but running the application as a less privileged user. With default pub settings, my packages will get cached in /root/.pub-cache but the application will look in /home/app-user/.pub-cache.I see there is a PUB_CACHE environment variable, so I tried setting that to a global location and then running `sudo pub get`. This still doesn't work, because (for some unknown reason), the package directories are set with 700 permissions:root@localhost:/opt/pub-cache/hosted/pub.dartlang.org# ltotal 112Kdrwxr-xr-x 28 root root 4.0K Feb 23 16:20 .drwxr-xr-x 3 root root 4.0K Feb 23 16:19 ..drwx------ 6 root root 4.0K Feb 23 16:19 analyzer-0.15.7drwx------ 12 root root 4.0K Feb 23 16:19 angular-1.0.0drwx------ 5 root root 4.0K Feb 23 16:19 args-0.10.0+2drwx------ 5 root root 4.0K Feb 23 16:19 barback-0.14.2drwx------ 5 root root 4.0K Feb 23 16:19 bootjack-0.6.5+2drwx------ 3 root root 4.0K Feb 23 16:19 browser-0.10.0+2drwx------ 4 root root 4.0K Feb 23 16:19 code_transformers-0.1.6drwx------ 4 root root 4.0K Feb 23 16:19 collection-0.9.4Root's umask is 0022, which I confirmed by starting a root shell and touching a new file and mkdir'ing a new directory. The files and folders inside these packages have reasonable permissions, but for some reason this top level does not.I found a thread here from Feb 2013 with a similar topic, but it ended without any resolution. It's now 2 years later, so I'm thinking the resolution might have changed anyway.
Is there any way to disable pub caching completely? I don't see a significant benefit. I'm not downloading gigs of Dart packages every day, and the time saved by caching has been outweighed 10000-to-1 by time spent fiddling with Pub.
On Mon, Feb 23, 2015 at 8:39 AM, Mark Haase <meh...@gmail.com> wrote:I really like Dart -- a lot -- but the Dart ecosystem is... meh. Pub is especially confounding. I find myself fighting against it and trying to work around it more often than not.How is Pub envisioned for use on production servers? I've run into a few problems while trying to migrate Dart applications from my development server to a production server. (The application uses a Dart client and a Python application server, if that matters.)Most users run pub build and deploy the output from that. Your build output physically contains all of the dependencies, so there are no more symlinks or usage of the cache.The main issue is the pub cache. By default, it is stored in ~/.pub-cache, which quickly gets annoying if you want to install as one user but run as a different user. E.g. I'm used to installing most of my application files as root but running the application as a less privileged user. With default pub settings, my packages will get cached in /root/.pub-cache but the application will look in /home/app-user/.pub-cache.I see there is a PUB_CACHE environment variable, so I tried setting that to a global location and then running `sudo pub get`. This still doesn't work, because (for some unknown reason), the package directories are set with 700 permissions:root@localhost:/opt/pub-cache/hosted/pub.dartlang.org# ltotal 112Kdrwxr-xr-x 28 root root 4.0K Feb 23 16:20 .drwxr-xr-x 3 root root 4.0K Feb 23 16:19 ..drwx------ 6 root root 4.0K Feb 23 16:19 analyzer-0.15.7drwx------ 12 root root 4.0K Feb 23 16:19 angular-1.0.0drwx------ 5 root root 4.0K Feb 23 16:19 args-0.10.0+2drwx------ 5 root root 4.0K Feb 23 16:19 barback-0.14.2drwx------ 5 root root 4.0K Feb 23 16:19 bootjack-0.6.5+2drwx------ 3 root root 4.0K Feb 23 16:19 browser-0.10.0+2drwx------ 4 root root 4.0K Feb 23 16:19 code_transformers-0.1.6drwx------ 4 root root 4.0K Feb 23 16:19 collection-0.9.4Root's umask is 0022, which I confirmed by starting a root shell and touching a new file and mkdir'ing a new directory. The files and folders inside these packages have reasonable permissions, but for some reason this top level does not.I found a thread here from Feb 2013 with a similar topic, but it ended without any resolution. It's now 2 years later, so I'm thinking the resolution might have changed anyway.Weird. I recall that issue, but I'm not at all an expert on how permissions work on Unix. We're just using the normal dart:io APIs to create directories, symlinks, etc.This may be because we create a new directory in there by creating a temp directory and then rename it after the download completes. (That way we don't leave a broken directory if something fails.) Perhaps that affects the permissions?
Is there any way to disable pub caching completely? I don't see a significant benefit. I'm not downloading gigs of Dart packages every day, and the time saved by caching has been outweighed 10000-to-1 by time spent fiddling with Pub.
No, there isn't. The packages you use have to be downloaded somewhere. Since they are the same for each application that uses them, it makes sense to have a shared cache.We could definitely support installing them locally in your application's package too, but never had a chance to do that. If you think it would be helpful, can you file a bug?Thanks!- bob
--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.
Pub build is basically for client side codo. I saw there was some work done on dart2dart but I guess it's still not officially supported.
We are waiting for this for a long time. Any ETA on pub build with support of dart2dart for server-side deployments?
Most users run pub build and deploy the output from that. Your build output physically contains all of the dependencies, so there are no more symlinks or usage of the cache.
No, there isn't. The packages you use have to be downloaded somewhere. Since they are the same for each application that uses them, it makes sense to have a shared cache.
We could definitely support installing them locally in your application's package too, but never had a chance to do that. If you think it would be helpful, can you file a bug?
It's definitely an unusual design. Neither Pip nor NPM (which have similar purposes, no?) uses symlinks to a per-user cache.
It would make more sense to me to have a search path and a standard location for installing system packages, but it's not a big deal to work with it now that I know what it's doing.
This leads me to think that the issue has to do with the build process. If people are building tarballs with files owned by UIDs that don't exist on my system and tar heeds those UIDs when it extracts, then I would end up with this:mhaase@ubuntu:~$ sudo ls -l /opt/pub-cache/hosted/pub.dartlang.org/analyzer-0.22.4[sudo] password for mhaase:total 32drwxr-xr-x 2 root root 4096 Feb 25 11:14 bin-rw-r--r-- 1 117706 5000 231 Aug 20 2014 CHANGELOG.mddrwxr-xr-x 2 root root 4096 Feb 25 11:14 exampledrwxr-xr-x 5 root root 4096 Feb 25 11:14 lib-rw-r--r-- 1 117706 5000 1524 Jun 16 2014 LICENSE-rw-r--r-- 1 117706 5000 385 Aug 28 07:54 pubspec.yaml-rw-r--r-- 1 117706 5000 1275 Jun 16 2014 README.mddrwxr-xr-x 6 root root 4096 Feb 25 11:14 test
I guess you're probably exec'ing the system's `tar`, right?
(I can't find a source code repo for Pub on GitHub.)
So dollars to donuts, Pub is exec'ing tar without '--no-same-owner', which would work fine when running as a normal user but has undesirable behavior when running as root. I'm guessing few people are running Pub as root because they are doing a build on their dev machine and then deploying the whole build directory to production.
Nonetheless, if my hypothesis is correct, I'd like to submit a bug report to add the '--no-same-owner' flag when running tar. Does that sound reasonable?
Pip also installs to a global cache by default as far as I know.
I don't think we want to encourage users to run pub as root, so I don't know if it makes much sense to add features that are specifically aimed at that use case.
Søren and Rico, thanks to both of you for the detailed explanations.I now understand why 700 permissions are the default, but that seems impractical for many real world situations involving non-temp directories. Once Pub renames the directory, it should probably set sensible permissions using the current umask, no? (On platforms where umask exists.) I made a 'fixpub' alias just for this purpose, but I'm contemplating writing a wrapper for pub that fixes permissions every time I run pub, because just yesterday I repeated that mistake several times.It feels like Pub is booby trapped: you can't pub get as one user and then pub serve (or any other method of serving) as a different user.Do you agree?
Bob: could we simply fix this by not relying on system generated temporary dirs for this? You are already creating these in a directory you control, how about temp/temp_package_name_millisecondsSinceEpoch for the temporary name? Creating that with the normal Directory.create function? It should be a pretty local change if I remember correctly on how pub has the io library wrapped.