Maybe change the way to install packages?

[mezoni]

Maybe change the way to install packages?

My suggestion is as follows.

Packages install to a global repository of packages.

Each version separately.

The "#import" directive can specify the version.

The package manager does not create folders within the application (package).

The package manager does not create any symbolic links in an application (package).

Package Manager installs the packages in the global repository.

Search and include (import) the package provides a virtual machine by convention.

Convention is as follows.

Gloabal repo located in home directory in directory .dartpackages.

Environment variable "DART_PACKAGES" may override this location.

Structure of repo is simple as possible.

Like this.

.dartpackages

  packages/

    package_name1/

      1.2.3/

      1.5.6/

      2.0.0/

      index.lock

    package_name2/

      2.3.4/

      2.5.6/

      3.0.0/

      index.lock

File index.lock ia a version index file.

It contains info about installed versions of package.

It structure is simple.

==index.lock==

1.2.3

1.5.6

2.0.0

==eof==

This index file created by package "pub" manager.

Package manager need support additional operations.

1. Rebuild indexes in global repo.

2. Pack application (package).

3. Unpack application (package).

1. Rebuild operation updates the indexes in repo.

2. Pack application create app (package) archive that may include or not necessary packages in archive from repo.

The directory name of included in archive packages may be .dartpackages.

Or this maybe a lightweight version (by default) without dependencies included in archive.

3. Unpack application.

Unpack application unpack app (package) in current directory and may also install included in archive packages (dependencies) in global repo;

And finally, how it can be implemented in a virtual machine.

Sample written in Dart language but this, of course, should be on C++.

== test_this.dart ===============

#import('dart:math');

#import('dart:io');

void main() {

  // Import directive

  //#import('package:cool_package/cool_package.dart', version: '>=1.2.3 <2.0.0');

  var path = 'cool_package/cool_package.dart';

  var version = '>=1.2.3 <2.0.0';

  var pkgspec = new PackageSpec(path, version);

  var pkgdir = PackageLoader.findPackageDir(pkgspec);

  if(!pkgdir.isEmpty()) {

    print('Package $pkgspec found at global cache $pkgdir');

  }

}

class PackageLoader {

  static String findPackageDir(PackageSpec pkgspec) {

    if(pkgspec == null || pkgspec.name == null || pkgspec.name.isEmpty()) {

      return '';

    }

    var globalPath = getGlobalPackagesPath();

    if(globalPath == null || globalPath.isEmpty()) {

      // No global packages path found.

      return '';

    }

    var sep = Platform.pathSeparator;

    var pkgRootPath = '${globalPath}${sep}packages${sep}${pkgspec.name}';

    var pkgRootDir = new Directory(pkgRootPath);

    if(!pkgRootDir.existsSync()) {

      // No package root directory found.

      return '';

    }

    var indexFile = new File('${pkgRootPath}${sep}index.lock');

    if(!indexFile.existsSync()) {

      // No index file found.

      return '';

    }

    var text = indexFile.readAsTextSync();

    text = text.replaceAll('\r\n', '\n');

    var versions = text.split('\n');

    if(versions.length == 0) {

      // Possibly need to rebuild index?

      return '';

    }

    var version = findSuitableVersion(versions, pkgspec.version);

    if(version.isEmpty()) {

      // No suitable version found.

      return '';

    }

    var versionPath = '${pkgRootPath}${sep}${version}';

    var versionDir = new Directory(versionPath);

    if(versionDir.existsSync()) {

      return versionDir.path;

    }

    return '';

  }

  static String getGlobalPackagesPath() {

    var path = '';

    path = Platform.environment['DART_PACKAGES'];

    // Skip here

    if(false) {

      return path;

    }

    var sep = Platform.pathSeparator;

    path = getUserHomeDir();

    // Skip here

    if(false) {

      return '${path}${sep}.dartpackages';

    }

    // Only for emulate how this works

    path = new Directory.current().path;

    path = new Directory('${path}${sep}global_dart_packages').path;

    return path;

  }

  static String getUserHomeDir() {

    // Not used here

    return null;

  }

  static String findSuitableVersion(List versions, String criteria) {

    if(versions == null || versions.length == 0) {

      return '';

    }

    // Only as example. Here must be an algorithm of version resolving.

    if(criteria == '>=1.2.3 <2.0.0') {

      for(var version in versions) {

        if(version == '1.5.6') {

          return version;

        }

      }

    }

  }

}

class PackageSpec {

  String name;

  String path;

  String version;

  PackageSpec(String this.path, [String this.version]) {

    if(path == null || path.isEmpty()) {

      return;

    }

    var index = indexOfSeparator(path);

    if(index == -1) {

      name = path;

    } else {

      name = path.substring(0, index);

    }

  }

  static indexOfSeparator(String path) {

    if(path == null || path.isEmpty()) {

      return -1;

    }

    var unix = path.indexOf('/');

    var windows = path.indexOf('\\');

    if(unix == -1 && windows == -1) {

      return -1;

    }

    if(unix == -1 || windows == -1) {

      return max(unix, windows);

    }

    return min(unix, windows);

  }

  String toString() => '[$name "$version"]';

}

== eof test_this.dart ==

Project directory structure is.

=============

test_this

global_dart_packages/ 

  packages

    cool_package

      1.2.3

        cool_package.dart

      1.5.6

        cool_package.dart

      2.0.0

        cool_package.dart

      index.lock

test_this.dart

=============

Directory called "global_dart_packages" is for emulation only of gloabal repo directory.

I think that it will not be difficult to implement in a virtual machine.

Let the virtual machine will load the correct version.

Why bother with symbolic links, and with everything else.

Maybe I'm wrong.

At least it will not depend on the particular file system.

The only thing that will need to be taken into account in the implementation and use it so that "pubspec.yaml" must meet the import directives in source files.

In order to install the required version of the packages in the global repo.

[Ladislav Thon]

Maybe change the way to install packages?

I already tried the exact thing you propose, see https://github.com/Ladicek/dpm. It works just fine. I'll let Pub people comment on why they didn't choose this way -- it's likely that the decisive factor was usage from the browser.

LT

[mezoni]

What exactly prevents the use of this technique in the browser?

Please detail.

I do not think that the browser problem.

Even if it is not Chromium.

суббота, 8 сентября 2012 г., 18:55:08 UTC+6 пользователь Ladislav Thon написал:

[mezoni]

Packages directory is like a "virtual" directory.

Alias for the convenience of addressing.

The virtual machine independently seeking a package in the directory.

The browser will include this VM.

VM first searches the local repository and then globally.

Local repositories are searched from top to bottom in a directory hierarchy.

The first found repository will stop search.

For example.

may_app/

  .dartpackages/

    pkg1/

    pkg2/

      .dartpackages/

      pkg4/        

    pkg3/

  test/

   .dartpackages/

 

In this way we not realy need these repositories

may_app/pkg2/.dartpackages

may_app/test/.dartpackages

This is enough "may_app/.dartpackages" for all packages.

If we run my_app/test/test.dart than local repo wil be found at "may_app/.dartpackages".

Package manager may support operation to move all higher packages in one directory.

Ie from "may_app/pkg2/.dartpackages" and "may_app/test/.dartpackages" to "may_app/.dartpackages".

Like "pub optimize".

You also may tell package manager to move all dependent packages into local repository from global repo.

Like "pub install --deployment"

I believe that many problems can be solved.

[mezoni]

P.S.

When I say..

..The first found repository will stop search. 

I wanted to say..

The first found LOCAL repository (NOT PACKAGE) will stop search NEXT LOCAL repository. 

If package not found in local repo the search continues in global repo.

[mezoni]

Simple modification in VM, Chromium and Editor source code.

==pseudocode==

#import('package:pkg1/pkg1.dart', version: "1.0.0");

get_package_dir_for_script(script_file, pkg_spec_from_import_directive) {

  local_repo_dir = get_local_repo_dir_for_script(script_file);

  package_dir = null;

  if(local_repo_dir != null) {

    package_dir = get_package_dir_in_repo(local_repo_dir, pkg_spec_from_import_directive);

  }

  if(package_dir != null) {

    return package_dir;

  }

  global_repo_dir = get_global_repo_dir();  

  if(global_repo_dir != null) {

    package_dir = get_package_dir_in_repo(global_repo_dir, pkg_spec_from_import_directive);

  }

  return package_dir;

}

get_local_repo_dir_for_script(script_file) {

  cur_dir = get_dir_where_script_located(script_file);

  while(true) {

    if(cur_dir == null) {

      break;

    }

    

    if(!cur_dir.contains(PACKAGE_REPO_DIRNAME)) {

      cur_dir = get_lower_dir(cur_dir );

      continue;

    }

     return cur_dir;

  }

}

==eof==

In Chromium in not "hosted mode" we can simple assume that application is deployed and all packages preinstalled and located in LOCAL_REPO_DIRNAME_OF_APPLICATION.

Otherwise, it is considered that the application is damaged.

[mezoni]

In browser requests this may be solved like this solution.

This import...

#import('package:cool_package/cool_package.dart', version: '>=1.2.3 <2.0.0');

Translated to...

http://example.com/my_app/packages/cool_package/ge_1_2_3__lt_2_0_0/cool_package.dart

Where uri segment "ge_1_2_3__lt_2_0_0" is a symlink on sever to local or global package with appropriate version for criteria '>=1.2.3 <2.0.0'.

This symlink created by package manager.

Browser not need to resolve appropriate version of package. It simply request package by criteria.

All work must be done before request by the package manager.

[mezoni]

There is not right solution from previous reply.

The easiest way, without the need to create symbolic links.

It can do so.

In local repository should be a file for package version resolving.

==.versions==

cool_package >=1.2.3 <2.0.0: 1.5.6 

cool_package: 3.0.0

==eof==

version 1.5.6 suitable for ">=1.2.3 <2.0.0".

And version 3.0.0 is latest current version.

Browser VM download this file then analize it and resolve versions of imported packages.

Then request appropriate url for installed versions.

No other unnecessary movements.

Simple and clear.

And not a single symlink.

[mezoni]

SUMMARY

Packages installed into global repo.

From global repo cache possibly install packages into local cache.

Only one local cache on the apllication (for browser packages requests comptability and other reasons).

Packages cache structures (global and local):

========================

pkg/

  package1/

    1.2.3/

      package1.dart

    1.3.0/

      package1.dart

    2.0.0/

      package1.dart

  package2/

    2.0.0/

      package2.dart

  package3/

    1.0.0/

      package3.dart

========================

Package cache does not contain inside package directories another caches.

All packages and applications used one the same cache (global or local).

REMARK:

++++++++++

If you want to develop or modify package then you need use package as application.

Copy package in your project directory and then develop your project.

Inside new application at first time you need run "pub install";

++++++++++

Global repo consists of

1. Packages cache

2. References info

Repo structure:

========================

global_repo_dir/

  pkg/

  referenses

========================

Where file "references" is.

========================

package1 1.2.0: C:\Users\jack\dart\cool_app1\pkg

package1 1.2.0: C:\Users\jack\dart\cool_app2\pkg

package2 2.0.0: D:\Project\dart\cool_app3\pkg

========================

It is not necessary. Only if you want to support the safe removal of packages.

Package manager may check references before "remove" package and warn if any usage ref exists.

Like this "pub remove --name package2 --version 2.0.0 --force".

Local cache is the directory where you copied packages from a global cache.

With the same structure.

========================

pkg/

========================

You can copy packages into local cache like this.

"pub install --locally".

The local cache is required for correct operation of the browser with packages not in "hosted mode".

Browser request the package with required version through "lookup table".

VM, Editor and Third Party Tools also use this approach.

This simplifies the implementation of this approach and to maintain code.

"Lookup table" created only by pub manager or corrected manualy if needs.

To swim quickly through the water needs to prepare everything on the waterside.

"Lookup table" is file with resolved package versions translation.

==VERSIONS==============

cool_package >=1.2.3 <2.0.0: 1.5.6 

cool_package: 3.0.0

another_package: 1.0.0

========================

This file (by convention) can be in application root directory.

To realize this needs to be extended import directive syntax.

#import('packages/cool_package/cool_package.dart', version: '>=1.2.3 <2.0.0');

It would also be useful to extend library syntax. 

#library('cool_package', version: '1.2.3');

But it is not necessary.

How VM, Editor and Third Party Tools search and import packages with appropriate version?

Local cache are searched from top to bottom in a directory hierarchy.

The first found cache directory will stop search next cache directory. 

If package not found in local cache the search continues in global cache.

Example.

==pkg/cool_package/2.0.0/lib/somefile.dart

#import('packages/unittest/unittest.dart', version: '<=1.0.0');

==

Search steps.

pkg/cool_package/2.0.0/lib/pkg/

pkg/cool_package/2.0.0/pkg

pkg/cool_package/pkg

pkg/cool_package

pkg/

Local cache found here.

Search file VERSION and if found (Not found? run pub install).

Lookup appropriable version through LOOKUP TABLE.

========================

cool_package >=1.2.3 <2.0.0: 1.5.6 

cool_package: 3.0.0

unittest <=1.0.0: 0.0.1

unittest: 0.0.1

========================

Check if exists directory "pkg/unittest/0.0.1".

If exists then done.

Otherwise search in global cache "pkg/unittest/0.0.1" directory.

If exists then done.

Otherwise no package installed or something broken.

Recomendtion is to run "pub install" or check pubspec.yaml configuration.

Browser searches easier.

==www.example.com/my_cool_apps/my_first_app/folder2/webgui .html==

<script type="application/dart" src="webgui.dart"></script>

==

Search steps.

www.example.com/my_cool_apps/my_first_app/folder2/VERSIONS/

www.example.com/my_cool_apps/my_first_app/VERSIONS/

Lookup table should be found here by convention.

If not found there is no packages lookup table used in application (I don't know why..).

Next search steps.

www.example.com/my_cool_apps/my_first_app/folder2/pkg/

www.example.com/my_cool_apps/my_first_app/pkg/

Packages should be here by convention.

If not found there is no packages used in application (May be broken or not deployed app?).

Then request package.

www.example.com/my_cool_apps/my_first_app/pkg/unittest/0.0.1/unittest.dart

====================================

To Dart Team.

If you do not like it then let's ask the users.

Need it or not.

As an experiment, it can be implemented as an alternative in parallel.

For example by new import directive.

#import('repo:packagename/path_to_file.dart', [version: 'version']);

In the VM to implement the algorithm search is not difficult. It is very simple.

In the Editor, I think it will do the same is not difficult.

In the browser, it seems to me, too, it is not difficult to add a few search request.

The compiler is also simple.

Is Dart Open Source or Not?

[mezoni]

Some mistake from me.

To be consistent with the above file "versions transaltion table lookup" should be located in the local cache.

If he is in the root directory of the appliction (package) then it will break the search algorithm.

It must be created and updated by package manager.

[mezoni]

Because files can be in apllication folders and in the local and global cache folders.

Here is local cache search algorithm for concrete file.

findLocalCacheDir(String filepath) {

  if(isPathInGlobalCache(filepath)) {

    return null; // No local cache for file that in global cache

  }

  var curDir = filepath.Dir;

  while(true) {    

    if(curDir == null) {

      return null;

    } 

    if(!curDir.containsDir('pkg')) {

      curDir = curDir.parentDir();

      continue;

    }

    retrun curDir;

  }

}

[mezoni]

It turns out that I was talking to myself.

I guess I'm crazy.

[Bob Nystrom]

On Sun, Sep 9, 2012 at 5:10 AM, mezoni <andrew...@gmail.com> wrote:

It turns out that I was talking to myself.

I guess I'm crazy.

Hi Andrew,

Please keep a couple of things in mind:

1. We don't usually work weekends, so if you send something out on Saturday, you likely won't get a response until the next Monday. Also, we are often not on the same time zone as you, so even though it may still be business hours for you, we may not be here.

2. A giant email, even if it's perfectly brilliant and full of wonderful ideas still takes a lot of time to read, process, understand, and reply to. If you want your thoughts to get full consideration, then it will take time for us (and the many other people on this mailing list) to consider them. Working on open source requires patience: all communication is asynchronous.

3. A giant email followed in quick succession by a flurry of revisions makes it seem like you're still thinking things through. That's fine, but it sends a signal that maybe the ideas in it are still being worked on. In that case, many people won't read it and will wait until it seems more complete. If you have a large proposal, it's probably better to create a separate doc (Google Docs, a github repo, whatever you prefer) and work on it there. Then, once you feel it's solid, send it to the list.

4. A wall of text and code with little formatting is hard to read and understand. Suggestions and ideas (even big ones!) are always welcome, but the bigger the change, the more work will be required on your side to clarify it, communicate it, and make it comprehensible.

Now, onto to your proposal. There is a lot of code there and I don't fully understand what you're getting at, but let me try to respond. At the very least, I can give some context which is not apparent but explains some of why Pub is designed the way it is.

Packages install to a global repository of packages.
Each version separately.

Yes, pub does that now. But, it's important to realize that this only happens on developer machines (and possibly production servers). Dart application end users are not pub users. They do not have package repositories.

> The "#import" directive can specify the version.

We considered this, but we are intentionally going out of our way to not put version numbers in import directives. This is one of the main reasons pubspec.yaml exists: it gives you a place to put version constraints outside of Dart code.

There are two main reasons for this:

1. Putting a concrete version number in there leads to version lock. Let's say I have package foo and it does:

  #import('package:bar/bar.dart', version: '1.2.3');

Then in my app, I import foo. But I'm also using bar, but a different version:

  #import('package:bar/bar.dart', version: '1.5.2');

That's bad. We don't want to bake specific details about how a dependency is loaded (in this case "specific details" means "concrete version") because it doesn't play nice with transitive dependencies. We want the app developer to have control over how all of its dependencies are located, even transitive ones.

So, the idea is that dependencies are "abstract" in your Dart code. You say what you want ("some package named bar") but not how to get it (some physical URL or concrete version number).

2. We don't want to bake package management details into the language specification. We want to keep pub decoupled from the Dart language itself. The language moves much more slowly than pub does, so it would be painful if every change to it (like what you're proposing here) required changing the language specification itself.

By having a layer of abstraction here, we can iterate on the package manager (or replace it entirely!) without interfering with the language.

This means that we can't bake details of how it works into the Dart import syntax. Things like the version constraint syntax are specific to pub and not something we want a Dart implementation to have to care about.

Let the virtual machine will load the correct version.

Yes, but where does it load the correct version from?

Even when you do have the Dart VM in a browser, it needs to load dependent packages from the application's server, not from the local file system. If some user goes to myawesomeapp.com and the Dart app there depends on some_package, that package isn't going to be on the user's local file system. It's going to be on myawesome.com. How does your propsal handle that?

How does it work with dart2js where there is no Dart VM at all?

Cheers,

- bob

[Dirk Detering]

Am 10.09.2012 22:35 schrieb "Bob Nystrom" <rnys...@google.com>:
> Please keep a couple of things in mind:
>

Wow, the now following four points could easily be transformed into some dev-list netiquette! (or did they come from such?).

> 1. We don't usually work weekends, [...]

[Bob Nystrom]

On Mon, Sep 10, 2012 at 8:31 PM, Dirk Detering <mail...@googlemail.com> wrote:

Am 10.09.2012 22:35 schrieb "Bob Nystrom" <rnys...@google.com>:

> Please keep a couple of things in mind:
>

Wow, the now following four points could easily be transformed into some dev-list netiquette! (or did they come from such?).

Nope, pulled them off the top of my head. I think it says really fantastic things about our community that we haven't needed to have any such list. Everyone here is awesome.

(And, just to be clear, Andrew is too. He's put a lot of thought and effort into his proposal, so it's natural to be frustrated when it seems that's met with silence.)

- bob

[mezoni]

Server and Browser VM do not know anything about "pubspec.yaml" files.

That is correct.

But what if I need to configure my library(es)?

How can I tell VM about that?

I propose to have a configuration file.

One file per library.

Like this example.

==mylibrary.config.yaml==

# This file is not a replacement of "pubspec.yaml".

# But "pub manager" may use this file for it own purpose.

# This file is library configuration file.

# You can not create this file. It will work without it.

# But if you need special settings then the file will be useful to you.

name: mylibrary

# 

# Name must be the same as library file name.

# 

version:

# 

# If not specified it is assumed that the version 0.0.0

# 

url: "examples"

#   

# This is an aplliction relative url.  

#   

# If this is your own inside application/package sub-library.

# By example. examples/example_how_it_works.dart

# We tell about that VM.

# If we run this library VM will know where main library (root) located.

#

# If we run this library directly than VM correctly

# search local cache by url at "../packages".

# 

# In another case, the VM searches the local cache at default location;

import:

  remote_package1:

    url: http://www.awesome.com/packages

    #

    # url: http://www.awesome.com/packages/remote_package1/current

    #

  

  remote_package2:  

    url: http://www.awesome.com/packages

    url_type: repository

    #

    # url: http://www.awesome.com/packages/remote_package2/current

    #

  remote_package3:  

    url: http://www.awesome.com/packages

    version: 5.0.0

    #

    # Repository is default value for url_type

    # And url will be combined from "url" and "package_name" and "version".

    # url: http://www.awesome.com/packages/remote_package1/5.0.0

    #

  remote_package4:

    url: http://www.awesome.com/packages/remote_package4/6.0.0

    url_type: direct

    #

    # url: http://www.awesome.com/packages/remote_package4/6.0.0

    #

  physical_location1:

    url: file:///home/jack/dart/development/physical_location1

    url_type: direct

    #

    # url: file:///home/jack/dart/development/physical_location1

    # Not use url "file://" in production mode.

    # "Pub manager" may detect this and warn if run "pub deploy".

    #

  installed_package1:

    version: 1.0.0

    # 

    # We want concrete version and we don't specify url.

    # Default location.

    # 

    # Sever VM - local and global cache.

    #

    # Browser VM - baseUrl/packages/installed_package1/1.0.0

    # 

    # dart2js compile this into one file.

  installed_package2:

    #

    # This is not necessarily. There is no useful information.

    #

==mylibrary.config.yaml==

[mezoni]

http://ajax.googleapis.com/ajax/libs/jquery/1.4.4/jquery.min.js

http://dart.googleapis.com/dart/packages/unittest/1.0.0/unittest.dart

Compare this.

Only needs for version resolving this file.

http://dart.googleapis.com/dart/packages/unittest/versions.txt

==versions.txt==

1.0.0

1.2.0

1.8.0

==versions.txt==

It is a remote cache.

Local and global cache on a machine have the same structure.

You can open your global cache for others through internet.

Cache is not a "pub.dartlang.org".

Cache is similar to CDN (Content Delivery Network).

[mezoni]

1. If you get access through internet for your global cache.

2. And add file "packages.txt".

http://yourdomain.com/dart/packages/packages.txt

==packages.txt==

unittest

awesome

==packages.txt==

REM: "Pub manager" update this file for you when installs packages.

3. Then add folder "packed" in each package folder.

REM: "Pub manager" by default stored downloaded packages in this folder.

http://yourdomain.com/dart/packages/unittest/packed/1.0.0/unittest.zip

http://yourdomain.com/dart/packages/unittest/packed/1.2.0/unittest.zip

You get lite version of package repository.

We call this "CDN repository".

If all Dart VM's will use CDN-like cache natively it is this will only benefit.

==code.dart==

import(package:unittest/unittest.dart);

==code.dart==

==library.config.yaml==

import:

  unittest:

    # Later we change url and it is still works.

    # url: http://dart.googleapis.com/dart

    # cache structure the same

    # Only <scheme>://<authority><path> changed.

==library.config.yaml==

P.S.

Last in list element is a current version.

[Justin Fagnani]

Andrew,

I'm having a very hard time following this thread. There's so much text I can't tell if it's interesting and if I should read it all or not. Can you summarize?

Cheers,

  Justin

--

Consider asking HOWTO questions at Stack Overflow: http://stackoverflow.com/tags/dart
 
 

[mezoni]

All the replies can not read.

Read the last three reply. After Bob Nystrom reply.

VM configuration file is not a main sense.

It is only a possible product of this proposal.

=======================

SUMMARY:

The basic meaning of the unification method of organization of the package 

cache.

I propose to use CDN-like style cache.

Both for server and web appications.

Like this.

http://ajax.googleapis.com/ajax/libs/jquery/1.4.4/jquery.min.js

http://dart.googleapis.com/dart/packages/unittest/1.0.0/unittest.dart

file///home/jack/dart/global_package_cache/packages/unittest/1.0.0/unittest.

dart

CDN-like style cache with some minor enhancements to turning this cache into package repository.

1. With support listing installed packages in cache.

Possibly like this:

http://dart.googleapis.com/dart/packages/PACKAGES.TXT

==PACKAGES.TXT==

unittest

awesome

==PACKAGES.TXT==

2. With support listing of versions of installed packages in cache.

http://dart.googleapis.com/dart/packages/unittest

==VERSIONS.TXT==

1.0.0

1.1.0

1.2.0

1.8.0

==VERSIONS.TXT==

3. Maintenance will be the responsibility of the package manager.

Any manager not only "pub". If desired, even by hand or other usefull tools.

Not package manager dictates the rules how organize the cache localy, globay or remotely.

One style (structure) cache native used by all VM's.

=======================

Lot of writing code to implementation this?

I do not agree.

среда, 12 сентября 2012 г., 4:58:32 UTC+6 пользователь Justin Fagnani написал:

[mezoni]

At current time, the package manager does two jobs.

Ours and theirs.

1. Ours

Finds and installs the packages.

2. Theirs.

Dictates the rules of integration packages in the application.

VM do not care where the package located.

It is stupid and hungry.

VM relies on the work of the package manager. It depends on him.

The package manager must adapt to the VM.

At the same time, the VM must be progressive and use current technology.

Main advantage of Dart is a web applications.

CDN is not applicable to web applications?

When I'm talking about, I do not mean CDN speed and other advantages.

No. I'm talking about the compatibility of the method to search and access data.

In this case, search and access to the packages.

If I want the latest version of the package.

What should I do now?

If this is the localy that I will update the package.

But what is my app run on other machines?

I cannot update packages on all other machines.

But if VM use CDN-like cache there is no problem.

All other machines will get from CDN-like cache latest current versions of packages.

1. VM request list of packackes from cache.

2. If package found then request list of versions.

3. Get last version from he list.

3. Then reference data by the respective protocol. In this case 'http://'.

On local mashine all also only another protocol 'file://'.

Simple install new version in global cache and all done.

VM independently find the latest version.

[Bob Nystrom]

Sorry, I never got a chance to reply to this last week.

If I understand you, you're proposing:

1. That all Dart implementations (including Dartium) support downloading and caching Dart packages. Sort of like the Global Assembly Cache for .NET.

2. That pub and pub.dartlang.org act as a CDN for packages so that end user web apps will request packages directly from Google instead of from whatever web host the app developer is using.

Assuming I have that right, both of those ideas have been considered, though we haven't put much thought into either one. There are a couple of problems:

1. Caching and reusing packages on a user's local browser can be a major security hole. Let's say someone makes a wildly popular Dart packaged called "underbone". Hundreds of applications are using it. If an attacker can get a compromised copy of it on a CDN, then any application using it can be broken. This isn't an entirely unsolvable problem, but it's a very hard one.

2. While Google has lots of bandwidth and storage, being the world's Dart CDN is still a pretty expensive proposition. We want Dart to succeed, but I don't know if we want to fund the entire world's infrastructure for it.

3. Web developers are very careful about which parts of their production infrastructure they want to outsource. If they rely on some Google package manager to serve up their Dart code, when that site goes down, their app is down and there's nothing they can do about it. Uptime is a big deal, which is why things like AppEngine and EC2 have like SLAs and guarantees and stuff like that. The Dart team doesn't have the staff or expertise to do that.

4. Most production web apps are minified before deployment. That minimizes the number of network roundtrips you do, and shrinks code size. That's usually done at the application level: you minify the entire app and all of its dependencies into a single giant blob of code. Reusing packages at runtime doesn't work in that model. Likewise, I think snapshotting works at the isolate, not the library granularity.

Both of your ideas are interesting, and at some point in the web's future I can see it going that way, but I think it's probably too ambitious for the Dart team to take on right now. Just getting a new language in everyone's browser is hard enough!

Cheers,

- bob

[mezoni]

Almost but not quite so well.

There is no such thing as a cache.

Any cache is a repository for its functionality.

Local repository in the program directory.

Local repository on a disk.

CND repository.

pub.dartlang.org is a vendor of packages. (Soft) warehouse.

You use 'pub' to install package into your local repository.

Git is a vendor of packages.

Each package (even application) MUST HAVE A VERSION.

We assume that the default version is 0.0.0

The repository structure is the same. Functionality varies.

You can not add a package to a remote CDN.

You can not delete a package form a remote CDN.

Remote repository is a no support for version constraint.

But they are all standard repository.

All VMs are able to do at least one operation.

This operation is quickly to resolve (transform) the url of dependent package in the repository.

Special file for the VM only specified "host" of repository and possibly version of dependency.

This special file managed by "pub" or manualy.

==dart.code==

import('package:mypackage1/mypackage1.dart');

import('package:mypackage2/mypackage2.dart');

import('package:mypackage3/mypackage3.dart');

==dart.code==

This file fot VM

==dependencies.config.yaml==

mypackage1.host=http://dart.google.org/cdn/packages // Not good idea request from CDN. But why not?

mypackage1.version=1.2.0

mypackage2.version= >=2.0.0 <=3.0.0 // Only version constraint specified. 

//mypackage2=

// I am already on machime. Look me at 'app repo' and 'user repo'.

// if I am a 'web app' then look me at 'http://awesome.com/my_cool_app1/packages'

// Well, I would venture to the new version?

==dependencies.config.yaml==

VM transform this into follwing urls.

For remote:

http://dart.google.org/cdn/packages/mypackage1/1.2.0

For local request used search paths.

[APPLICTION_ROOT]/pub-cache/packages

[USERPROFIFE]/pub-cache/packages

Limitation of the remote repository is a no support for version constraint.

mypackage1.host=http://dart.google.org/cdn/packages

mypackage1.version=>=2.0.0 <=3.0.0 // Error: version constraint not allowed for remote repository.

[mezoni]

You ask and how to be a web application?

Because it is essentially a remote application.

Will help you 'pub'.

He will correct 'vm.config.file' (dependencies file) for your Web application.

He will adjust the 'vm.config.file' beforehand given the installed packages.

Limitation of web application it needs 'vm.config.file' with no versions constrait.

This is a limitation of Browser VM.

You can not adjust concrete versions by 'pub' for Standalone VM.

Standalone VM works corretly with version constraits because VM can search files in local repo.

[mezoni]

You run 'pub install'.

Pub does not create symlinks.

He install packages in repository.

Pub create correct 'vm.config'.

Pub do everything for you.

configure the VM

=='local.vm.config'==

[vm.options]

memory_size=768M

[vm.options.development]

option1=value

[packages]

mypackage1.version=1.0.0

mypackage2.version=5.0.0

=='vm.config'==

Is VM config file bad solution?

[mezoni]

You can tell VM to be compatible.

=='local.vm.config'==

[vm.options]

emulate_version=1.0.0

=='vm.config'==

[mezoni]

It turns out that by hand or by other tools can not be any notes and hints for VM.

[mezoni]

Error: VM cannot load package 'mypackage1' because VM config file not found at 'location'.

This is bad.

But currently can you use packages in non configured by pub application?

No.

Except you munualy create packages directories in each appropriate folder.

But this is nonsense.

Each package necessarily need 'pubspec.yaml'.

But 'pubspec.yaml' is configuration instructions. 

This is not a configuration how to run application (not a package) in VM even in browser.

But if you want convert package into application then you need VM configuration.

Package may have preconfigured VM options for use as application.

Or this options may be found in  'pubspec.yaml'.

[mezoni]

Pre-deployment operation is simplest.

1. Copy packages into application repository.

2. Configure VM for package versions.

[Florian Loitsch]

disclaimer: I'm not working on Pub.

Afaics this means that the VM needs to deal with versions and package downloading. Something we really want to avoid. Package management should be outside of the VM. If we could we would have even avoided the "package:" prefix.

//mypackage2=

// I am already on machime. Look me at 'app repo' and 'user repo'.

// if I am a 'web app' then look me at 'http://awesome.com/my_cool_app1/packages'

// Well, I would venture to the new version?

==dependencies.config.yaml==

VM transform this into follwing urls.

For remote:

http://dart.google.org/cdn/packages/mypackage1/1.2.0

For local request used search paths.

[APPLICTION_ROOT]/pub-cache/packages

[USERPROFIFE]/pub-cache/packages

Limitation of the remote repository is a no support for version constraint.

mypackage1.host=http://dart.google.org/cdn/packages

mypackage1.version=>=2.0.0 <=3.0.0 // Error: version constraint not allowed for remote repository.

--

Consider asking HOWTO questions at Stack Overflow: http://stackoverflow.com/tags/dart
 
 

--
Give a man a fire and he's warm for the whole day,
but set fire to him and he's warm for the rest of his life. - Terry Pratchett

[mezoni]

>> Afaics this means that the VM needs to deal with versions and package downloading. Something we really want to avoid. Package management should be outside of the VM. If we could we would have even avoided the "package:" prefix 

NO. NO. NO

VM needs to deal with versions  - Not needs.

VM needs package downloading  - Not downloading .

Package management should be outside of the VM - YES. VM don't manage packages.

I don't speak English.

Why can not you understand me?

VM to deal with routes.

VM can deal without routes. 

Routes created by package manager.

Routes are static files.

Format is simplest.

mylib_a.dart

mylib_a.imports

SYNTAX for mylib_a.dart 

==  mylib_a.imports ==

imported_package_name = name_of_symlink_created_by_pub

== EOF

VM "search imported_package_name" for "script_name.dart" in packages/"name_of_symlink_created_by_pub"

==========================

Now pub created only flat folders.

E.g

packages/unittest

packages/args

With routes "pub" may create folders with names which it wants.

"Pub" tell about these names through routes.

Routes is static files.

VM uses routes as map.

==  mylib_a.imports ==

unittest = 1.2.3

== EOF

var subfolder = mylib.imports[imported_package]

print(imported_package);

print(subfolder);

OUTPUT:

---------

unittest

1.2.3

---------

VM search "unittest" for "mylib" in "packages/unittest/[1.2.3]/"

I no route exists nor found than VM search at "packages/unittest/[]/"

More precisely at "packages/unittest/"

Again, it is not clear explain?

See this topic May be put around "dart script" package information?

[mezoni]

Each library gets its version of a dependency package.

Name of symlinks in "package" folders is not flat.

lib_a.dart want unittest 1.0.0. Please 1.0.0

lib_b.dart want unittest 2.0.0. Please 2.0.0

Pub resolve import not per folder lib/bin/test.

Pub resolve import per library (dart script with #import).

Pub know about that through "pubspec.yaml".

This needs extended pubspec.yaml syntax.

== pubspec.yaml==

name: mypackage

libraries:

  lib/lib_a:

    dependencies:

      pkg_1:

  lib/lib_b:

    dependencies:

      pkg_1: 3.0.0

      pkg_2: 1.0.0

  test/test1:

    dependencies:

      unittest: 1.0.0

  test/test2:

    dependencies:

      unittest: 2.0.0

==

==lib/lib_a.imports==

pkg_1 = 1.0.0

==EOF

==lib/lib_b.imports==

pkg_1 = 3.0.0

pkg_2 = 1.0.0

==EOF

==test/test1.imports==

unittest = 1.0.0

==EOF

==test/test1.imports==

unittest = 2.0.0

==EOF

==packages/pkg1/1.0.0/pkg1.packages ==

cool_package = 2.0.0

==EOF

These values only symlink names created by "pub".

"packages/pkg1/1.0.0/pkg1.packages" is from symlink "packages/pkg1/1.0.0/"

cool_package = 2.0.0 is "packages/cool_package/2.0.0"

VM use only static *.import files as lookup tables.

Hard for me to explain the basic things.

[mezoni]

>> If we could we would have even avoided the "package:" prefix  

This is not possible without namesapces

So do not talk nonsense

[Ladislav Thon]

> VM to deal with routes.
> VM can deal without routes. 

The VM only deals with URI resolving. Libraries are imported using an URI -- which can look like http://..., file:// etc. Depends only on the set of protocols that the VM supports. So far so good.

1. If the string doesn't look like an URI but like a path on a file system, it will be resolved against local file system.

2. If the scheme is package:, it is resolved like an URI to the entrypoint script with the entrypoint script name removed and '/packages/${whatTheImportUriContained}' appended. Not sure if I explain it clearly, but the point here is that the VM (and all other Dart implementations, because they all share the same algorithm) should contain _the absolute minimum_ support for packages.

LT

P.S.: I think that the reason people don't understand you (at least this is how I feel) isn't your English, but the fact that you are basically dumping your brain on the list. No structure, no clear goal, no nothing. This used to be a nice artistic technique (about a hundred years ago), but it doesn't work here.

[mezoni]

It's much easier

we have this code

==lib/lib_a.dart

#import('package:unittest/unittest.dart');

#import('package:cool_package/cool_package.dart');

==EOF

Now VM solve this by that algorithm.

Script is "lib_a.dart".

Script_name is "lib_a".

$Path is "lib/".

$Path_to_packages is "$Path/packages".

$Symlink_name is $Package_name 

Path to package is "$Path_to_packages/ $Symlink_name ".

Correct?

Correct.

I propose this.

Script is "lib_a.dart".

Script_name is "lib_a".

$Path is "lib/".

$Path_to_packages is "$Path/packages". 

---

$Import_file is '${Script_name}.imports'

$Packages_map is new Map($Import_file); {'unittest': 1.0.0, 'cool_package': 2.0.0 }

$Symlink_name is $Packages_map[$Package_name] # 2.0.0 for 'cool_package'

---

Path to package is "$Path_to_packages/ $Symlink_name ". 

Not only these variant.

{'unittest': 1.0.0, 'cool_package': 2.0.0 }

Mat be any "pub" consider necessary.

like this if cool_package from git.

{'unittest': 1.0.0, 'cool_package': git.cool_package.2.0.0 }

Path to cool_package by this algorithm is.

lib/packages/cool_package/git.cool_package.2.0.0/ # No conflicts with other "cool_package"'s

This work (lib_a.imports) done by "pub".

Different versions and sources do not interfere with each other.

All used simultaneously.

Each library uses its version dependencies from different sources without conflict.

[mezoni]

Uri is not critical.

file://[path]/lib/packages/cool_package/git.cool_package.2.0.0/ 

http://[path]/lib/packages/cool_package/git.cool_package.2.0.0/

path for 'file://' is '/home/user/dart/apps/app1'

path for ' http:// ' is 'www.awesome.com/myap1' 

file:/ /home/user/dart/apps/app1 /lib/packages/cool_package/git.cool_package.2.0.0/ 

http:// www.awesome.com/myap1 /lib/packages/cool_package/git.cool_package.2.0.0/

[mezoni]

Only needs than VM's uses static "imports" files.

Each per library.

Is is no hard.

If for "lib/lib_a.dart" file "lib/lib_a.imports" look as follows

==lib/lib_a.imports==

unittest = blahblahblah

==EOF

Then VM's search "unittets" for "lib/lib_a.dart" at...

packages/unittest/blahblahblah

Understand?

Billions of file name possible.

Billions of options to avoid version conflicts.

"Pub" itself decides how to call path.

It takes into account the source and version dependencies.

But that is another question. 

[Florian Loitsch]

So I went through this whole thread again (almost regretting of having revived it ;)

I am still not completely sure what you want to solve but here is my guess:

1) you don't like the fact that pub creates symlinks.

2) you would like package caches to be unified into repositories. That is, instead of having a repository on pub.dartlang.org, and just a cache on the local harddrive, you would like the local harddrive to function as repository too (without any changes).

I'm definitely less sure about (2).

The way you want to solve this, is by having a mapping on the side that tells the VM where to find its packages. You call this mappings file import-file ("${script_name}.imports"). When the VM loads a package it first reads the .imports file, which provides the necessary information to find the package in the local package-repository. For example if the local package-repository contained lib1 with versions 1.0.0 and 1.5.0 the mappings file would contain the correct version-number, (say 1.5.0) so that the VM could open ${packages_root}/lib1/[1.5.0]/lib1.dart.

Please correct me, if I got this wrong again.

==

Interestingly when we started discussing pub, we had a similar setup. Instead of having a .imports file we allowed redirecting imports. Something like (from memory):

import "redirect:../../../foo.dart"

We would still have our packages directory, but instead of having symlinks in there we would have a separate directory for each imported package, and have one lib.dart file in there that would redirect to the actual location. For example:

We would have a package-cache (for instance in ~/.dart-packages) with:

lib1/...

lib2/...

an application app1 would then have:

app1/

  main.dart

  packages/lib1/lib1.dart

  packages/lib2/lib2.dart

lib1.dart would contain just one line:

import "redirect:/home/floitsch/.dart-packages/lib1/lib1.dart"

While this is not exactly the same as having a .imports file it is pretty close (unless I'm missing something).

To be honest, I don't know why we switched to symlinks instead, but it simplifies the dart-vm (since it doesn't need to deal with redirecting libraries anymore). So I'm happy :)

Wrt to imports-files: it requires one additional (tiny) step: loading the imports file into the VM. While this is trivial to implement, it is putting something into the VM that shouldn't be there. The advantage of these .imports files would need to be huge to get this accepted.

I hope this makes sense.

// florian

--

Consider asking HOWTO questions at Stack Overflow: http://stackoverflow.com/tags/dart
 
 

[mezoni]

You are right.

>> Wrt to imports-files: it requires one additional (tiny) step: loading the imports file into the VM.

>> While this is trivial to implement, it is putting something into the VM that shouldn't be there.

>> The advantage of these .imports files would need to be huge to get this accepted.

Cons: 

1. Another extra file

Pros:

1. VM's may work without it. Pub create univocal and flat symlinks to packages and not create "imports" files.

2. No version conflicts.

3. No sources conficts.

4. Path segments (and even full paths) to dependencies may be any allowable. Not only simplest flat.

5. In the future, you can expand the syntax of "imports" if that needs.

6. import "redirect:/home/floitsch/.dart-packages/lib1/lib1.dart". You can add this feature in the "imports" is very simple.

================================

>>1) you don't like the fact that pub creates symlinks.

Yes and no.

Eventually eliminate errors.

But "simplest flat path" symlinks - this irritates me.

It is so naive.

Use versions and not to use it at full capacity.

Create problems themselves.

If I want and need Visual C++ 2008 runtime why I cant use it if it already installed?

Packages like runtime libraries.

=================================

>> 2) you would like package caches to be unified into repositories.

>> That is, instead of having a repository on pub.dartlang.org, and just a cache on the local harddrive, you would like the local harddrive to function as repository too (without any changes).

Unfication not needs.

The main thing there was the use of different versions.

"Imports" solve this problem.

=================================

>> When the VM loads a package it first reads the .imports file, which provides the necessary information to find the package in the local package-repository.

Not only the local path.

But by default, of course, the path is relative.

But you can specify the VM that the path is absolute.

[mezoni]

When you run pub and pub not detects version conflicts then pub not create "imports" by default.

But when pub detects version conflicts then pub ask you how to solve this problem.

Run 'pub --imports' or it does not guarantee compliance with the versions 'pub --noimports'.

Or is specified extra option in "pubspec.yml" to force use "imports".

[mezoni]

Only colored light.

Only the smoke cigarettes at the door of the toilet.

I am not here.

I am far for a thousand years.

I long ago vowed to never be in that situation.

"Time machine". Free translation.

[Florian Loitsch]

On Mon, Sep 24, 2012 at 7:46 AM, mezoni <andrew...@gmail.com> wrote:

You are right.

>> Wrt to imports-files: it requires one additional (tiny) step: loading the imports file into the VM.

>> While this is trivial to implement, it is putting something into the VM that shouldn't be there.

>> The advantage of these .imports files would need to be huge to get this accepted.

Cons: 

1. Another extra file

and (slightly) more work in the VM.

Pros:

1. VM's may work without it. Pub create univocal and flat symlinks to packages and not create "imports" files.

2. No version conflicts. 

3. No sources conficts.

4. Path segments (and even full paths) to dependencies may be any allowable. Not only simplest flat.

5. In the future, you can expand the syntax of "imports" if that needs.

6. import "redirect:/home/floitsch/.dart-packages/lib1/lib1.dart". You can add this feature in the "imports" is very simple.

================================

>>1) you don't like the fact that pub creates symlinks.

Yes and no.

Eventually eliminate errors.

But "simplest flat path" symlinks - this irritates me.

It is so naive.

Use versions and not to use it at full capacity.

Create problems themselves.

If I want and need Visual C++ 2008 runtime why I cant use it if it already installed?

Packages like runtime libraries.

I'm not sure I see your problems. Pub has the following restrictions:

- all libraries must agree on a specific version of the used libraries. It is not possible to use libA.1.0.0 and libA.1.5.0 in the same program.

- it should keep the VM as simple as possible.

Afaik Pub thus does the following (Bob please correct me, if I'm wrong):

- it has a repository samewhere on the hd. I don't know its layout, but I could imagine something like:

 /home/floitsch/.dart_repository

    /libA/1.0.0/lib

    /libA/1.5.0/lib

    /libB/2.2.1/lib

...

 

- inside your app you have a packages folder that symlinks into one of these directories:

 /app

    /lib/...

    /packages

       /libA -> /home/floitsch/libA/1.5.0/lib

       /libB -> /home/floitsch/libB/2.2.1/lib

Note: these symlinks are created by Pub (which has a configuration file and finds the best fitting versions).

Given that we are not allowed to have several versions of the same lib, we are fine with only having one flat namespace for each app. The repository itself, however, might contain multiple versions of the same library.

Now there is still one issue: if two vendors have a package with the same name we get into troubles:

in libA (from vendor1) we could have "import package lib2".

it provides its own "lib2".

in libB (from vendor2) we could have "import package lib2".

it also provides its own "lib2".

In this case we would have two "import('package:lib2')" with different meaning. Afaics this would require pub to modify the sources of either libA or libB to make this work.

If this is a problem you are addressing, please say so, and I will reread your posts focusing on this issue. (No additional explanations needed. I would just reread the posts.)

=================================

>> 2) you would like package caches to be unified into repositories.

>> That is, instead of having a repository on pub.dartlang.org, and just a cache on the local harddrive, you would like the local harddrive to function as repository too (without any changes).

Unfication not needs.

The main thing there was the use of different versions.

"Imports" solve this problem.

=================================

>> When the VM loads a package it first reads the .imports file, which provides the necessary information to find the package in the local package-repository.

Not only the local path.

But by default, of course, the path is relative.

But you can specify the VM that the path is absolute.

--

Consider asking HOWTO questions at Stack Overflow: http://stackoverflow.com/tags/dart
 
 

[mezoni]

Sorry, but I accidentally wrote all these posts.

Please do not view this as something serious.

It's just something that fits in my head.

Thanks for audience.

[Bob Nystrom]

On Sun, Sep 23, 2012 at 9:26 AM, mezoni <andrew...@gmail.com> wrote:

Different versions and sources do not interfere with each other.

All used simultaneously.

Each library uses its version dependencies from different sources without conflict.

I'm having trouble following this thread, but I can respond to this part.

We designed pub to specifically not allow the above case. Unlike, say npm, pub does not allow you to have two versions of a dependency used simultaneously within one app. In a language with type annotations, doing so is just asking for trouble.

Let's say you have:

myapp/pubspec.yaml

  dependencies:

    a:

    b:

a/pubspec.yaml

  dependencies:

    c: 1.0.0

b/pubspec.yaml

  dependencies:

    c: 2.0.0

So myapp uses a, b. Those both use c, but different versions. Lets say pub did allow that. Then say the program does this:

1. 'a' creates an instance of a class defined in 'c' 1.0.0. It passes it up to 'myapp'.

2. 'myapp' passes that to 'b'.

3. 'b' passes it down to its version (2.0.0) of 'c'.

From the language's perspective, those two 'c' libraries are entirely unrelated. So in the last step, the instance from 'c' 1.0.0 will fail to match any type annotations in 'c' 2.0.0.

I think the most robust and comprehensible way to deal with this is to not allow multiple versions of a dependency (which is also Bundler's model).

Cheers,

- bob

[Bob Nystrom]

On Sun, Sep 23, 2012 at 3:46 PM, Florian Loitsch <floi...@google.com> wrote:

So I went through this whole thread again (almost regretting of having revived it ;)

I am still not completely sure what you want to solve but here is my guess:

1) you don't like the fact that pub creates symlinks.

2) you would like package caches to be unified into repositories. That is, instead of having a repository on pub.dartlang.org, and just a cache on the local harddrive, you would like the local harddrive to function as repository too (without any changes).

I'm definitely less sure about (2).

The way you want to solve this, is by having a mapping on the side that tells the VM where to find its packages. You call this mappings file import-file ("${script_name}.imports"). When the VM loads a package it first reads the .imports file, which provides the necessary information to find the package in the local package-repository. For example if the local package-repository contained lib1 with versions 1.0.0 and 1.5.0 the mappings file would contain the correct version-number, (say 1.5.0) so that the VM could open ${packages_root}/lib1/[1.5.0]/lib1.dart.

Please correct me, if I got this wrong again.

==

Interestingly when we started discussing pub, we had a similar setup. Instead of having a .imports file we allowed redirecting imports. Something like (from memory):

import "redirect:../../../foo.dart"

We would still have our packages directory, but instead of having symlinks in there we would have a separate directory for each imported package, and have one lib.dart file in there that would redirect to the actual location. For example:

We would have a package-cache (for instance in ~/.dart-packages) with:

lib1/...

lib2/...

an application app1 would then have:

app1/

  main.dart

  packages/lib1/lib1.dart

  packages/lib2/lib2.dart

lib1.dart would contain just one line:

import "redirect:/home/floitsch/.dart-packages/lib1/lib1.dart"

While this is not exactly the same as having a .imports file it is pretty close (unless I'm missing something).

To be honest, I don't know why we switched to symlinks instead, but it simplifies the dart-vm (since it doesn't need to deal with redirecting libraries anymore). So I'm happy :)

"Link libraries" are still on the roadmap at some point. We started with symlinks because:

1. They're simpler to implement.

2. They work with a regular static web server. With a link library, if it redirects to a path outside of your server's root directory (which is likely), then you're hosed. With symlinks, it just works.

3. If you're willing to exclude Windows XP and require Vista or later, you can even get symlinks working on Windows, which was our main concern with them.

4. It lets us get something working without requiring the VM, dart2js, and the editor all implementing link libraries.

- bob

[mezoni]

On static servers it is also work. With symlinks.

VM request

request: www.awesome.org/myapps/myapp1/lib/lib_a.dart

request: www.awesome.org/myapps/myapp1/lib/lib_a.imports

request: www.awesome.org/myapps/myapp1/lib/packages/[route_a]

request: www.awesome.org/myapps/myapp1/lib/packages/[route_a]/pkg_a.dart

request: www.awesome.org/myapps/myapp1/lib/packages/[route_a]/pkg_a.imports

request: www.awesome.org/myapps/myapp1/lib/packages/[route_c.1.0.0]/pkg_dart.imports

request: www.awesome.org/myapps/myapp1/lib/packages/[route_c.1.0.0]/pkg_c.imports

request: www.awesome.org/myapps/myapp1/lib/packages/[route_b]

request: www.awesome.org/myapps/myapp1/lib/packages/[route_b]/pkg_b.dart

request: www.awesome.org/myapps/myapp1/lib/packages/[route_b]/pkg_b.imports

request: www.awesome.org/myapps/myapp1/lib/packages/[route_c.2.0.0]/pkg_dart.imports

request: www.awesome.org/myapps/myapp1/lib/packages/[route_c.2.0.0]/pkg_c.imports

Where relative routes defined by "pub" as  [source] + name + ["." + version]. 

for pub.dartlang.org source is empty.

for sdk version is empty

for other source is source.hash # or as in pub-cache/hosted/????

E.g. relative routes (symlink names) that works even if package in cache but correct symlinked.

html5lib.0.0.1

html5lib.1.0.0

sdk.args

git.author1.cool.1.0.0 # as specified for girt

723467234967293675.cool.1.0.0 # or awesome.org.cool.1.0.0. May be as in pub-cache/hosted/???

E.g.

request: www.awesome.org/myapps/myapp1/lib/packages/[route_b]

request: www.awesome.org/myapps/myapp1/lib/packages/[route_b]/pkg_b.dart

request: www.awesome.org/myapps/myapp1/lib/packages/[route_b]/pkg_b.imports

==pkg_b.imports

htmllib = html5lib.1.0.0 # html5lib.1.0.0 is symlink in any [lib/bin/...] application packages/ folder

args=sdk.args

==EOF

request: www.awesome.org/myapps/myapp1/lib/packages/[html5lib]/pkg_dart.imports

request: www.awesome.org/myapps/myapp1/lib/packages/[html5lib]/pkg_c.imports

Whrere "html5lib" route is html5lib.1.0.0

And "html5lib.1.0.0" is symlink in package folder in aplication folder.

Only needs secification for symlink name.

If "a.dart" script is in cache than "a.imports" has relative for any application routes.

E.g

c = c.1.1.0

args = sdk.args

myapp/lib/packages/c.1.1.0/c.dart

myapp/lib/packages/c.1.1.0/c.import

c.1.1.0 is symlink to /home/user/pub-cache/hosted/c/1.0.0

sdk.args is symlink to /home/user/dart/dart-sdk/pkg/args/

c.1.1.0  and sdk.args is system wide aliases (symlink name for "pub")

relative_route = pub.getAlias(package_name, [source, version]);

var args = pub.getAlias('args', source: 'sdk'); # sdk.args

var htmllib = pub.getAlias('htmllib', version: "1.0.0"); # htmllib.1.0.0

var from_git = pub.getAlias('cool', source: git_source); # cool.git.2394873948723906873269875690586

SYSTEM AND WORLDWIDE Aliases defined by pub that work anywhere.

[mezoni]

VM's don't create aliases.

VM's get aliases from *.imports.

Aliases is unique names of symlinks for localhost (local mashine).

Aliases is unique_package_keys.

Through alias any may request in app package/ folders concrete package if this symlink created by pub.

You can take them out of the *.imports.

Pub install packages.

Installation for "a".

Download...

Other work...

Install dependency "a".

How to call dependency alias for "htmllib5" from "pub.dartlanf.org" that will be works anywhere?

Hmmm?

Use unique alias!!!

var unique_htmllib5_alias = pub.getAlias("htmllib5", htmllib5.source, htmllib5.version);

var routes_for_a = new Map();

routes_for_a[htmllib5.name] = unique_htmllib5_alias; # htmllib.1.0.0

==a.imports==

htmllibl = unique_htmllib5_alias # Symlink names not confilcts and at least system wide unique

==EOF

Pub uses aliases as symlink names when creates symlinks in app/packages folders.

[mezoni]

I do not understand why you can not understand the simplest things.

[mezoni]

Unique alias is an alternative to unification cache structure.

A symbolic link can be referenced anywhere.

But the name of the link must be unified.

It must be unique and unambiguous.

Then there is no problem with relative path segments with all protocols and hosts.

E.g.

cache dir is c:\pub-cache

cool1 is in cache at [cache]/****

cool2 is in cache at [cache]/****

==[cache]/****/cool1.imports

cool2 = unique_alias_for_concrete_cool2

==EOF

How "cool1" may refer "cool2" in "app/lib/packages"?

At "app/lib/packages/unique_alias_for_concrete_cool2"

"unique_alias_for_concrete_cool2" is an alias for concrete "cool2".

Where is "unique_alias_for_concrete_cool2" for "cool1"?

At "app/lib/packages/unique_alias_for_concrete_cool1/cool1.imports".

Where is "unique_alias_for_concrete_cool1" for app/lib/a.dart?

At "app/lib/a.imports".

==app/lib/a.imports

cool1 = unique_alias_for_concrete_cool1

==EOF

What is an alias "unique_alias_for_concrete_cool1"?

It is folder (symlinked) in app/[lib/bin/..]/packages/ created by pub.

[mezoni]

"Impors" file can be structured a little more complicated if you have other plans.

E.g.

package1.alias = alias

package2.host = host

[mezoni]

>> 1. 'a' creates an instance of a class defined in 'c' 1.0.0. It passes it up to 'myapp'.

>> 2. 'myapp' passes that to 'b'.

>> 3. 'b' passes it down to its version (2.0.0) of 'c'.

Firstly it is not a typical situation. 

Then that means that it is not the reason for the refusal.

Second this is not directly related to the package management system, and this is offtopic.

Thirdly there may be other situations.

Different libraries in cache or in application may differently use different dependencies.

This is already on the conscience of the programmer.

The package management system is not to blame.

What if you do not want to change my library 'a'?

But I have a lib 'b' and she had new versions.

The application is not one big library.

An application can contain multiple libs, bins, tests and others.

And each one is a stand alone product.

And they have their own requirements.

[mezoni]

"myapp.liba" use "a" 1.5.0

"myapp.liba" use "b" 1.0.0

"a" 1.5.0. and "b" 1.0.0 is compatible.

All works.

myapp.pubspec.yaml

"myapp.liba" use "a" >=1.5.0. <2

"myapp.liba" use "b" >=1.0.0. <2

"myapp.libx" use "a" >2  < 3

===================

"myapp.libx" use "a" 2.5.0

All works.

[mezoni]

You say it not allowed.

OK. I agree.

But what is myapp use ["fortan", "algol"]

"fortan" not use "algol".

"algol" not use "fortan".

"fortan" use "string_utils" v.1.0.0.

"algo" use "string_utils" v.2.0.0.

string_utils" v.1.0.0. is not comptible with string_utils" v.2.0.0.

How "fortan" may refer "string_utils" v.1.0.0?

How "algol" may refer "string_utils" v.2.0.0?

How to refer different "string_utils" in one package folder?

Refer BY UNIQUE_ALIASES.

[Ladislav Thon]

>> 1. 'a' creates an instance of a class defined in 'c' 1.0.0. It passes it up to 'myapp'.

>> 2. 'myapp' passes that to 'b'.

>> 3. 'b' passes it down to its version (2.0.0) of 'c'.

Firstly it is not a typical situation. 

It is. Go spend few days in Java/Maven world, then you will know.
 

Then that means that it is not the reason for the refusal.

It is.
 

Second this is not directly related to the package management system, and this is offtopic.

It is.

LT

[mezoni]

VM, editors and others useful tools does not have to know the algorithm generate a unique alias.

They know where to take it.

When dart2js generates into a single file then it uses a similar technique for class name resolution for internal naming dependencies.

[mezoni]

>> Go spend few days in Java/Maven world, then you will know. 

I didn't know about that.

When I'm wrong.

[Florian Loitsch]

It is not possible (and should not be). This is explained here:

http://www.dartlang.org/docs/pub-package-manager/versioning.html 

Refer BY UNIQUE_ALIASES.

--
Consider asking HOWTO questions at Stack Overflow: http://stackoverflow.com/tags/dart
 
 

[mezoni]

>> It is not possible (and should not be). This is explained here:

>> http://www.dartlang.org/docs/pub-package-manager/versioning.html  

I know this, and so I suggest another option.

You answer me that is impossible.

And I say change it and it will be possible.

You say that it is impossible because it is written that it is impossible.

I say that with the new system it would be possible but you will find other reasons.

I will not argue with you.

=============

This reminds me of an old anecdote.

Two deaf men are walking along the road.

One asks the other.

- Where are you going? Go shopping?

The other answers.

  - No. I'm going shopping.

The first says.

- It is a pity that you do not go out shopping. 

The second man says.

- Good-bye, good man.

- It is a pity that you do not go out shopping.

============= 

One application can not use different versions.

I knew it was bad. And I agree.

But I do not agree that it is necessary to limit the already installed packages.

[mezoni]

OK.

I understand my mistake.

I read how it works in Ruby

I was wrong.

You are right. 

[mezoni]

I want this work on Windows XP.

When can we expect "pub --deploy" or "pub install -nosymlinks"?

Copy all deps into package folders.

[Bob Nystrom]

Right now, we don't support XP. Doing so is hard for us (because of the lack of symlinks) and it isn't clear how many users would benefit from it. Can you file a bug for this? If it's something that a lot of users care about, that will help us prioritize it appropriately.

Thanks!

- bob