Changing the editor workflow

[Joao Pedrosa]

Hi all,

With Pub build and so on, the way to package and develop an application has changed. We hear that more should go into the package's lib directory to make the build happier.

Users of older editor versions and early adopters may have their own way of developing and building an application. With the editor changes, those folks would need to adapt to the new way of doing things.

Based on some recent discussions, it appears as though editor users would need to formalize their libraries and applications into packages that depended on one another to avoid copying dependency files everywhere. Allowing the tool to do that instead with appropriate package information.

I am writing this in part to try to get to terms with these changes.

My worry is that with packages we are going more coarse-grained in handling the files. I am not sure yet how that would pan out for everyone. I think independent users would need to start creating tens and potentially hundreds of packages for every small library and application they could create. How would that be handled then?

The alternative is users creating their own build conventions may drive them away from the standard workflow.

We really need to love Pub now to enjoy the standard workflow. We need all the information on Pub we can get just to be sure. And creating alternatives is hard since Pub is integrated into the editor by default.

I need to rethink my current directory layouts to take advantage of Pub. I imagine having to create tens of packages already. Does Pub support circular package dependency?

I am leaning towards creating my own custom build as well. As I imagine others will do.

Cheers,

Joao

[Matthew Butler]

I'm not sure I follow personally on what limitation you feel you're running into?

On Saturday, April 12, 2014 3:58:46 AM UTC-3, Joao Pedrosa wrote:

Hi all,

With Pub build and so on, the way to package and develop an application has changed. We hear that more should go into the package's lib directory to make the build happier.

Users of older editor versions and early adopters may have their own way of developing and building an application. With the editor changes, those folks would need to adapt to the new way of doing things.

Based on some recent discussions, it appears as though editor users would need to formalize their libraries and applications into packages that depended on one another to avoid copying dependency files everywhere. Allowing the tool to do that instead with appropriate package information.

I am writing this in part to try to get to terms with these changes.

My worry is that with packages we are going more coarse-grained in handling the files. I am not sure yet how that would pan out for everyone. I think independent users would need to start creating tens and potentially hundreds of packages for every small library and application they could create. How would that be handled then?

That most certainly shouldn't be the case and in fact sounds like the opposite of what should be done. A package (be it a libraries or an application) could contain tens or hundreds of libraries but you shouldn't need to create a separate pub package for each (and most certainly do not create multiple packages for one library). 

The alternative is users creating their own build conventions may drive them away from the standard workflow.

While it is totally an option, it certainly hope this doesn't come to pass. I'm not sure why the pub package manager and build tool would make you feel you need to completely replace it. 

We really need to love Pub now to enjoy the standard workflow. We need all the information on Pub we can get just to be sure. And creating alternatives is hard since Pub is integrated into the editor by default.

I need to rethink my current directory layouts to take advantage of Pub. I imagine having to create tens of packages already. Does Pub support circular package dependency?

See this document for some information on package layouts (though note, that it is a little out dated and the assets directory has been depreciated.)
https://www.dartlang.org/tools/pub/package-layout.html

 

I am leaning towards creating my own custom build as well. As I imagine others will do.

Cheers,

Joao

 

Personally at the moment I'm working on a DAMN project (Dart Angular MongoDb and Nginx). Both my client-side and server side apps are contained in the same package. They share the libraries internally rather than creating a separate package to share between the two of them. I did create a base separate base package for my server-side which is a very bare-bones framework which I plan to reuse for other projects in the future, but everything specific to this particular application (eg the specific path handling and interfaces) are all within the same app as the client-facing parts. 

Apart from my custom web-framework (which is just a layer over the http_server package to handle routing since the serve-side routing package is lacking and mongodb interface) and angular I don't really have any external package dependencies.

Generally my directory layout follows along the lines of separation of interest. For instance instead of using a lib/ and lib/src/ folder for most files my layout is something like:

lib/admin/ # contains admin components of the app

lib/sales/ # Contains the sales components of app

lib/login/ # login components

... etc.

So for instance in lib/login/ I have the .html, .dart and any specific .css files that may be required.

I also have folder similar to:

lib/common/ # resources such as .css which are shared by multiple components.

I have a similar structure under the lib/src/ for any parts which should not be used in the app independently and should rather be part of one of the main components.

Note: I'm using the term 'components' but they may not be actual Angular Components (or polymer). It may be just a controller, or a service etc.

I then have just a couple of entry points under the web/ and bin/ parts of my package which import the other parts of my app. E.g., my main.dart in web/ imports my various services, components, controllers, modules etc. Add them to a module and then run ngBootstrap.

My main in my bin/ directory adds my paths and associates the handlers and then initializes and starts the server.

Matt

[Joao Pedrosa]

Hi Matt,

If we are going to try to have fewer packages that include more libraries and applications, it also means moving more stuff into the package's lib directory.

I just counted the number of dart files I have in many different directories. About 379. Some are duplicates. They are spread over many libraries and applications.

Considering that I don't depend on any current Pub package, I haven't been using Pub until now. I don't have the needed Yaml files yet.

Generally speaking I have 5 major directories with sub-projects:

c:/repo/estava/

c:/repo/reluzir/

c:/repo/segments/

c:/repo/try_tetris/

c:/repo/wrong_bench/

The plan was to have reusable libraries mostly under segments and try_tetris. I started segments only recently.

Until now I have been able to have the libraries spread out over as many directories as needed. I generally used a directory layout like this:

c:/repo/estava/hotr/bench/

c:/repo/estava/hotr/bin/

c:/repo/estava/hotr/lib/

c:/repo/estava/hotr/test/

Under the test directory I also had combination of dart + html application code for testing components. Now with Pub build I think I would need to move those inside lib. And currently I also have dart + html application code under the c:/repo/estava/hotr/ directory, and those too would be moved inside lib.

The problem though is that paths are relative. Once pub starts using paths, everything being referenced by the files would have to be turned into a package. So once they got moved into pub's build directory the paths would still work. So basically the reusable libraries in the try_tetris and segments directories would need to reference each other and be referenced by others using the pub package interface. As I don't yet have experience with pub or Dart packages beyond using what's in the core APIs, it's quite a jump as of now.

I guess when they said that Dart was for structured programming they really meant it. :-)

Before I can run with pub packages I first need to walk with pub packages. One step at a time. In a lot of places where I reference files using relative paths I think it would need to change. It's all so new to me.

Cheers,

Joao

--
For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 
To file a bug report or feature request, go to http://www.dartbug.com/new

To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

[Joao Pedrosa]

Hi,

I just tried pub build on my largest library try_tetris and I don't know what is going on. It compiled files to JavaScript that it doesn't have. It seemed to pick up on the directory named "test" inside of "lib" as I moved all the relevant files inside "lib". But files in other directories were missed. It appears that we should label directories according to some convention.

I also really did not like the idea of starting with an empty directory with every pub build command. It potentially means more work to compile everything so it can be tested for cross-browser compatibility issues and so on.

I guess I will revert the changes and try to do without pub build for now. Until Dart 2.0.

Thanks! Don't worry. It's good. I think I can just call "dart2js" when I need to.

Cheers,

Joao

[Matthew Butler]

On Monday, April 14, 2014 11:36:53 AM UTC-3, Joao Pedrosa wrote:

Hi Matt,

If we are going to try to have fewer packages that include more libraries and applications, it also means moving more stuff into the package's lib directory.

I just counted the number of dart files I have in many different directories. About 379. Some are duplicates. They are spread over many libraries and applications.

Considering that I don't depend on any current Pub package, I haven't been using Pub until now. I don't have the needed Yaml files yet.

Generally speaking I have 5 major directories with sub-projects:

c:/repo/estava/

c:/repo/reluzir/

c:/repo/segments/

c:/repo/try_tetris/

c:/repo/wrong_bench/

The plan was to have reusable libraries mostly under segments and try_tetris. I started segments only recently.

Until now I have been able to have the libraries spread out over as many directories as needed. I generally used a directory layout like this:

c:/repo/estava/hotr/bench/

c:/repo/estava/hotr/bin/

c:/repo/estava/hotr/lib/

c:/repo/estava/hotr/test/

Under the test directory I also had combination of dart + html application code for testing components. Now with Pub build I think I would need to move those inside lib. And currently I also have dart + html application code under the c:/repo/estava/hotr/ directory, and those too would be moved inside lib.

The problem though is that paths are relative. Once pub starts using paths, everything being referenced by the files would have to be turned into a package. So once they got moved into pub's build directory the paths would still work. So basically the reusable libraries in the try_tetris and segments directories would need to reference each other and be referenced by others using the pub package interface. As I don't yet have experience with pub or Dart packages beyond using what's in the core APIs, it's quite a jump as of now.

I guess when they said that Dart was for structured programming they really meant it. :-)

Before I can run with pub packages I first need to walk with pub packages. One step at a time. In a lot of places where I reference files using relative paths I think it would need to change. It's all so new to me.

Cheers,

Joao

Mm I can see some of the issues you'll be experiencing then. As you mentioned, Pub love is important, even if you don't explicitly plan to use pub for any dependencies etc. [Aside: I would highly recommend investigating if it would be worth it as there are many great packages already in place many even by the dart team themselves. Some of which even replaced functionality of previously core libraries including crypto, unittest, matchers, args and more].

My first suggestion is to create a package (or possibly several depending on the scope of the code). These will be considered your 'shared libraries'. Any libraries code which you can/will reuse in other projects should be compiled into at least one package. Just one utility package would suffice, look at quiver as an example[1]. Once you do this, you can have a test directory in the package (not in the lib/ folder) which provides the unittest coverage for the public methods/functions/classes in your librar(y|ies) of that package. Within the test directory, you can have one main test file and sub folders for the specific libraries contained within the package.

In the lib/ directory most/all of your .dart files will reside (with the exception of the test ones). Within the lib directory you can use whatever structure suites you fancy. Any 'entry points' will usually be in a file in lib/xxx.dart where xxx is the same as your package's name. A difference is quiver where it has many upperlevel entry points (eg quiver/io.dart, quiver/time.dart). those files will refer to the implementation files usually by a relative import. eg:

// quiver/lib/time.dart
part 'src/time/clock.dart';
part 'src/time/duration_unit_constants.dart';

(Alternatively you can import them, or import and show depending on how you want to structure your library).

In your package's test/ directory (again top-level not under the lib directory) you import your package's libraries the same as you would import you would from another package:

// sharedlibrary_test.dart

import 'packages:sharedlibrary/sharedlibrary.dart';

import 'packages:unittest/unittest.dart';

void main() {

  // full tests/groups here

  test(....);

}

Next within your applications (like try_tetris), you would include a path dependency[2] to your shared package in the pubspec.yaml file (note your sharedlibrary package will also require a pubspec.yaml file naming it, etc even if you don't have any dependencies). Then within your bin/ or web/ directories you would import it with a package declaration:

// web/try_tetris.dart

import 'package:sharedlibrary/sharedlibrary.dart';

void main() {

  // whatever code here

}

The try_testris package would have whatever app specific files in lib/, except your web page entry point in the web/ folder. Any tests specific to to the try_testris package would go in the toplevel folder test/ and it can also import any package local libraries or sharedlibrary files with the same mechanism mentioned above.

While I realize since you've been working quite a bit without using the pub package manager that it would involve a lot of changes to the current layout of your application, I think ultimately it would pay off for not only these projects to be shared with others and distributed how you may wish, it would also help with getting the fundamentals in place for future projects you may have which may require much more work and external packages.

Finally if you wanted to share some or any of the projects/files with a little documentation as to how you have them organized and what's shared etc, I may be able to help you wish sending a pull request to best convert them to packages and allow for the dart type package organization.

Matt

[1]: https://github.com/google/quiver-dart 

[2]: https://www.dartlang.org/tools/pub/dependencies.html#path-packages

[Bob Nystrom]

On Mon, Apr 14, 2014 at 7:36 AM, Joao Pedrosa <joaop...@gmail.com> wrote:

Generally speaking I have 5 major directories with sub-projects:

c:/repo/estava/

c:/repo/reluzir/

c:/repo/segments/

c:/repo/try_tetris/

c:/repo/wrong_bench/

That looks fine. Each of those would likely become one package.

 

The plan was to have reusable libraries mostly under segments and try_tetris. I started segments only recently.

You can give estava, reluzir, and wrong_bench access to those using path dependencies on segments and try_tetris.

For example, let's say today you have:

c:/repo/segments/reusable_lib.dart:

    someFunction() {

      print("I am reusable!");

    }

c:/repo/estava/application.dart:

    import '../segments/reusable_lib.dart';

    main() {

      someFunction();

    }

To turn that into pub packages, you just need to create:

c:/repo/segments/pubspec.yaml:

    name: segments

(Literally just a one line YAML file.)

c:/repo/estava/pubspec.yaml:

    name: estava

    dependencies:

      segments:

        path: ../segments

Then take reusable_lib.dart and move it to c:/repo/segments/lib/reusable_lib.dart.

Finally, change the import in application.dart to:

    import 'package:segments/reusable_lib.dart';

After that, run "pub get" from within c:\repo\estava and everything should work.

Cheers!

- bob

PS: I see Matthew just replied with almost the same thing. Dang it! He beat me to it!