Metacello - Seaside 2.9 proof of concept
Dale Henrichs
'The Seaside Builder' is a Seaside application where you select Seaside 2.9 packages that you would like to have loaded into your Squeak/pharo image. You can select from one of two Seaside2.9 alpha versions: 2.9.0-alpha2 or 2.9.0-alpha3. When you're done making your package selections a load script is generated for you using one of the 4 supported loading engines (Installer script , (Monticello) Load script, Monticello Configuration, Package Universe, and Sake Package Definition).
Each of the load scripts includes all of the (explicit) prerequisite packages as well as implicit packages (platform specific packages, etc.).
I didn't duplicate the exact functionality of 'The Seaside Builder' in my proof of concept, because I felt that the packages should be organized a little differently.
Metacello _is_ defined in terms of Monticello packages, but it also has "Grouping of packages by function to help eliminate user confusion"[2]. In particular, when looking at 'The Seaside Builder' it wasn't clear what packages I would need if I wanted to include all of the packages I get when selecting 'Seaside-Development', but excluding the development specific packages. There was no obvious selection for something like 'Seaside-Deployment' - this is the area where defining groups of packages becomes useful.
[For those of you who were on the private mail list, I have used the terms 'lineUps' and 'features' in the past, but all I was really talking about was groupings of packages. I blame my confusion on the fact that I didn't look at the package management Wikipedia page until Sunday:)].
If you take a look at the LoadOrder[3] page, you'll see that packages are grouped under titles like:
Basic
Traditional
Comanche
These groupings are closer to what I was looking for, but in the end I settled on even finer grained groupings:
Platform
Platform Tests
Core
Core Tests
Basic
Basic Tests
Traditional
Traditional Tests
Traditional Development
Traditional Development Tests
Examples
Comanche
Comanche Tests
So, for the proof of concept, I created a class called WAVersionMap (a subclass of MetacelloVersionMap[4]) that encapsulates the list of packages, package versions, package dependencies, repositories, groupings, and group dependencies for both 2.9.0-alpha2 and 2.9.0-alpha3.
If you would like to play with the proof of concept, manually load Metacello-dkh.147 from http://seaside.gemstone.com/ss/test to get Metacello loaded. Metacello can be loaded into both Squeak and Pharo, however, if you are planning on loading 2.9.0-alpha2, then you should use Squeak, because that version doesn't run on Pharo.
With Metacello loaded we can start playing with the package management DSL. The DSL is in flux so feedback on better approaches is appreciated. For example, The following expression can be used load the WAVersionMap or you can do it manually if you'd like:
{{{ 'Seaside-VersionMap-dkh.7'}}.
{{ 'http://seaside.gemstone.com/ss/test'}}} mcLoad
As a side note if you end up poking around in the inspector on some of the Metacello structures you will see lots of structured arrays used as printStrings. Very early on, Metacello used structured arrays for defining the package management data structures and there are still plenty of structured array artifacts left lying around as I've pushed functionality around. Ultimately they will go away.
[for those of you who have already loaded an earlier version of WAVersionMap, you need to execute 'WAVersionMap resetInstance' ... one of things I will be getting rid of].
With that said, using structured arrays for communicating loadable sets of packages and their repositories seems to me to be very convenient:
{{{ 'Metacello-dkh.147'. }.
{ 'Seaside-VersionMap-dkh.7'. }.}.
{{ 'http://seaside.gemstone.com/ss/metacello'. }.
{ 'http://seaside.gemstone.com/ss/test'. }}} mcLoad
I'm interested in alternative forms of expressions.
Speaking of alternatives, here are a series of expressions that all end up loading the same set of packages from version '2.9.0-alpha3':
WAVersionMap loadBasicTests.
WAVersionMap latestVersion load: { 'Basic Tests'. }.
(WAVersionMap version: '2.9.0-alpha3') load: { 'Basic Tests'. }.
If you want to load 'Basic Tests' from '2.9.0-alpha2', then you'd execute the following expression:
(WAVersionMap version: '2.9.0-alpha2') load: { 'Basic Tests'. }.
From the perspective of mimicking the functionality of 'The Seaside Builder', you'd need to click on the following packages:
Seaside-Canvas
Seaside-Component
Seaside-Core
Seaside-Platform
Seaside-RenderLoop
Seaside-Session
Seaside-Tests-Canvas
Seaside-Tests-Component
Seaside-Tests-Core
Seaside-Tests-Environment
Seaside-Tests-RenderLoop
Seaside-Tests-Session
to get the same set of packages loaded (okay there's a bug in WAVersionMap and Seaside-Squeak-Environment isn't loaded:). For Seaside, the Monticello packages are loaded linearly in order to control class initialization order. Metacello allows for both #atomic and #linear loads.
The WAVersionMap expressions I've covered so far are all supported in the proof of concept. The intent with Metacello is to allow something like the following:
(WAVersionMap version: '2.9.0-alpha2')
load: { 'Traditional'. 'Seaside-Email'. 'Seaside-HTML5'. }.
Which would cause the Traditional group to be loaded along with the Seaside-Email and Seaside-HTML5 packages (and their dependent packages).
Packages and/or groups are the basic unit of loading/unloading. The pattern that is emerging is that groups and packages may be mixed and matched and then loaded against a particular version of a project.
Furthermore, a version, the result of evaluating:
(WAVersionMap version: '2.9.0-alpha2')
or
WAVersionMap latestVersion
should support the following operations:
load - load all packages defined for the version
load: - load the groups and packages in the given list
unload - unload the loaded packages for the version
unload: - unload the groups and packages in the given list
update - load new versions of the loaded packages based on the new version on
top of an existing installation
update: - load new versions of the loaded packages based on the new version plus
any additional packages included in the list of groups and packages on
top of an existing installation
load and update are very similar, the primary difference would be in how warnings and errors are generated (i.e., a warning/error would be raised for a load operation where there are already packages from the project loaded and a warning/error would be raised if an update encountered no loaded packages for the project). An update may also do an #atomic load instead of a #linear load.
So a sequence of Metacello operations may be something like the following:
(WAVersionMap version: '2.9.0-alpha2')
load: { 'Traditional Tests'. 'Seaside-Email'. 'Seaside-HTML5'. }.
initial load followed by runnning a bunch of tests then:
(WAVersionMap version: '2.9.0-alpha2')
unload: { 'Tests'. }.
which would unload all of the tests (assuming that 'Tests' was defined as a group of all test packages), leaving the Traditional group, and the Seaside-Email and Seaside-HTML5 packages still loaded. Which then might be followed by:
(WAVersionMap version: '2.9.0-alpha3')
update: { 'Traditional Development'. }.
which would be cause the 2.9.0-alpha3 versions of the all of the already loaded packages to be updated in addition to loading the packages included in the Traditional Development group.
Well at least that's the thought....
I'm pretty confident that all of these things can be done with Metacello. #update: presents some interesting problems that _could_ be problematic under certain conditions...
Next up, I will go into more detail about how the versionMaps are/will be defined.
After working through the Seaside proof of concept, I've seen areas where I want to restructure things, so I will be describing what I'm thinking about in terms of the next version of Metacello, rather than a description of the current version.
I am interested in feedback on the Metacello DSL that I've described so far.
If you look at the code in WAVersionMap, I'm interested in feedback there as well (remember to translate features to groups), also keep in mind that I there will be changes in both the package and group methods - I intend to:
- change the way prerequisite and dependent packages are defined
- add a way to specify pre/post load/unload actions
- include project/version to project/version dependencies
Dale
[1] http://builder.seaside.st/
[2] http://groups.google.com/group/monticello-dev/browse_thread/thread/1d9b19e40e0aaa7b
[3] http://code.google.com/p/seaside/wiki/LoadOrder
[4] http://groups.google.com/group/monticello-dev/browse_thread/thread/2d93bc7e32088c73?hl=en
Adrian Lienhard
At cmsbox/netstyle.ch we have been using a combination of package
dependencies and configuration maps. There are situation where our
solution does not work that well and I wonder whether Metacello could
help here.
We have the following (simplified) setup:
PACKAGES
ARoot (in repository X)
- A-view
- A-model
- A-model-tests
- B-library
- B-library-tests
BRoot (in repository Y)
- B-library
- B-library-tests
CONFIGURATIONS
Production
- A-view
- A-model
- B-library
Developer
- A-view
- A-model
- A-model-tests
- B-library
- B-library-tests
BRoot is a common package that is used by several projects. The config
maps are used to define sort of a slice of versions.
Now, when loading, merging, and committing code, one always works with
the root packages ARoot and BRoot. The problem is that when B-library
is dirty and you save ARoot before BRoot, then B-library will be saved
to the repository X.
As far as I see, Metacello is supposed to provide configurations for
loading, upgrading and unloading code, but could it as well be used to
manage the scenario described above?
Cheers,
Adrian
Dale Henrichs
Adrian,
I think that it could, to the extent that Metacello doesn't try to dictate where packages are stored. As I get into more detail you'll see that the class MetacelloPackagesSpec has many things in common with Monticello Configurations, but MetacelloPackagesSpec is designed to solve slightly different problems.
Metacello itself is not aimed at solving the day to day development team problems, but if you were using Metacello for package management and configuration, a "separate mechanism" could be used for saving dirty packages. For example, adding a #save command that traversed the list of packages in a versionMap and saved dirty packages is conceivable.
If you look at the class MetacelloMCSpecLoader, you'll see that there is TrueRepositories class variable. When it is set to true, the repository that a package is loaded from is put in its repositoryGroup, so that the source repository is preserved. It's a little more expensive to load packages this way, but if you are planning on doing development on the things that Metacello loads, then it is important to preserve the source repository.
I hesitate to tackle all of the team day to day development issues with Metacello, but I think that Metacello could work well if used on a day to day basis for managing package configurations.
Dale
Dale Henrichs
Adrian,
I thought I'd mock up the way that I'd approach your configuration in Metacello.
First I'd imagine that you'd no longer use ARoot and BRoot, since they are the root (:<) of the problem.
So the contents of CMSVersionMap based on your example:
packages:
A-view
A-model
A-model-tests
B-library
B-library-tests
repositories:
repository X
repository Y
groups:
Production group:
A-view
A-model
B-library
Development group:
Production group [include packages from Production group]
A-model-tests
B-library-tests
Loading the Development group would load everything and loading the Production group would exclude the tests.
I do intend on supplying methods for updating the package versions for a particular version, since that would be a required step for preparing a new version for release...
Dale
stephane ducasse
default empty package with required packages?
In Moose we have the same problems.
Now one problem we have is that publishing MooseAll to publish all the
packages
is good but when we are coding in parallel we are constantly merging.
I feel that blessing level as in Store is still missing in MC.
May be this is supported in SqueakSource as there is an option.
Then I have the question of how a package is responsible to load well
Now for Moose we have a convention that each high level package is
able to load itself
MooseLoader class>>load
"self load"
ScriptLoader new installingInstaller.
ScriptLoader new installingUniverse.
self loadMondrian.
self loadGlamour.
self loadSmacc.
self loadNile.
self loadFame.
self loadMoose.
self loadRIO.
MooseLoader class>>loadMoose
Installer ss
project: 'Moose';
"answer: '*CollectionExtensions*' with: true;"
addPackage: 'Moose-All';
installQuietly .
MooseLoader>>loadGlamour
"self loadGlamour"
Installer ss
project: 'Glamour';
install: 'GlamourLoader'.
(Smalltalk at: #GlamourLoader) load
So we do not want that clients have to know the dependencies
Stef
stephane ducasse
groups inside it?
I see packages and repositories
I prefer the smalltalk way you use below.
Dale Henrichs
Stef,
In Metacello, I would assume that you'd define a versionMap for each of the units: Mondrian, Glamour, Smacc, Nile, Fame, Moose, and RIO.
With Metacello you can express project to project dependencies, so if Moose depends upon Fame, Nile and Smacc. That dependency can be expressed.
I plan on talking about how you will express project to project dependencies (versionMap to versionMap dependencies) later today...BTW, there is a way to express versionMap to versionMap dependencies in the current prototype...
With Metacello, I want the dependencies to be completely transparent to the client. The MooseVersionMap class will be the moral equivalent of your MooseLoader class - the client wants Moose, the load the MooseVersionMap class, then load a version of Moose and all of the other required projects will be loaded.
Dale
Dale Henrichs
----- "stephane ducasse" <stephane...@gmail.com> wrote:
| On May 19, 2009, at 10:20 PM, Dale Henrichs wrote:
| > {{{ 'Metacello-dkh.147'. }.
| > { 'Seaside-VersionMap-dkh.7'. }.}.
| > {{ 'http://seaside.gemstone.com/ss/metacello'. }.
| > { 'http://seaside.gemstone.com/ss/test'. }}} mcLoad
|
|
| I'm not sure that this format is readable. What are the variable
| groups inside it?
| I see packages and repositories
| I prefer the smalltalk way you use below.
|
Well that's enough for me. I'll stop messing around with structured arrays and expunge their use from Metacello for both constructors and printStrings.
Dale
Dale Henrichs
----- "stephane ducasse" <stephane...@gmail.com> wrote:
| I like the idea of have groups but I do not see how they different
| from
| default empty package with required packages?
The difficulty with using "empty package with required package" is that there are problems with using the technique. The "empty package" is very brittle for use within a development group and is nearly impossible to share with people outside of your development group (except as a readOnly artifact).
Metacello groups are intended to provide an easy way to share groups of packages across development group boundaries.
It seems that every project has the following elements: core packages, test packages, addon packages.
The 'core packages' are often dependent upon a set of other projects, lets say: Seaside and Smacc.
With Seaside2.9 you have to be more specific ... what package(s) do you depend upon? ... however, I don't think that it is a good idea to use package to package dependencies across project boundaries - what if the Seaside folks decide to refactor a package and the class that I depended upon has moved. I think that for project to project dependencies it is safer to use groups. Of course, there are times, when package to package dependency is the right absolutely right mechanism, so I think that both ways need to be supported.
Getting back to my example, I would prefer to say that my 'core packages' is dependent upon 'Seaside core' and 'Smacc core'. Seaside has separate tests, so my 'test packages' should depend upon 'Seaside core tests', etc.
It is possible to _express_ the above dependencies using 'empty packages', it isn't practical to do so. Metacello is aimed at addressing the sharing of code with folks outside of the immediate developer group and things that work fine within a group start breaking down real quickly when applied to disparate groups.
Metacello groups are designed for sharing (I hope:), and therefor don't carry the baggage of a technique designed for one thing and "used for sharing".
| In Moose we have the same problems.
| Now one problem we have is that publishing MooseAll to publish all the packages
| is good but when we are coding in parallel we are constantly merging.
There two things that are really good when using "empty packages": bulk saving (to the same repository) and bulk loading.
Heaven help you if you want to save to different repositories and heaven help you if you don't want everything in the "empty package" loaded.
If one is releasing a reasonably complex project for others to use, you have to be comfortable with the fact that it is highly likely that folks will modify your code to make the best use of it in their application. It's not a goal, but it is a reality. Monticello makes it real easy maintaining your own version of a package.
I feel exactly the same way about the package configuration issues. If I modify base code for a project, it is highly likely that the configuration requirements will change as well. Metacello should make it real easy to maintain your own version of a versionMap (using merge, etc.)
The Pragmas style for defining the contents of versionMap, is aimed at making it real easy to extend and maintain those extensions across multiple versions of the source project.
| I feel that blessing level as in Store is still missing in MC.
| May be this is supported in SqueakSource as there is an option.
Just this morning in the shower, I was thinking about blessings and Metacello. Since all of the versions are tracked in the single class and every time to update the version information (for public consumption) you will commit a new version of the package, it should be relatively straightforward to maintain a list of the blessed versions within a versionMap. Latest version would then always refer to the latest *blessed* version.
If blessing levels were defined, versions could be categorized as:
#release
#beta
#alpha
#internal
Then you'd write code like:
(MooseVersionMap latestVersion: #internal) load
Dale