Metacello - more background

[Dale Henrichs]

When I started thinking about Metacello there were a number of things that I considered before even starting the design work:

- I wanted to handle common code and platform-specific code (common, squeak, pharo,
gemstone)
- I wanted to have a nice scripting API (DSL)
- I wanted to have named versions
- I didn't want to have to create a new file format
- I didn't want to have to create a new repository
- I didn't want to have to create a new tool

Platform-Specific code

I think it's obvious that I'd be interested in handling platform-specific code.
For nearly every project that I port to GemStone, I end up with varying amounts
of GemStone-specific code and that means that one or more of the Monticello
packages in a multi-package project end up having a GemStone-specific branch. So
any pacakge management system that I create will deal with platform-specific
issues up front.

With Pharo coming on the scene, I anticipate that there will be Pharo-specific
code written that will be ported back to Squeak and vice versa.

DSL

A clean API for scripts is essential, especially if you are going to forgo tools:)

Named Versions

When version '1.0' of a project is released, there needs to be an artifact named
'1.0' that I can manipulated within the context of the DSL. At a minimum, I
imagined that a list of Monticello files would be associated with a named version.

I didn't want...

A package management system has to have some kind of external file format that
can be shared with others through a repository, but I hated the idea of creating
a new type of repository to support a new file format. Requiring new repository
servers for Metacello would be a real barrier to acceptance.

I didn't relish the idea of inventing a new tool, either. Having a DSL
ameliorates the tool requirement somewhat, but doesn't solve the problem of
providing a mechanism for creating package management artifacts ... whatever they
might be.

MetacelloVersionMap

With the "didn't wants" dominating the stage, I began thinking that a class-based
approach might work out pretty well. A class-based approach would nail all three
"didn't wants" in one fell swoop, Monticello package for the file format,
Monticello repositories for the repository and code browsers for the tools ...
Extra credit for being able to create versions of the package management
information and view differences between versions...

So that's pretty much the box that I've built for Metacello:

Each project gets it's own subclass of MetacelloVersionMap: instance-side methods
define the package management structure and class-side methods provide access to
the named versions.

Enough blathering ... next up I'll finally get into some concrete examples.

Dale

[Bergel, Alexandre]

Hi Dale,

Thanks for your effort. Are you suggesting a list of convention to
build packages? I.e., the Mondrian package will contains a subclass
MetacelloVersionMap called MondrianMetacelloVersionMap that would
contain meta information.
In that case, all existing tools will be reusable.

I know that the Seaside crew has strong requirement on expressing
dependencies between packages. They are moving toward a Debian-like
system.

Cheers,
Alexandre

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

[Dale Henrichs]

----- "Alexandre Bergel" <ber...@iam.unibe.ch> wrote:

| Hi Dale,
|
| Thanks for your effort. Are you suggesting a list of convention to
| build packages? I.e., the Mondrian package will contains a subclass
| MetacelloVersionMap called MondrianMetacelloVersionMap that would
| contain meta information.
| In that case, all existing tools will be reusable.
|
| I know that the Seaside crew has strong requirement on expressing
| dependencies between packages. They are moving toward a Debian-like
| system.
|
| Cheers,
| Alexandre

Alexandre,

I actually think that the class MondrianMetacelloVersionMap with it's meta-information would be published in a separate Monticello package from the rest of Mondrian. That way the meta information that contains critical information about "how to load Mondrian" is itself loaded separately.

The point of being class-based is that it doesn't require new tools/file formats/repositories. The image itself becomes the package management system "installation data base".

I think that Metacello provides a solution for package to package dependencies that I've discussed in later messages and the Seaside proof of concept illustrates that Metacello can be used to load Seaside2.9 with it's current constraints and dependencies.

Dale

Dale

[Bergel, Alexandre]

> I actually think that the class MondrianMetacelloVersionMap with
> it's meta-information would be published in a separate Monticello
> package from the rest of Mondrian. That way the meta information
> that contains critical information about "how to load Mondrian" is
> itself loaded separately.

This is what we have in the ad hoc MondrianLoader and MooseLoader.
I will be glad to try out Metacello.

Cheers,
Alexandre

[Dale Henrichs]

----- "Alexandre Bergel" <ber...@iam.unibe.ch> wrote:

| > I actually think that the class MondrianMetacelloVersionMap with
| > it's meta-information would be published in a separate Monticello
| > package from the rest of Mondrian. That way the meta information
| > that contains critical information about "how to load Mondrian" is
|
| > itself loaded separately.
|
| This is what we have in the ad hoc MondrianLoader and MooseLoader.
| I will be glad to try out Metacello.

Alexandre,

I'd love to get more feedback on Metacello before I start the rewrite.

I've just run the unit tests against the latest version of Metacello (Metacello-dkh.147.mcz) and they're all green, so you shouldn't have too much trouble putting together a functional proof of concept.

WAVersionMap ('Seaside-VersionMap-dkh.7' in 'http://seaside.gemstone.com/ss/test') is the best _working_ example for defining packages and groups, but WAVersionMap doesn't have any project to project dependencies.

In the current code base, you need to supply the following pieces of information:

- name of the target versionMap class
- MetacelloPackageSpec to load the versionMap class
- package name
- filename
- repository and type
- target version
- name of the target feature to load (think group)

The following code will create a MetacelloRequiredFeatureSpec that specifies a dependency opn the 'Core' group of version '0.1' of the class named 'MetacelloMockVersionMapA', defined in the package MetacelloMockA-Version-dkh.21 in the repository http://seaside.gemstone.com/ss/MockMetacelloA:

| versionMap |
versionMap := MetacelloMCVersionMap new.
^(MetacelloRequiredFeatureSpec for: versionMap)
feature: 'Core'
requiredVersion:
((MetacelloMCRequiredVersionSpec for: versionMap)
versionMapClassName: 'MetacelloMockVersionMapA'
versionString: '0.1'
packages:
((MetacelloPackagesSpec for: versionMap)
addPackage: 'MetacelloMockA-Version'
file: 'MetacelloMockA-Version-dkh.21';
addRepository: 'http://seaside.gemstone.com/ss/MockMetacelloA'
type: 'http';
yourself);
yourself);
yourself.

You might recognize that the above code bears a relationship the code in 'Metacello - a little more DSL', but isn't quite as stream-lined.

In WAVersionMap you would use the above expression in place of 'MetacelloRequiredFeatureSpec for: self' in the *features methods.

If you execute:

MetacelloMockScripts loadMocksMeta

The two versionMaps MetacelloMockVersionMap and MetacelloMockVersionMapA will be loaded. Look at the method MetacelloMockVersionMap>>mockARequiredFeature:version: and you'll see some examples of how that code is used.

If you're going to write a versionMap (with the current code) here are some things to keep in mind:

- think group when you see feature:)
- use #resetInstance liberally to clear caches. currently the cached information
is way too sticky
- groups reference another group using addRequiredSpec: and package members are
defined using addPackageName:. You can't use group names and package names
interchangeably.

Define 'helper' methods like the methods #coreFeatureSpec:requiredVersion: and #coreTestsFeatureSpec:requiredVersion: in MetacelloMockVersionMap to reduce repetition.

I'll be online off and on over the three day weekend, so if you have questions or problems send me mail.

Dale

[Dale Henrichs]

One other thing. Beware that there is minimal loop detection logic for package and required version resolution, so if you do define a requirement loop, you may get into an infinite loop:) You can test your version map for loops by doing 'XXVersionMap instance map' and if execution is taking longer than a few seconds you've got a loop.

Dale

[Bergel, Alexandre]

Dear Dale,

Sorry for my late answer. Changing continent is sucking all my time
these days (looking for a flat, bad internet connection at the
hotel, ...). I downloaded Metacello and I am trying to use it for
Mondrian and Moose. I will come back to you this evening hopefully.

Cheers,
Alexandre

[Bergel, Alexandre]

Ok, I just gave a try. 

The last version of Mondrian requires an extension of SystemWindow and ScrollPane (Morphic Package) which is not part of Pharo yet. 

I defined two features, mondrian and windowNotification.

The code is below. Is it well defined? How can I use it to load the stuff?

I defined: 

MetacelloMCVersionMap subclass: #MondrianVersionMap

with the following methods:

label

^'Mondrian Tests'

loadMondrian

"self new loadMondrian "

(self version01Features latestVersion) load.

mondrian

| packagesSpec requiredVersion requiredFeature |

packagesSpec := 

(MetacelloPackagesSpec for: self)

addPackage: 'Mondrian' file: 'Mondrian';

addRepository: 'http://www.squeaksource.com/Mondrian' type: 'http';

yourself.

requiredVersion := 

(MetacelloMCRequiredVersionSpec for: self)

versionMapClassName: 'WhatShouldIPutHere?' 

versionString: '' 

packages: packagesSpec;

yourself.

requiredFeature := 

(MetacelloRequiredFeatureSpec for: self)

feature: '' requiredVersion: requiredVersion; 

yourself.

^requiredFeature

version01Features

<features: '0.1' attribute: #common >

^(MetacelloFeaturesSpec for: self)

addFeature: 

((MetacelloFeatureSpec for: self) 

featureName: 'core'

description: 'Code for loading Mondrian';

addPackageName: 'Mondrian';

addRequiredSpec: self windowNotification;

addRequiredSpec: self mondrian;

yourself);

yourself.

windowNotification

| packagesSpec requiredVersion requiredFeature |

packagesSpec := 

(MetacelloPackagesSpec for: self)

addPackage: 'Morphic' file: 'Morphic-Alexandre_Bergel.306.mcz';

addRepository: 'http://www.squeaksource.com/PharoInbox' type: 'http';

yourself.

requiredVersion := 

(MetacelloMCRequiredVersionSpec for: self)

versionMapClassName: 'WhatShouldIPutHere?' 

versionString: '' 

packages: packagesSpec;

yourself.

requiredFeature := 

(MetacelloRequiredFeatureSpec for: self)

feature: '' requiredVersion: requiredVersion; 

yourself.

^requiredFeature

How does this look? 

Alexandre

On 22 May 2009, at 18:24, Dale Henrichs wrote:

[Dale Henrichs]

Alexandre,

Close, but not quite. I've put a couple of versions of the ModrianVersionMap up on http://seaside.gemstone.com/ss/test:

MondrianVersionMap-dkh.1 - is for Pharo
MondrianVersionMap-dkh.2 - has an extension for Squeak (Announcements)

I changed, your #versionO1Features method and and added #version01Packages and moved the #loadMondrian method to the class side:

version01Features
<features: '0.1' attribute: #common >

^(MetacelloFeaturesSpec for: self)
addFeature:
((MetacelloFeatureSpec for: self)
featureName: 'core'

description: 'Code for Mondrian';
addPackageName: 'Mondrian';
addPackageName: 'Morphic';
yourself);
yourself.

version01Packages
<packages: '0.1' attribute: #common >

^(MetacelloPackagesSpec for: self)
addPackage: 'Morphic' file: 'Morphic-Alexandre_Bergel.306.mcz';
addPackage: 'Mondrian' file: 'Mondrian-Alexandre_Bergel.214';

addRepository: 'http://www.squeaksource.com/Mondrian' type: 'http';
yourself

MondrianVersionMap-dkh.1 is loadable on Pharo. MondrianVersionMap-dkh.2 is loadable on Squeak and Pharo (i.e., Announcements pacakge is only loaded on Squeak). I didn't want to complicate the example with Shout and OB dependencies, so they are assumed to be loaded.

Does this example make things clearer?

Dale

[Dale Henrichs]

Alexandre,

For extra credit, I added in an OmniBrowser dependency. It's actually added as a conditional dependency, so if OBNode exists, no OmniBrowser information is added to the versionMap (i.e., on Pharo, if you inspect the ModrianVersionMap for ModrianVersionMap-dkh.1, ModrianVersionMap-dkh.2, ModrianVersionMap-dkh.3, they are identical. if you look at the versionMap on a basic 3.10 squeak vm, the ModrianVersionMap-dkh.2 will have Announcements added as a package and ModrianVersionMap-dkh.3 will have Announcements added as a package and a required version added for OBVersionMap).

To see OmniBrowser loaded when Modrian is loaded start with a Squeak3.10-7159-basic image. Load in Metacello-dkh.148 (note the version number change - I fixed a bug for OmniBrowser loading) manually, then load MondrianVersionMap-dkh.3:

((MetacelloPackagesSpec for: self)
addPackage: 'MondrianVersionMap' file: 'MondrianVersionMap-dkh.3';
addRepository: 'http://seaside.gemstone.com/ss/test' type: 'http';
yourself) load

Then in a workspace evaluate:

ModrianVersionMap latestVersion load.

You should get OmniBrowser, Announcements and the two Modrian packages loaded.

----- "Dale Henrichs" <dale.h...@gemstone.com> wrote:

[Dale Henrichs]

The mail slipped out of my fingers before I could test the code. Here's the code that actually works:

((MetacelloPackagesSpec for: MetacelloMCVersionMap new)

addPackage: 'MondrianVersionMap' file: 'MondrianVersionMap-dkh.3';
addRepository: 'http://seaside.gemstone.com/ss/test' type: 'http';
yourself) load

Then in a workspace evaluate:

MondrianVersionMap latestVersion load.

Dale

[Bergel, Alexandre]

Hi Dale, 

sorry to be so slow to answer, but I am still looking for a flat.

The package Morphic-Alexandre_Bergel.306.mcz needs to be loaded from PharoInbox since it hasn't been included in Pharo yet. How this package can be loaded?

Your version01Packages seems to load a Morphic package from the Mondrian repository, which is not quite what I wish for:

version01Packages

<packages: '0.1' attribute: #common >

^(MetacelloPackagesSpec for: self)

addPackage: 'Morphic' file: 'Morphic-Alexandre_Bergel.306.mcz';

addPackage: 'Mondrian' file: 'Mondrian-Alexandre_Bergel.214';

addRepository: 'http://www.squeaksource.com/Mondrian' type: 'http';

yourself

Other thing, it seems that a old or basic OB browser is loaded. Best would be to have David's package browser.

Once I have a working loader, I will include it in the Mondrian distribution.

Cheers,

Alexandre

[Dale Henrichs]

Alexandre,

Two things about : Morphic-Alexandre_Bergel.306

1. Morphic-Alexandre_Bergel.306 is actually in http://www.squeaksource.com/Mondrian and http://www.squeaksource.com/PharoInbox, so technically loading it from http://www.squeaksource.com/Mondrian is fine.
2. To include PharoInbox as a repository add the repository:

version01Packages
<packages: '0.1' attribute: #common >

^(MetacelloPackagesSpec for: self)
addPackage: 'Morphic' file: 'Morphic-Alexandre_Bergel.306.mcz';
addPackage: 'Mondrian' file: 'Mondrian-Alexandre_Bergel.214';
addRepository: 'http://www.squeaksource.com/Mondrian' type: 'http';

addRepository: 'http://www.squeaksource.com/PharoInbox' type: 'http';
yourself

For the OB requirement, I included a minimal OB install that just loaded the OB-Standard. The OBVersionMap could be extended with package enhancements and refactoring browser definitions, then you can choose which OB group (standard, enhanced, etc.) you wanted loaded if OB isn't present.

Regarding using a Metacello versionMap for loading Mondrian, I would prefer that you wait a bit for the next rewrite - I will be renaming the Monticello package and including a versionMap for Metacello itself.

I am anxious to begin using Metacello myself and appreciate your enthusiasm.

[Bergel, Alexandre]

Hi Dale,

Again, sorry for my slow latency when answering.

> To see OmniBrowser loaded when Modrian is loaded start with a
> Squeak3.10-7159-basic image. Load in Metacello-dkh.148 (note the
> version number change - I fixed a bug for OmniBrowser loading)
> manually, then load MondrianVersionMap-dkh.3:
>
> ((MetacelloPackagesSpec for: self)
> addPackage: 'MondrianVersionMap' file: 'MondrianVersionMap-dkh.3';
> addRepository: 'http://seaside.gemstone.com/ss/test' type: 'http';
> yourself) load

I tried this with the last version of Metacello (#148), but I get a
roll back. versionMap in the MetacelloPackagesSpec seems to be nil.
However, what 'self' must be bound to?

Cheers,
Alexandre

[Bergel, Alexandre]

Sorry, do not bother answering my previous email

Alexandre

On 26 May 2009, at 18:27, Dale Henrichs wrote:

> The mail slipped out of my fingers before I could test the code.
> Here's the code that actually works:
>
> ((MetacelloPackagesSpec for: MetacelloMCVersionMap new)
> addPackage: 'MondrianVersionMap' file: 'MondrianVersionMap-dkh.3';
> addRepository: 'http://seaside.gemstone.com/ss/test' type: 'http';
> yourself) load
>
> Then in a workspace evaluate:
>
> MondrianVersionMap latestVersion load.

--